From oscar.j.benjamin at gmail.com Sat Feb 1 14:32:31 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sat, 1 Feb 2014 13:32:31 +0000 Subject: [Python-ideas] statistics module in Python3.4 In-Reply-To: References: <20140131010724.GE3799@ando> Message-ID: On 31 January 2014 03:47, Andrew Barnert wrote: > On Jan 30, 2014, at 17:32, Chris Angelico wrote: > >> On Fri, Jan 31, 2014 at 12:07 PM, Steven D'Aprano wrote: >>> One of my aims is to avoid raising TypeError unnecessarily. The >>> statistics module is aimed at casual users who may not understand, or >>> care about, the subtleties of numeric coercions, they just want to take >>> the average of two values regardless of what sort of number they are. >>> But having said that, I realise that mixed-type arithmetic is difficult, >>> and I've avoided documenting the fact that the module will work on mixed >>> types. >> >> Based on the current docs and common sense, I would expect that >> Fraction and Decimal should normally be there exclusively, and that >> the only type coercions would be int->float->complex (because it makes >> natural sense to write a list of "floats" as [1.4, 2, 3.7], but it >> doesn't make sense to write a list of Fractions as [Fraction(1,2), >> 7.8, Fraction(12,35)]). Any mishandling of Fraction or Decimal with >> the other three types can be answered with "Well, you should be using >> the same type everywhere". (Though it might be useful to allow >> int->anything coercion, since that one's easy and safe.) > > Except that large enough int values lose information, and even larger ones raise an exception: > > >>> float(pow(3, 50)) == pow(3, 50) > False > >>> float(1<<2000) > OverflowError: int too large to convert to float > > And that first one is the reason why statistics needs a custom sum in the first place. > > When there are only 2 types involved in the sequence, you get the answer you wanted. The only problem raised by the examples in this thread is that with 3 or more types that aren't all mutually coercible but do have a path through them, you can sometimes get imprecise answers and other times get exceptions, and you might come to rely on one or the other. > > So, rather than throwing out Stephen's carefully crafted and clearly worded rules and trying to come up with new ones, why not (for 3.4) just say that the order of coercions given values of 3 or more types is not documented and subject to change in the future (maybe even giving the examples from the initial email)? You're making this sound a lot more complicated than it is. The problem is simple: Decimal doesn't integrate with the numeric tower. This is explicit in the PEP that brought in the numeric tower: http://www.python.org/dev/peps/pep-3141/#the-decimal-type See also this thread (that I started during extensive off-list discussions about the statistics.sum function with Steven): https://mail.python.org/pipermail//python-ideas/2013-August/023034.html Decimal makes the following concessions for mixing numeric types: 1) It will promote integers in arithmetic. 2) It will compare correctly against all numeric types (as long as FloatOperation isn't trapped). 3) It will coerce int and float in its constructor. The recently added FloatOperation trap suggests that there's more interest in prohibiting the mixing of Decimals with other numeric types than facilitating it. I can imagine getting in that camp myself: speaking as someone who finds uses for both the fractions module and the decimal module I feel qualified to say that there is no good use case for mixing these types. Similarly there's no good use-case for mixing floats with Fractions or Decimals although mixing float/Fraction does work. If you choose to use Decimals then it is precisely because you do need to care about the numeric types you use and the sort of accuracy they provide. If you find yourself mixing Decimals with other numeric types then it's more likely a mistake/bug than a convenience. In any case the current implementation of statistics._sum (AIUI, I don't have it to hand for testing) will do the right thing for any mix of types in the numeric tower. It will also do the right thing for Decimals: it will compute the exact result and then round once according to the current decimal context. It's also possible to mix int and Decimal but there's no sensible way to handle mixing Decimal with anything else. If there is to be a documented limitation on mixing types then it should be explicitly about Decimal: The statistics module works very well with Decimal but doesn't really support mixing Decimal with other types. This is a limitation of Python rather than the statistics module itself. That being said I think that guaranteeing an error is better than the current order-dependent behaviour (and agree that that should be considered a bug). If there is to be a more drastic rearrangement of the _sum function then it should actually be to solve the problem that the current implementation of mean, variance etc. uses Fractions for all the heavy lifting but then rounds in the wrong place (when returning from _sum()) rather than in the mean, variance function itself. The clever algorithm in the variance function (unless it changed since I last looked) is entirely unnecessary when all of the intensive computation is performed with exact arithmetic. In the absence of rounding error you could compute a perfectly good variance using the computational formula for variance in a single pass. Similarly although the _sum() function is correctly rounded, the mean() function calls _sum() and then rounds again so that the return value from mean() is rounded twice. _sum() computes an exact value as a fraction and then coerces it with return T(total_numerator) / total_denominator so that the division causes it to be correctly rounded. However the mean function effectively ends up doing return (T(total_numerator) / total_denominator) / num_items which uses 2 divisions and hence rounds twice. It's trivial to rearrange that so that you round once return T(total_numerator) / (total_denominator * num_items) except that to do this the _sum function should be changed to return the exact result as a Fraction (and perhaps the type T). Similar changes would need to be made to the some of squares function (_ss() IIRC). The double rounding in mean() isn't a big deal but the corresponding effect for the variance functions is significant. It was after realising this that the sum function was renamed _sum and made nominally private. To be clear, statistics.variance(list_of_decimals) is very accurate. However it uses more passes than is necessary and it can be inaccurate in the situation that you have Decimals whose precision exceeds that of the current decimal context e.g.: >>> import decimal >>> d = decimal.Decimal('300000000000000000000000000000000000000000') >>> d Decimal('300000000000000000000000000000000000000000') >>> d+1 # Any arithmetic operation loses precision Decimal('3.000000000000000000000000000E+41') >>> +d # Use context precision Decimal('3.000000000000000000000000000E+41') If you're using Fractions for all of your computation then you can change this since no precision is lost when calling Fraction(Decimal): >>> import fractions >>> fractions.Fraction(d)+1 Fraction(300000000000000000000000000000000000000001, 1) Oscar From g.brandl at gmx.net Sat Feb 1 10:03:23 2014 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 01 Feb 2014 10:03:23 +0100 Subject: [Python-ideas] [off-topic] Insults, English, and Aspergers In-Reply-To: References: <23EC770C-0A37-4370-AD15-537069CC6C77@yahoo.com> <52EBCF49.6070302@stoneleaf.us> Message-ID: Am 31.01.2014 18:28, schrieb Mark Lawrence: > Yet again we're into the dual standards that annoy me so much, yet for > speaking my mind I'm the one in the wrong. Would you like to show me a recent post from Anatoly with the word "idiot" in it? Oh right, there isn't one, because he's moderated. Double standards? Georg From greg.ewing at canterbury.ac.nz Sat Feb 1 00:09:17 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 01 Feb 2014 12:09:17 +1300 Subject: [Python-ideas] statistics module in Python3.4 In-Reply-To: References: <52EAA20A.7090704@hastings.org> <20140131012705.GF3799@ando> <52EB2D6C.9040803@hastings.org> <20140131085610.GI3799@ando> Message-ID: <52EC2D1D.1060401@canterbury.ac.nz> Wolfgang Maier wrote: > Mappings may be an excellent way of specifying frequencies and weights in an > elegant way. That may be, but I think I'd rather have a separate function, or a mode argument, explicitly indicating that this is what you want. Detecting whether something is a mapping in a duck-typed way is dodgy in general. -- Greg From ncoghlan at gmail.com Sat Feb 1 10:07:03 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 1 Feb 2014 19:07:03 +1000 Subject: [Python-ideas] We don't know each other's challenges (was Re: Iterative development) Message-ID: On 1 February 2014 03:42, Chris Angelico wrote: > I wouldn't withdraw my comment, because I still stand by it. If you > genuinely meant no specifics, then when someone pointed out how they > interpreted your statement, you would have apologized and made a > correction: "I didn't mean anyone in particular, I meant the way > there've been 50 issues reopened unnecessarily by 30 different people > lately", or something. But that wouldn't be true, would it? You really > did mean Anatoly, and that's why you said what you did. Believe you > me, I know more than you think I do. Think of Emma from "Once Upon A > Time" if you like - a strong ability to detect lying, based on a > metric ton of experience with it. Chris, while Mark's behaviour has been out of line recently, that isn't anywhere near adequate justification for suggesting (even by implication) that another list participant is lying about their health status or their motives. It is impossible to diagnose *anyone* accurately over the internet - we can only give them the benefit of the doubt, take their word for it, and judge the outcome by whether they appear to be making genuine efforts to improve their behaviour, rather than assuming that everyone is starting from an identical baseline of expectations and capabilities in relation to civil discourse (especially once cultural variations are taken into account). Mark hasn't been trying to use his diagnosis as a get out of jail free card - he has been working with other members of the community on his coping strategies for dealing with mailing list discussions, and curbing his impulse to respond to poorly thought out ideas with unconstructive sarcasm. Now, I suggested to Mark that he consider asking the moderators to set his moderator flag for the time being, but he has instead chosen to step away from the core development lists entirely. While we *do* try to be inclusive of everyone, the thing that *will* get someone moderated, suspended and perhaps eventually banned entirely, is a consistent *pattern* of inappropriate behaviour, with no indication of genuine attempts to eliminate that behaviour (or even to understand why it is inappropriate). So if something seems out of line, *please* contact the list moderators (via python-ideas-owner at python.org), rather than retaliating directly on the list. If replying directly on the list, please try to assume temporary stress rather than persistent malice or obstinance on the part of the other poster in the absence of an extended history of interacting with them. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ben+python at benfinney.id.au Sat Feb 1 00:07:31 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 01 Feb 2014 10:07:31 +1100 Subject: [Python-ideas] Iterative development References: <23EC770C-0A37-4370-AD15-537069CC6C77@yahoo.com> Message-ID: <854n4jiwcs.fsf@benfinney.id.au> Mark Lawrence writes: > Asperger Syndrome sufferers are always honest. Sadly I find it a > major weakness that I have to live with. That's unfortunate. Please don't impose it on others too. -- \ ?When in doubt tell the truth. It will confound your enemies | `\ and astound your friends.? ?Mark Twain, _Following the Equator_ | _o__) | Ben Finney From rosuav at gmail.com Sat Feb 1 10:11:18 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 1 Feb 2014 20:11:18 +1100 Subject: [Python-ideas] We don't know each other's challenges (was Re: Iterative development) In-Reply-To: References: Message-ID: On Sat, Feb 1, 2014 at 8:07 PM, Nick Coghlan wrote: > Chris, while Mark's behaviour has been out of line recently, that > isn't anywhere near adequate justification for suggesting (even by > implication) that another list participant is lying about their health > status or their motives. Fair enough. I made the analysis that lying was a significantly more plausible explanation than honesty, and continued my thought process from that point, but you're right, I should have stopped posting on the subject rather sooner than I did. My apologies to the list. ChrisA From rosuav at gmail.com Sat Feb 1 18:50:10 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 2 Feb 2014 04:50:10 +1100 Subject: [Python-ideas] We don't know each other's challenges (was Re: Iterative development) In-Reply-To: References: Message-ID: On Sat, Feb 1, 2014 at 8:11 PM, Chris Angelico wrote: > On Sat, Feb 1, 2014 at 8:07 PM, Nick Coghlan wrote: >> Chris, while Mark's behaviour has been out of line recently, that >> isn't anywhere near adequate justification for suggesting (even by >> implication) that another list participant is lying about their health >> status or their motives. > > Fair enough. I made the analysis that lying was a significantly more > plausible explanation than honesty, and continued my thought process > from that point, but you're right, I should have stopped posting on > the subject rather sooner than I did. My apologies to the list. Hmm. Was I put onto moderation? I sent that shortly after Nick's post, and it's only just now shown up. If I haven't been, then Gmail is doing something screwy, I think. ChrisA From phd at phdru.name Sat Feb 1 18:57:30 2014 From: phd at phdru.name (Oleg Broytman) Date: Sat, 1 Feb 2014 18:57:30 +0100 Subject: [Python-ideas] mail problems (was: We don't know each other's challenges) In-Reply-To: References: Message-ID: <20140201175730.GA12731@phdru.name> On Sun, Feb 02, 2014 at 04:50:10AM +1100, Chris Angelico wrote: > Hmm. Was I put onto moderation? I sent that shortly after Nick's post, > and it's only just now shown up. If I haven't been, then Gmail is > doing something screwy, I think. It also could be a hiccup in greylisting at mail.python.org. For example, I seldom post to the python lists so greylisting at mail.p.o foregets me and every mail from me usually lie in the spool at my mail server for about 30 minutes. Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From ethan at stoneleaf.us Sat Feb 1 19:00:05 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 01 Feb 2014 10:00:05 -0800 Subject: [Python-ideas] We don't know each other's challenges (was Re: Iterative development) In-Reply-To: References: Message-ID: <52ED3625.9010404@stoneleaf.us> On 02/01/2014 09:50 AM, Chris Angelico wrote: > > Hmm. Was I put onto moderation? I sent that shortly after Nick's post, > and it's only just now shown up. If I haven't been, then Gmail is > doing something screwy, I think. The entire list was for a short time. It's been lifted. -- ~Ethan~ From wolfgang.maier at biologie.uni-freiburg.de Sat Feb 1 21:10:25 2014 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Sat, 1 Feb 2014 20:10:25 +0000 (UTC) Subject: [Python-ideas] statistics module in Python3.4 References: <52EAA20A.7090704@hastings.org> <20140131012705.GF3799@ando> <52EB2D6C.9040803@hastings.org> <20140131085610.GI3799@ando> <52EC2D1D.1060401@canterbury.ac.nz> Message-ID: Greg Ewing writes: > > Wolfgang Maier wrote: > > Mappings may be an excellent way of specifying frequencies and weights in an > > elegant way. > > That may be, but I think I'd rather have a separate > function, or a mode argument, explicitly indicating > that this is what you want. Detecting whether something > is a mapping in a duck-typed way is dodgy in general. > There should be nothing dodgy about this with the abstract base class Mapping. Best, Wolfgang From wolfgang.maier at biologie.uni-freiburg.de Sat Feb 1 21:47:14 2014 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Sat, 1 Feb 2014 21:47:14 +0100 Subject: [Python-ideas] statistics module in Python3.4 Message-ID: <000c01cf1f8e$ca1e1560$5e5a4020$@biologie.uni-freiburg.de> Oscar Benjamin writes: Hi Oscar, and thanks for this very detailed post. > > You're making this sound a lot more complicated than it is. The > problem is simple: Decimal doesn't integrate with the numeric tower. > This is explicit in the PEP that brought in the numeric tower: > http://www.python.org/dev/peps/pep-3141/#the-decimal-type > You're perfectly right about this as far as built-in number types and the standard library types Fraction and Decimal are concerned. > That being said I think that guaranteeing an error is > better than the current order-dependent behaviour (and agree that that > should be considered a bug). > For custom types, the type returned by _sum can also be order-dependent due to this part in _coerce-types: def _coerce_types(T1, T2): [..] if issubclass(T2, float): return T2 if issubclass(T1, float): return T1 # Subclasses of the same base class give priority to the second. if T1.__base__ is T2.__base__: return T2 I chose the more drastic example with Fraction and Decimal for my initial post because there the difference is between a result and an error, but the above may illustrate better why I said that the returned type of _sum is hard to predict. > If there is to be a more drastic rearrangement of the _sum function > then it should actually be to solve the problem that the current > implementation of mean, variance etc. uses Fractions for all the heavy > lifting but then rounds in the wrong place (when returning from > _sum()) rather than in the mean, variance function itself. > This is an excellent remark and I agree absolutely with your point here. It's one of the aspects of the statistics module that I pondered over for weeks. Essentially, the fact that all current functions that rely on _sum do round imprecisely anyway was my motivation for suggesting the simple: def _coerce_types (types): if len(types) == 1: return next(iter(types)) return float because it certainly makes sense to return the type found in the input if there is only one, but with ambiguity, why make the effort of guessing when it does not help precision anyway. However, I realized that I probably rushed this because the implementation of functions that call _sum may change later to rely on an exact return value. > The clever algorithm in the variance function (unless it changed since > I last looked) is entirely unnecessary when all of the intensive > computation is performed with exact arithmetic. In the absence of > rounding error you could compute a perfectly good variance using the > computational formula for variance in a single pass. Similarly > although the _sum() function is correctly rounded, the mean() function > calls _sum() and then rounds again so that the return value from > mean() is rounded twice. _sum() computes an exact value as a fraction > and then coerces it with > > return T(total_numerator) / total_denominator > > so that the division causes it to be correctly rounded. However the > mean function effectively ends up doing > > return (T(total_numerator) / total_denominator) / num_items > > which uses 2 divisions and hence rounds twice. It's trivial to > rearrange that so that you round once > > return T(total_numerator) / (total_denominator * num_items) > > except that to do this the _sum function should be changed to return > the exact result as a Fraction (and perhaps the type T). Similar > changes would need to be made to the some of squares function (_ss() > IIRC). The double rounding in mean() isn't a big deal but the > corresponding effect for the variance functions is significant. It was > after realising this that the sum function was renamed _sum and made > nominally private. > I have been thinking about this solution as well, but I think you really have to return a tuple of the sum as a Fraction and the type (not perhaps) since it would be really weird if the public functions in statistics always return a Fraction even if the input sequence consisted of only one standard type like int, float or Decimal. The obvious criticism then is that such a _sum is not really a sum function anymore like the existing ones. Then again, since this is a module private function it may be ok to do this? Best, Wolfgang From oscar.j.benjamin at gmail.com Sat Feb 1 21:54:40 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sat, 1 Feb 2014 20:54:40 +0000 Subject: [Python-ideas] statistics module in Python3.4 In-Reply-To: References: <52EAA20A.7090704@hastings.org> <20140131012705.GF3799@ando> <52EB2D6C.9040803@hastings.org> <20140131085610.GI3799@ando> <52EC2D1D.1060401@canterbury.ac.nz> Message-ID: On 1 February 2014 20:10, Wolfgang Maier wrote: > Greg Ewing writes: > >> >> Wolfgang Maier wrote: >> > Mappings may be an excellent way of specifying frequencies and weights in an >> > elegant way. >> >> That may be, but I think I'd rather have a separate >> function, or a mode argument, explicitly indicating >> that this is what you want. Detecting whether something >> is a mapping in a duck-typed way is dodgy in general. >> > > There should be nothing dodgy about this with the abstract base class Mapping. I agree with Greg about this. I dislike APIs that try to be too clever about accepting different types of input. The mode() function is clearly intended to accept an iterable not a Counter and I consider it a bug that it does. It can be fixed to treat a Counter as an iterable by changing the _counts function to do collections.Counter(data) to collections.Counter(iter(data)) If you want the option for doing statistics with Counter style data formats then they should be invoked explicitly as Greg says. Oscar From wolfgang.maier at biologie.uni-freiburg.de Sat Feb 1 22:24:58 2014 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Sat, 1 Feb 2014 22:24:58 +0100 Subject: [Python-ideas] statistics module in Python3.4 Message-ID: <000e01cf1f94$0f5eddd0$2e1c9970$@biologie.uni-freiburg.de> Oscar Benjamin writes: > > On 1 February 2014 20:10, Wolfgang Maier > wrote: > > Greg Ewing ...> writes: > > > >> > >> Wolfgang Maier wrote: > >> > Mappings may be an excellent way of specifying frequencies and weights in an > >> > elegant way. > >> > >> That may be, but I think I'd rather have a separate > >> function, or a mode argument, explicitly indicating > >> that this is what you want. Detecting whether something > >> is a mapping in a duck-typed way is dodgy in general. > >> > > > > There should be nothing dodgy about this with the abstract base class Mapping. > > I agree with Greg about this. I dislike APIs that try to be too clever > about accepting different types of input. The mode() function is > clearly intended to accept an iterable not a Counter and I consider it > a bug that it does. It can be fixed to treat a Counter as an iterable > by changing the _counts function to do > > collections.Counter(data) > > to > > collections.Counter(iter(data)) > > If you want the option for doing statistics with Counter style data > formats then they should be invoked explicitly as Greg says. > I would accept this as a bug-fix, but I do not agree with you and Greg about an API trying to be too clever here. A Mapping is a clearly defined term and if the module doc stated that for Mappings passed to functions in statistics their values will be interpreted as weights/frequencies of the corresponding keys that should be clear enough. If what you really want is to do statistics just on the keys, you can easily pass just these (e.g., mean(mydict.keys()). On the other hand, separate functions would complicate the API because you would end up with a weighted counterpart for almost every function in the module. The alternative of passing weights in the form of a second sequence sounds more attractive, but I guess both specifications could coexist peacefully (just like there are several ways of constructing a dictionary). Then if you have data and weights in a Mapping already, you can just go ahead and use it, and likewise, if you have them in two separate sequences. Best, Wolfgang From wolfgang.maier at biologie.uni-freiburg.de Sat Feb 1 23:00:36 2014 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Sat, 1 Feb 2014 22:00:36 +0000 (UTC) Subject: [Python-ideas] statistics module in Python3.4 References: <000e01cf1f94$0f5eddd0$2e1c9970$@biologie.uni-freiburg.de> Message-ID: Wolfgang Maier writes: > > Oscar Benjamin ...> writes: > > > > On 1 February 2014 20:10, Wolfgang Maier > > ...> wrote: > > > Greg Ewing ...> writes: > > > > > >> > > >> Wolfgang Maier wrote: > > >> > Mappings may be an excellent way of specifying frequencies and > weights in an > > >> > elegant way. > > >> > > >> That may be, but I think I'd rather have a separate > > >> function, or a mode argument, explicitly indicating > > >> that this is what you want. Detecting whether something > > >> is a mapping in a duck-typed way is dodgy in general. > > >> > > > > > > There should be nothing dodgy about this with the abstract base class > Mapping. > > > > I agree with Greg about this. I dislike APIs that try to be too clever > > about accepting different types of input. It may also help to address this from the users' perspective. What possible other use-cases could there be to pass a Mapping (let alone a Counter) to one of the functions in statistics? Best, Wolfgang From p.f.moore at gmail.com Sun Feb 2 00:13:55 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 1 Feb 2014 23:13:55 +0000 Subject: [Python-ideas] statistics module in Python3.4 In-Reply-To: References: <000e01cf1f94$0f5eddd0$2e1c9970$@biologie.uni-freiburg.de> Message-ID: On 1 February 2014 22:00, Wolfgang Maier wrote: > It may also help to address this from the users' perspective. What possible > other use-cases could there be to pass a Mapping (let alone a Counter) to > one of the functions in statistics? Why not just pass counter.elements() to the functions if you want to use a counter as a frequency table? Maybe you're arguing that Mappings should be rejected with an exception? But that seems like an unnecessary restriction just to catch the mistaken usage of forgetting to call elements() on a Counter. Paul From wolfgang.maier at biologie.uni-freiburg.de Sun Feb 2 00:24:11 2014 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Sat, 1 Feb 2014 23:24:11 +0000 (UTC) Subject: [Python-ideas] statistics module in Python3.4 References: <000e01cf1f94$0f5eddd0$2e1c9970$@biologie.uni-freiburg.de> Message-ID: > -----Urspr?ngliche Nachricht----- > Von: Paul Moore [mailto:p.f.moore at gmail.com] > Gesendet: Sonntag, 2. Februar 2014 00:14 > An: Wolfgang Maier > Cc: Python-Ideas > Betreff: Re: [Python-ideas] statistics module in Python3.4 > > On 1 February 2014 22:00, Wolfgang Maier freiburg.de> wrote: > > It may also help to address this from the users' perspective. What > > possible other use-cases could there be to pass a Mapping (let alone a > > Counter) to one of the functions in statistics? > > Why not just pass counter.elements() to the functions if you want to use a > counter as a frequency table? > The difference is that by accepting a Counter directly functions like statistics._sum and mode can do their job faster and more space- efficient. Using counter.elements means exploding the data structure, then summing up (for _sum) all the values, while you could just sum all key*value of the Counter. > Maybe you're arguing that Mappings should be rejected with an exception? > But that seems like an unnecessary restriction just to catch the mistaken > usage of forgetting to call elements() on a Counter. > I don't know if they should be rejected, an explicit warning in the docs that their treatment may be subject to change in Python3.5 may also be an option. Best, Wolfgang From ncoghlan at gmail.com Sun Feb 2 02:24:59 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 2 Feb 2014 11:24:59 +1000 Subject: [Python-ideas] statistics module in Python3.4 In-Reply-To: References: <000e01cf1f94$0f5eddd0$2e1c9970$@biologie.uni-freiburg.de> Message-ID: On 2 February 2014 09:24, Wolfgang Maier wrote: >> -----Urspr?ngliche Nachricht----- >> Von: Paul Moore [mailto:p.f.moore at gmail.com] >> Gesendet: Sonntag, 2. Februar 2014 00:14 >> An: Wolfgang Maier >> Cc: Python-Ideas >> Betreff: Re: [Python-ideas] statistics module in Python3.4 >> >> On 1 February 2014 22:00, Wolfgang Maier > freiburg.de> wrote: >> > It may also help to address this from the users' perspective. What >> > possible other use-cases could there be to pass a Mapping (let alone a >> > Counter) to one of the functions in statistics? >> >> Why not just pass counter.elements() to the functions if you want to use a >> counter as a frequency table? >> > > The difference is that by accepting a Counter directly functions like > statistics._sum and mode can do their job faster and more space- > efficient. Using counter.elements means exploding the data structure, > then summing up (for _sum) all the values, while you could just sum > all key*value of the Counter. First make it correct, *then* make it fast. I'm inclined to favour the approach of forcing the "iter()" call here, so Counter is always treated the same as any other iterable. There's a plausible case to be made for offering a more efficient alternative API for working directly with collections.Counter objects in the statistics module, but that should be considered specifically for 3.5, rather than relying on an implicit detail of the current implementation. For a historical precedent, compare the printf-style formatting API, which changes behaviour based on whether the RHS is a tuple, dict or other type, and the newer format()/format_map() APIs where the special case to handle passing in an existing dict without going through **kwargs expansion was moved out to a suitably named dedicated method. I have now filed two related issues for this: 3.4 release blocker to avoid inadvertently special casing Counter: http://bugs.python.org/issue20478 3.5 RFE to provide tools to work directly with weight/frequency mappings: http://bugs.python.org/issue20479 Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Sun Feb 2 02:35:46 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 2 Feb 2014 11:35:46 +1000 Subject: [Python-ideas] statistics module in Python3.4 In-Reply-To: References: <20140131010724.GE3799@ando> Message-ID: On 1 February 2014 23:32, Oscar Benjamin wrote: > You're making this sound a lot more complicated than it is. The > problem is simple: Decimal doesn't integrate with the numeric tower. > This is explicit in the PEP that brought in the numeric tower: > http://www.python.org/dev/peps/pep-3141/#the-decimal-type http://bugs.python.org/issue20481 now covers the concerns over avoiding making any guarantees that the current type coercion behaviour of the statistics module will be preserved indefinitely (it includes a link back to the archived copy of Oscar's post on mail.python.org). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From wolfgang.maier at biologie.uni-freiburg.de Sun Feb 2 08:49:23 2014 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Sun, 2 Feb 2014 07:49:23 +0000 (UTC) Subject: [Python-ideas] statistics module in Python3.4 References: <20140131010724.GE3799@ando> Message-ID: Nick Coghlan writes: > > On 1 February 2014 23:32, Oscar Benjamin > wrote: > > You're making this sound a lot more complicated than it is. The > > problem is simple: Decimal doesn't integrate with the numeric tower. > > This is explicit in the PEP that brought in the numeric tower: > > http://www.python.org/dev/peps/pep-3141/#the-decimal-type > > http://bugs.python.org/issue20481 now covers the concerns over > avoiding making any guarantees that the current type coercion > behaviour of the statistics module will be preserved indefinitely (it > includes a link back to the archived copy of Oscar's post on > mail.python.org). > > Cheers, > Nick. > Thanks a lot, Nick, for all your efforts in filing the bugs. I just added a possible patch for http://bugs.python.org/issue20481 to the bug tracker. Best, Wolfgang From ceronman at gmail.com Mon Feb 3 21:27:08 2014 From: ceronman at gmail.com (=?ISO-8859-1?Q?Manuel_Cer=F3n?=) Date: Mon, 3 Feb 2014 21:27:08 +0100 Subject: [Python-ideas] Unify global and nonlocal Message-ID: Hello, The 'global' and 'nonlocal' statements serve very similar purposes. They both allow to rebind names in an outer scope. While 'nonlocal' allows to rebind a name in the closest outer scope, *except* the global module scope, 'global' is used for the later case. Is there a reason why 'nonlocal' has to stop at the scope immediately after the global module scope? If 'nonlocal' could continue until the global scope, then it would become (almost) a generalized case of 'global'. Having both statements goes against the "There should be one - and preferably only one - obvious way to do it" principle. It's also confusing for beginners and people used to a functional style. I think 'nonlocal' should be changed to support all the enclosing scopes, including the global module scope, and the 'global' statement should be marked as deprecated. There is one specific use case that breaks though, which is rebinding to global names from a deep function skipping names in the middle, for example: x = 'global' def outer1(): x = 'outer1' def outer2(): global x x = 'outer2' # binds the global module name But you know "special cases aren't special enough to break the rules". My apologizes if this topic was already discussed in the past. Manuel. From amber.yust at gmail.com Mon Feb 3 21:40:08 2014 From: amber.yust at gmail.com (Amber Yust) Date: Mon, 03 Feb 2014 20:40:08 +0000 Subject: [Python-ideas] Unify global and nonlocal References: Message-ID: <-5515647872496007270@gmail297201516> global and nonlocal are different specifically because they do different things. Yes, there is a reason why nonlocal stops at the first scope up - because that's what it exists to do. On Mon Feb 03 2014 at 12:28:42 PM, Manuel Cer?n wrote: > Hello, > > The 'global' and 'nonlocal' statements serve very similar purposes. > They both allow to rebind names in an outer scope. While 'nonlocal' > allows to rebind a name in the closest outer scope, *except* the > global module scope, 'global' is used for the later case. > > Is there a reason why 'nonlocal' has to stop at the scope immediately > after the global module scope? If 'nonlocal' could continue until the > global scope, then it would become (almost) a generalized case of > 'global'. Having both statements goes against the "There should be one > - and preferably only one - obvious way to do it" principle. It's > also confusing for beginners and people used to a functional style. > > I think 'nonlocal' should be changed to support all the enclosing > scopes, including the global module scope, and the 'global' statement > should be marked as deprecated. > > There is one specific use case that breaks though, which is rebinding > to global names from a deep function skipping names in the middle, for > example: > > x = 'global' > def outer1(): > x = 'outer1' > def outer2(): > global x > x = 'outer2' # binds the global module name > > But you know "special cases aren't special enough to break the rules". > > My apologizes if this topic was already discussed in the past. > > Manuel. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From skip at pobox.com Mon Feb 3 21:33:10 2014 From: skip at pobox.com (Skip Montanaro) Date: Mon, 3 Feb 2014 14:33:10 -0600 Subject: [Python-ideas] Unify global and nonlocal In-Reply-To: References: Message-ID: Manuel, You might want to read over PEP 3104: http://www.python.org/dev/peps/pep-3104/ especially the section about backward compatibility. Skip From abarnert at yahoo.com Mon Feb 3 21:47:56 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Mon, 3 Feb 2014 12:47:56 -0800 Subject: [Python-ideas] Unify global and nonlocal In-Reply-To: References: Message-ID: <53D0BF19-70A6-44BB-9D5E-A34B8ADC453F@yahoo.com> On Feb 3, 2014, at 12:27, Manuel Cer?n wrote: > Is there a reason why 'nonlocal' has to stop at the scope immediately > after the global module scope? If 'nonlocal' could continue until the > global scope, then it would become (almost) a generalized case of > 'global'. Having both statements goes against the "There should be one > - and preferably only one - obvious way to do it" principle. It's > also confusing for beginners and people used to a functional style. In my experience dealing with people coming to python from functional languages, they generally want closure variables to be as restricted as possible, and it's actually the fact that you can use a global as a closure variable _at all_ that they find confusing, not the fact that the syntax and semantics are a bit different than for a local closure variable. In fact, there was a question on StackOverflow a few weeks ago from someone who wanted to know how this could possibly work: a = [] def spam(): return a[0] a.append(1) print spam.__closure__ print spam() (Yes, it was Python 2, and it's an implicit rather than explicit global... But same idea.) I don't think this change would help people with that kind of confusion at all. That being said, I don't think it would _hurt_ them, and it would probably help novices learning the concept of closures for the first time, in Python. And it could make some kinds of refactoring easier to mechanize. From mistersheik at gmail.com Mon Feb 3 23:11:43 2014 From: mistersheik at gmail.com (Neil Girdhar) Date: Mon, 3 Feb 2014 14:11:43 -0800 (PST) Subject: [Python-ideas] str.rreplace In-Reply-To: <20140128123349.GH3915@ando> References: <20140124183633.60f215f6@fsol> <20140124192021.7dcc1c77@fsol> <1AF1E6EA-17FF-4FA1-8582-9365B22E4714@yahoo.com> <1390613768.85265.YahooMailNeo@web181006.mail.ne1.yahoo.com> <1390881834.74969.YahooMailNeo@web181003.mail.ne1.yahoo.com> <20140128123349.GH3915@ando> Message-ID: Isn't the only reason that you don't like keyword arguments that take constant values a matter of efficiency? It would be much better to make the compiler smarter than to make the language more verbose. The advantage of keyword arguments is that you never end up with a = x.listrip(...) if blah else x.rstrip(...) On Tuesday, January 28, 2014 7:33:50 AM UTC-5, Steven D'Aprano wrote: > > On Mon, Jan 27, 2014 at 08:03:54PM -0800, Andrew Barnert wrote: > > From: Ron Adam > > > > > How about a keyword to specify which end to index from? > > -1 > > As a general rule, when you have a function that takes a parameter which > selects between two different sets of behaviour, and you normally > specify that parameter as a literal or constant known at edit time, then > the function should be split into two. > > E.g.: > > # Good API > string.upper(), string.lower() > > # Bad API > string.convert_case(to_upper=True|False) > > sorted() and list.sort() (for example) are a counter-example. Sometimes > you know which direction you want at edit-time, but there are many > use-cases for leaving the decision to run-time. Nearly every application > that sorts data lets the user decide which direction to sort. > > In the case of replace/rreplace, it is more like the upper vs. lower > situation than the sorted situation. For almost any reasonable use-case, > you will know at edit-time whether you want to go from the left or from > the right, so you'll specify the "direction" parameter as a edit-time > literal or constant. The same applies to find/rfind. > > > > > When used, it would > > > disable negative indexing as well. > > -1 > > Negative indexing is a standard Python feature. There is nothing wrong > with negative indexing, no more than there is something wrong with > zero-based positive indexing. > > It's also irrelevant to the replace/rreplace example, since replace > doesn't take start/end indexes, and presumably rreplace wouldn't either. > > > > > When not used the current behaviour with > > > negative indexing would be the default. > > > > > > > > direction=0 # The default with the current > > > (or not specified) # negative indexing allowed. > > > > > > direction=1 # From first. Negative indexing disallowed. > > > direction=-1 # From last. Negative indexing disallowed. > > And if you want to operate from the right, with negative indexing > allowed? But really, having a flag to decide whether to allow negative > indexing is silly. If you don't want negative indexes, just don't use > them. > > > > > (A shorter key word would be nice, but I can't think of any that is as > > > clear.) > > > > Why does it have to be -1/0/1 instead of just True/False? > > > > In which case we could use "reverse", the same name that's already > > used for similar things in other methods like list.sort (and that's > > implied in the current names "rfind", etc.). > > sorted(alist, reverse=True) gives the same result as sorted(alist, > reverse=False) only reversed. That is not the case here: > > "Hello world".replace("o", "u", 1, reverse=True) # rreplace > > ought to return "Hello wurld", not "dlrow ulleH". > > > > > The reason for turning off the negative indexing is it would also > offer a way to > > > > > avoid some indexing bugs as well. (Using negative indexing with a > reversed > > > index is just asking for trouble I think.) > > > > But str.rfind takes negative indices today: > > > > >>> 'abccba'.rfind('b', -5, -3) > > 1 > > > > Why take away functionality that already works? > > Exactly. Here, I agree strongly with Andrew. Negative indexing works > perfectly well with find/rfind. Slices with negative strides are weird, > but negative indexes are well-defined and easy to understand. > > > > And of course str.find takes negative indices and that's actually used > > in some quick&dirty scripts: > > > > >>> has_ext = path.find('.', -4) > > > > Of course you could make an argument that any such scripts deserve to > > be broken? > > It would be an awfully bogus argument. Negative indexes are a > well-defined part of Python indexing semantics. One might as well argue > that any scripts that rely on list slicing making a copy "deserve to be > broken". > > > -- > Steven > _______________________________________________ > Python-ideas mailing list > Python... at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Tue Feb 4 00:30:23 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Mon, 3 Feb 2014 15:30:23 -0800 Subject: [Python-ideas] str.rreplace In-Reply-To: References: <20140124183633.60f215f6@fsol> <20140124192021.7dcc1c77@fsol> <1AF1E6EA-17FF-4FA1-8582-9365B22E4714@yahoo.com> <1390613768.85265.YahooMailNeo@web181006.mail.ne1.yahoo.com> <1390881834.74969.YahooMailNeo@web181003.mail.ne1.yahoo.com> <20140128123349.GH3915@ando> Message-ID: On Feb 3, 2014, at 14:11, Neil Girdhar wrote: > Isn't the only reason that you don't like keyword arguments that take constant values a matter of efficiency? I'm pretty sure it's about clarity--letting you write what you mean, and making that meaning explicit at a glance--than performance. (On the rare occasions where performance makes a difference, surely you're already copying the method to a local variable anyway, right?) Keyword arguments imply a dynamic choice, different functions a static one, in basically the same way as, say, keyed values vs. attributes. So, if it would be very rare to choose between foo and rfoo dynamically, it makes sense for them to be separate methods; if it's relatively common, it makes sense to have a single method. (I'm not sure that it _would_ be rare, but if so, I agree with the rest of the argument against my point.) From steve at pearwood.info Tue Feb 4 02:23:52 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 4 Feb 2014 12:23:52 +1100 Subject: [Python-ideas] Unify global and nonlocal In-Reply-To: <-5515647872496007270@gmail297201516> References: <-5515647872496007270@gmail297201516> Message-ID: <20140204012350.GU3799@ando> On Mon, Feb 03, 2014 at 08:40:08PM +0000, Amber Yust wrote: > global and nonlocal are different specifically because they do different > things. Yes, there is a reason why nonlocal stops at the first scope up - > because that's what it exists to do. +1 "There should be one obvious way to do it" doesn't apply here, because there is no *it*, there is *them*. Two distinct things to do, rather than one: - enable writing to the top-level global namespace; - enable writing to some enclosing non-local scope. I say *some enclosing* scope rather than *the enclosing scope* because you can have more than one. The nonlocal keyword enables writing to the nearest enclosing scope which already contains the given name, not necessarily the closest enclosing scope: py> def outer(): ... a = "outer" ... def inner(): ... def innermost(): ... nonlocal a ... a = "modified" ... innermost() ... print("Before:", a) ... inner() ... print("After:", a) ... py> outer() Before: outer After: modified The global namespace is special, and special enough to justify being handled specially. Of course, it is possible to generalise the idea that globals are "just" a case of lexical scoping, and there is some truth to that. But it's an over-generalisation, a bit like deciding that since the built-in len() function is just a many-to-one transformation of arbitrary sequences to integers, it belongs as a codec: import codecs length_of_list = codecs.encode(my_list, "len") In the case of globals and non-locals: * Non-locals always have an enclosing function, globals do not. * Non-locals and globals are typically used for different purposes, by users at different levels of expertise. Writing to a non-local scope using the nonlocal keyword tends to be done only by advanced users for advanced purposes; writing to global variables using the global keyword tends to be done only by beginners who haven't yet learned that global variables are considered harmful. * A corrolary of the above is that it is difficult to think of any use-cases where you don't know at edit-time whether you intend to write to a variable in the global scope or a non-local scope. I can't think of any case where I wouldn't know whether to declare something global or nonlocal ahead of time, so there's no advantage to having a single keyword handle both cases. -- Steven From mistersheik at gmail.com Tue Feb 4 04:15:25 2014 From: mistersheik at gmail.com (Neil Girdhar) Date: Mon, 3 Feb 2014 22:15:25 -0500 Subject: [Python-ideas] str.rreplace In-Reply-To: References: <20140124183633.60f215f6@fsol> <20140124192021.7dcc1c77@fsol> <1AF1E6EA-17FF-4FA1-8582-9365B22E4714@yahoo.com> <1390613768.85265.YahooMailNeo@web181006.mail.ne1.yahoo.com> <1390881834.74969.YahooMailNeo@web181003.mail.ne1.yahoo.com> <20140128123349.GH3915@ando> Message-ID: I agree with everything you're saying. Another reason to prefer keyword arguments is that they simplify the interface and make it easier to learn. On Mon, Feb 3, 2014 at 6:30 PM, Andrew Barnert wrote: > On Feb 3, 2014, at 14:11, Neil Girdhar wrote: > > > Isn't the only reason that you don't like keyword arguments that take > constant values a matter of efficiency? > > I'm pretty sure it's about clarity--letting you write what you mean, and > making that meaning explicit at a glance--than performance. (On the rare > occasions where performance makes a difference, surely you're already > copying the method to a local variable anyway, right?) > > Keyword arguments imply a dynamic choice, different functions a static > one, in basically the same way as, say, keyed values vs. attributes. So, if > it would be very rare to choose between foo and rfoo dynamically, it makes > sense for them to be separate methods; if it's relatively common, it makes > sense to have a single method. > > (I'm not sure that it _would_ be rare, but if so, I agree with the rest of > the argument against my point.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ram.rachum at gmail.com Tue Feb 4 11:45:51 2014 From: ram.rachum at gmail.com (Ram Rachum) Date: Tue, 4 Feb 2014 02:45:51 -0800 (PST) Subject: [Python-ideas] Make all builtin functions accept None for optional arguments Message-ID: <5902a367-e65f-44e2-a57d-7d8eb908eab4@googlegroups.com> Here is something that always annoys me. I was going to write my own rreplace function, like this: def rreplace(s, old, new, count=None): return new.join(s.rsplit(old, count)) But lo and behold, I have to write it like this: def rreplace(s, old, new, count=None): return new.join(s.rsplit(old, count) if count is not None else s.rsplit(old)) Why? Because the `str.rsplit` can't handle a count of `None`. That is quite annoying. There are many more builtin functions except `str.rsplit` that behave like this. What do you think about going over all such functions and making them able to accept `None`? Thanks, Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Tue Feb 4 15:54:06 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 04 Feb 2014 23:54:06 +0900 Subject: [Python-ideas] str.rreplace In-Reply-To: References: <20140124183633.60f215f6@fsol> <20140124192021.7dcc1c77@fsol> <1AF1E6EA-17FF-4FA1-8582-9365B22E4714@yahoo.com> <1390613768.85265.YahooMailNeo@web181006.mail.ne1.yahoo.com> <1390881834.74969.YahooMailNeo@web181003.mail.ne1.yahoo.com> <20140128123349.GH3915@ando> Message-ID: <87fvnz53ox.fsf@uwakimon.sk.tsukuba.ac.jp> Neil Girdhar writes: > I agree with everything you're saying. ?Another reason to prefer > keyword arguments is that they simplify the interface and make it > easier to learn. No, they simplify the *default* interface and make *that* easier to learn. Who could be against that? The issue is "what about the case where there's no TOOWTDI default?" Where TOOWTDI is somewhere around "more than 80% of the time for more than 80% of the users when they want a specific value for the parameter every time on this code path" (vs. choosing that value at runtime). From python at mrabarnett.plus.com Tue Feb 4 16:20:52 2014 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 04 Feb 2014 15:20:52 +0000 Subject: [Python-ideas] Make all builtin functions accept None for optional arguments In-Reply-To: <5902a367-e65f-44e2-a57d-7d8eb908eab4@googlegroups.com> References: <5902a367-e65f-44e2-a57d-7d8eb908eab4@googlegroups.com> Message-ID: <52F10554.5090905@mrabarnett.plus.com> On 2014-02-04 10:45, Ram Rachum wrote: > Here is something that always annoys me. > > I was going to write my own rreplace function, like this: > > def rreplace(s, old, new, count=None): > return new.join(s.rsplit(old, count)) > > > But lo and behold, I have to write it like this: > > def rreplace(s, old, new, count=None): > return new.join(s.rsplit(old, count) if count is not None > else s.rsplit(old)) > > > Why? Because the `str.rsplit` can't handle a count of `None`. That is > quite annoying. > > There are many more builtin functions except `str.rsplit` that behave > like this. > > What do you think about going over all such functions and making them > able to accept `None`? > It accepts -1: def rreplace(s, old, new, count=-1): return new.join(s.rsplit(old, count)) From ncoghlan at gmail.com Tue Feb 4 16:24:28 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 5 Feb 2014 01:24:28 +1000 Subject: [Python-ideas] Unify global and nonlocal In-Reply-To: <20140204012350.GU3799@ando> References: <-5515647872496007270@gmail297201516> <20140204012350.GU3799@ando> Message-ID: On 4 February 2014 11:23, Steven D'Aprano wrote: > In the case of globals and non-locals: > > * Non-locals always have an enclosing function, globals do not. > > * Non-locals and globals are typically used for different purposes, > by users at different levels of expertise. Writing to a non-local > scope using the nonlocal keyword tends to be done only by advanced > users for advanced purposes; writing to global variables using the > global keyword tends to be done only by beginners who haven't yet > learned that global variables are considered harmful. > > * A corrolary of the above is that it is difficult to think of any > use-cases where you don't know at edit-time whether you intend to > write to a variable in the global scope or a non-local scope. I can't > think of any case where I wouldn't know whether to declare something > global or nonlocal ahead of time, so there's no advantage to having a > single keyword handle both cases. There's also a critical operational distinction: 'nonlocal' demands that it be able to find the name at compile time, while 'global' will happily create a *new* name binding at module global scope. This difference in intent allows an appropriate error message to be generated when nonlocal is used incorrectly: >>> def outer(): ... a_name = 1 ... def inner(): ... nonlocal a_nme ... a_name += 2 ... return a_name ... return inner ... SyntaxError: no binding for nonlocal 'a_nme' found By contrast, global will happily bind new names, because the compiler makes no assumptions about what names may eventually show up at module global scope at run time: >>> def f(): ... global a_name ... a_name += 2 ... return a_name ... >>> f() Traceback (most recent call last): File "", line 1, in File "", line 3, in f NameError: global name 'a_name' is not defined >>> a_name = 1 >>> f() 3 While the difference isn't often obvious, function namespaces are actually *very* special beasts in Python, compared to the relatively straightforward behaviour of module and the default class namespaces as key:value mappings. This is reflected both in the additional restrictions that apply to function namespaces (like not allowing star imports in Python 3), as well as in the fact that they are accessed via nonlocal, while the module namespace is accessed (for rebinding purposes) via the global statement. It's also why mutating the mapping returning by globals() is fully supported, while mutating the result of locals() is explicitly declared as having undefined behaviour (it may or may not affect the current scope either temporarily or permanently, depending on implementation details). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Tue Feb 4 16:26:37 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 5 Feb 2014 01:26:37 +1000 Subject: [Python-ideas] Make all builtin functions accept None for optional arguments In-Reply-To: <52F10554.5090905@mrabarnett.plus.com> References: <5902a367-e65f-44e2-a57d-7d8eb908eab4@googlegroups.com> <52F10554.5090905@mrabarnett.plus.com> Message-ID: Yes, this is a consequence of having to currently write the argument parsing code for builtins by hand (and a lot of those implementations predating the evolution of more modern conventions). It's likely to change in the future, as Argument Clinic makes such things easier to maintain and make consistent. Cheers, Nick. From jsbueno at python.org.br Tue Feb 4 17:19:34 2014 From: jsbueno at python.org.br (Joao S. O. Bueno) Date: Tue, 4 Feb 2014 14:19:34 -0200 Subject: [Python-ideas] str.rreplace In-Reply-To: References: <73e21a44-d667-4430-b06e-06dde692a3df@googlegroups.com> <20140124175645.66bb8daf@fsol> <20140124183633.60f215f6@fsol> <20140124192021.7dcc1c77@fsol> <1AF1E6EA-17FF-4FA1-8582-9365B22E4714@yahoo.com> <1390613768.85265.YahooMailNeo@web181006.mail.ne1.yahoo.com> Message-ID: On 28 January 2014 01:18, Ron Adam wrote: > > > On 01/24/2014 07:36 PM, Andrew Barnert wrote: >>>>>> >>>>>> While we're speculatively overgeneralizing, couldn't all of the >>>>>> index/find/remove/replace/etc. methods take a negative n to >>>>>> count from the end, making r variants unnecessary? >>>> >>>> Strings already provide rfind and rindex (they're just not part of >>>> the general sequence API). Since strings are immutable, there's also >>>> no call for an "remove". > > >> I was responding to Serhiy's (probably facetious or devil's advocate) >> suggestion that we should regularize the API: add rfind and rindex to >> tuple (and presumably Sequence), and those plus rremove to list (and >> presumably MutableSequence), and so on. >> >> My point was that if we're going to be that radical, we might as well >> consider removing methods instead of adding them. Some of the find-like >> methods already take negative indices; expanding that to all of the >> index-based methods, and doing the equivalent to the count-based ones, >> and adding a count or index to those that have neither, would mean all >> of the "r" variants could go away. > > > > How about a keyword to specify which end to index from? When used, it would > disable negative indexing as well. When not used the current behaviour with > negative indexing would be the default. > > direction=0 # The default with the current > (or not specified) # negative indexing allowed. > > direction=1 # From first. Negative indexing disallowed. > direction=-1 # From last. Negative indexing disallowed. > > (A shorter key word would be nice, but I can't think of any that is as > clear.) I've just picked the whole thread at once - and I am a little surprised no one suggested what looks obvious to me (or maybe someone did, I went over the e-mails rather quickly): Why not simply to allow negative indexes to the count argument? It is pretty much unambiguous, straightforward (hmm..actually, the opposite of that) and Pythonistas are used to think of negative indices as counting from the right. Moreover, the convention could be used for index, remove and even overloaded for split, and other methods as well. js -><- From python at mrabarnett.plus.com Tue Feb 4 19:14:33 2014 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 04 Feb 2014 18:14:33 +0000 Subject: [Python-ideas] str.rreplace In-Reply-To: References: <73e21a44-d667-4430-b06e-06dde692a3df@googlegroups.com> <20140124175645.66bb8daf@fsol> <20140124183633.60f215f6@fsol> <20140124192021.7dcc1c77@fsol> <1AF1E6EA-17FF-4FA1-8582-9365B22E4714@yahoo.com> <1390613768.85265.YahooMailNeo@web181006.mail.ne1.yahoo.com> Message-ID: <52F12E09.3060005@mrabarnett.plus.com> On 2014-02-04 16:19, Joao S. O. Bueno wrote: > On 28 January 2014 01:18, Ron Adam wrote: >> >> >> On 01/24/2014 07:36 PM, Andrew Barnert wrote: >>>>>>> >>>>>>> While we're speculatively overgeneralizing, couldn't all of the >>>>>>> index/find/remove/replace/etc. methods take a negative n to >>>>>>> count from the end, making r variants unnecessary? >>>>> >>>>> Strings already provide rfind and rindex (they're just not part of >>>>> the general sequence API). Since strings are immutable, there's also >>>>> no call for an "remove". >> >> >>> I was responding to Serhiy's (probably facetious or devil's advocate) >>> suggestion that we should regularize the API: add rfind and rindex to >>> tuple (and presumably Sequence), and those plus rremove to list (and >>> presumably MutableSequence), and so on. >>> >>> My point was that if we're going to be that radical, we might as well >>> consider removing methods instead of adding them. Some of the find-like >>> methods already take negative indices; expanding that to all of the >>> index-based methods, and doing the equivalent to the count-based ones, >>> and adding a count or index to those that have neither, would mean all >>> of the "r" variants could go away. >> >> >> >> How about a keyword to specify which end to index from? When used, it would >> disable negative indexing as well. When not used the current behaviour with >> negative indexing would be the default. >> >> direction=0 # The default with the current >> (or not specified) # negative indexing allowed. >> >> direction=1 # From first. Negative indexing disallowed. >> direction=-1 # From last. Negative indexing disallowed. >> >> (A shorter key word would be nice, but I can't think of any that is as >> clear.) > > > I've just picked the whole thread at once - > and I am a little surprised no one suggested what looks obvious to me > (or maybe someone did, I went over the e-mails > rather quickly): > > Why not simply to allow negative indexes to the count argument? > > It is pretty much unambiguous, straightforward (hmm..actually, the > opposite of that) and Pythonistas are used to think of negative > indices as counting from the right. Moreover, the convention could be > used for index, remove and even overloaded for split, and other > methods as well. > .split, .rsplit and .replace all currently use -1 to indicate no maximum. Oh, and .index and .rindex don't have a count argument! :-) From tjreedy at udel.edu Wed Feb 5 03:17:54 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 04 Feb 2014 21:17:54 -0500 Subject: [Python-ideas] Make all builtin functions accept None for optional arguments In-Reply-To: <5902a367-e65f-44e2-a57d-7d8eb908eab4@googlegroups.com> References: <5902a367-e65f-44e2-a57d-7d8eb908eab4@googlegroups.com> Message-ID: On 2/4/2014 5:45 AM, Ram Rachum wrote: > Here is something that always annoys me. > > I was going to write my own rreplace function, like this: > > def rreplace(s, old, new, count=None): > return new.join(s.rsplit(old, count)) > > > But lo and behold, I have to write it like this: > > def rreplace(s, old, new, count=None): > return new.join(s.rsplit(old, count) if count is not None > else s.rsplit(old)) > > > Why? Because the `str.rsplit` can't handle a count of `None`. That is > quite annoying. As MRAB pointed out, count does have a default >>> help(str.rsplit) Help on method_descriptor: rsplit(...) S.rsplit(sep=None, maxsplit=-1) -> list of strings But there are builtins with optional parameters with no default, and they *are* annoying because they do need conditional code like the above to not pass any value. Some developers would like to add None as default for such cases. For reasons explained by Nick, at least some will in 3.5. Replacing (or augmenting) -1 with None when -1 means None is trickier because of back compatibility. Another annoyance for builtins is the lack of documentation as to which parameters can only be passed by position and not by keyword. You cannot be sure from just reading the above that ''.rsplit(maxsplit = 3) is even legal. This should also be fixed in 3.5. -- Terry Jan Reedy From lemiant at hotmail.com Wed Feb 5 15:32:50 2014 From: lemiant at hotmail.com (Alex Rodrigues) Date: Wed, 5 Feb 2014 07:32:50 -0700 Subject: [Python-ideas] Inline Functions - idea Message-ID: Hi everyone, This is my first time on the Python mailing lists. I've been learning a lot about how python is run recently and today I thought I ran across an idea which might create in interesting discussion.Today I was writing a piece of software where I had a block of code that would take a bunch of local variables, apply some transformations to them and then output them as a a string to a log. Then I realized that I actually wanted to reuse this code in multiple locations - there were multiple cases where I might need to do this. My natural inclination was to write a function in order to maintain DRY programming. This was prohibitively challenging, however, since the code itself interacted with lots of variables in the namespace. The number of arguments to the function would have to be very large and possibly would change on a fairly regular basis.This seems like a fairly common problem in programming, having a piece of code which is both reused and heavily integrated with the namespace making it necessary to use copy-paste. As a solution to this I propose the idea of an inline function. An inline function would run in it's parent's namespace instead of creating a new one. This would allow you to avoid passing back and forth tons of values while still maintaining DRY code. It might look something like this if implemented in a log for a traction control system: # First without an inline functiondef main(): file = open('file.txt') counter = 0 while True: counter += 1 frontL, frontR, backL, backR = getWheelSpeeds() if counter > 100: # Log at least every 10 seconds slipL = abs(frontL - backL) slipR = abs(frontR - backR) file.write('Speeds: ('+frontL+', '+frontR+', '+backL+', '+backR+'), \n Slip: '+slipL+', '+slipR) counter = 0 elif abs(frontR-backR) > 1 or abs(frontL-backL) > 1: # Also log if the wheels are slipping slipL = abs(frontL - backL) slipR = abs(frontR - backR) file.write('Speeds: ('+frontL+', '+frontR+', '+backL+', '+backR+'), \n Slip: '+slipL+', '+slipR) counter = 0 elif average([frontL, frontR, backL, backR]) > 60: # Also log if we're going really fast slipL = abs(frontL - backL) slipR = abs(frontR - backR) file.write('Speeds: ('+frontL+', '+frontR+', '+backL+', '+backR+'), \n Slip: '+slipL+', '+slipR) counter = 0 time.sleep(.1) # And now with an inline functiondef main(): file = open('file.txt') counter = 0 while True: counter += 1 frontL, frontR, backL, backR = getWheelSpeeds() if counter > 100: # Log every 10 seconds no matter what saveLine() elif abs(frontR-backR) > 1 or abs(frontL-backL) > 1: # Also log if the wheels are slipping saveLine() elif average([frontL, frontR, backL, backR]) > 60: # Also log if we're going really fast saveLine() time.sleep(.1) inline def saveLine(): slipL = abs(frontL - backL) slipR = abs(frontR - backR) file.write('Speeds: ('+frontL+', '+frontR+', '+backL+', '+backR+'), \n Slip: '+slipL+', '+slipR) counter = 0 What do you think? - Alex -------------- next part -------------- An HTML attachment was scrubbed... URL: From allan.clark at gmail.com Wed Feb 5 15:44:36 2014 From: allan.clark at gmail.com (Allan Clark) Date: Wed, 5 Feb 2014 14:44:36 +0000 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: Why not just use 'or' in your conditional? # First without an inline function def main(): file = open('file.txt') counter = 0 while True: counter += 1 frontL, frontR, backL, backR = getWheelSpeeds() if ( counter > 100 or # Log at least every 10 seconds abs(frontR-backR) > 1 or abs(frontL-backL) > 1 or # Also log if the wheels are slipping average([frontL, frontR, backL, backR]) > 60): # Also log if we're going really fast slipL = abs(frontL - backL) slipR = abs(frontR - backR) file.write('Speeds: ('+frontL+', '+frontR+', '+backL+', '+backR+'), \n Slip: '+slipL+', '+slipR) counter = 0 time.sleep(.1) On 5 February 2014 14:32, Alex Rodrigues wrote: > Hi everyone, > > This is my first time on the Python mailing lists. I've been learning a > lot about how python is run recently and today I thought I ran across an > idea which might create in interesting discussion. > Today I was writing a piece of software where I had a block of code that > would take a bunch of local variables, apply some transformations to them > and then output them as a a string to a log. Then I realized that I > actually wanted to reuse this code in multiple locations - there were > multiple cases where I might need to do this. My natural inclination was to > write a function in order to maintain DRY programming. This was > prohibitively challenging, however, since the code itself interacted with > lots of variables in the namespace. The number of arguments to the function > would have to be very large and possibly would change on a fairly regular > basis. > This seems like a fairly common problem in programming, having a piece of > code which is both reused and heavily integrated with the namespace making > it necessary to use copy-paste. As a solution to this I propose the idea > of an inline function. An inline function would run in it's parent's > namespace instead of creating a new one. This would allow you to avoid > passing back and forth tons of values while still maintaining DRY code. It > might look something like this if implemented in a log for a traction > control system: > > # First without an inline function > def main(): > file = open('file.txt') > counter = 0 > while True: > counter += 1 > frontL, frontR, backL, backR = getWheelSpeeds() > if counter > 100: # Log at least every 10 seconds > slipL = abs(frontL - backL) > slipR = abs(frontR - backR) > file.write('Speeds: ('+frontL+', '+frontR+', '+backL+', > '+backR+'), \n Slip: '+slipL+', '+slipR) > counter = 0 > elif abs(frontR-backR) > 1 or abs(frontL-backL) > 1: # Also log > if the wheels are slipping > slipL = abs(frontL - backL) > slipR = abs(frontR - backR) > file.write('Speeds: ('+frontL+', '+frontR+', '+backL+', > '+backR+'), \n Slip: '+slipL+', '+slipR) > counter = 0 > elif average([frontL, frontR, backL, backR]) > 60: # Also log if > we're going really fast > slipL = abs(frontL - backL) > slipR = abs(frontR - backR) > file.write('Speeds: ('+frontL+', '+frontR+', '+backL+', > '+backR+'), \n Slip: '+slipL+', '+slipR) > counter = 0 > time.sleep(.1) > > > # And now with an inline function > def main(): > file = open('file.txt') > counter = 0 > while True: > counter += 1 > frontL, frontR, backL, backR = getWheelSpeeds() > if counter > 100: # Log every 10 seconds no matter what > saveLine() > elif abs(frontR-backR) > 1 or abs(frontL-backL) > 1: # Also log > if the wheels are slipping > saveLine() > elif average([frontL, frontR, backL, backR]) > 60: # Also log if > we're going really fast > saveLine() > time.sleep(.1) > > inline def saveLine(): > slipL = abs(frontL - backL) > slipR = abs(frontR - backR) > file.write('Speeds: ('+frontL+', '+frontR+', '+backL+', '+backR+'), > \n Slip: '+slipL+', '+slipR) > counter = 0 > > What do you think? > > - Alex > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Wed Feb 5 15:45:10 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 6 Feb 2014 01:45:10 +1100 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: On Thu, Feb 6, 2014 at 1:32 AM, Alex Rodrigues wrote: > An inline function would run in it's parent's namespace instead of creating > a new one. Does it have to be able to be called from different parents and use their different namespaces, or is it local to one specific parent function? Your example is the latter, and Python already supports this: closures. def main(): file = open('file.txt') counter = 0 def saveLine(): nonlocal counter slipL = abs(frontL - backL) slipR = abs(frontR - backR) file.write('Speeds: ('+frontL+', '+frontR+', '+backL+', '+backR+'), \n Slip: '+slipL+', '+slipR) counter = 0 while True: counter += 1 frontL, frontR, backL, backR = getWheelSpeeds() if counter > 100: # Log every 10 seconds no matter what saveLine() elif abs(frontR-backR) > 1 or abs(frontL-backL) > 1: # Also log if the wheels are slipping saveLine() elif average([frontL, frontR, backL, backR]) > 60: # Also log if we're going really fast saveLine() time.sleep(.1) It'll happily read anything in the parent's namespace (whether they're globals or locals in the parent), which includes calling methods (like your file.write() there); to change anything, you just have to declare it as 'nonlocal', as you see with counter. (The way I've written it, slipL and slipR are local to saveLine(). You could alternatively declare them to be nonlocal too, but there doesn't seem to be any need to, here.) Though in this case, you have three 'if' branches that do exactly the same thing. It might be easier to do your conditions differently, such that there really is just one inline block of code. But in a more complex scenario, you could perhaps parameterize the inner function, or call it in several different contexts, etcetera, etcetera, etcetera. It's a function, like any other, and it cheerfully references the parent's namespace. Note though that it's the parent by the function's definition, not by where it's called. That's why I dropped the question at the top. In your example it makes no difference, of course. ChrisA From oscar.j.benjamin at gmail.com Wed Feb 5 15:47:07 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 5 Feb 2014 14:47:07 +0000 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: On 5 February 2014 14:32, Alex Rodrigues wrote: > > # First without an inline function > def main(): > file = open('file.txt') > counter = 0 > while True: > counter += 1 > frontL, frontR, backL, backR = getWheelSpeeds() > if counter > 100: # Log at least every 10 seconds > slipL = abs(frontL - backL) > slipR = abs(frontR - backR) > file.write('Speeds: ('+frontL+', '+frontR+', '+backL+', > '+backR+'), \n Slip: '+slipL+', '+slipR) > counter = 0 > elif abs(frontR-backR) > 1 or abs(frontL-backL) > 1: # Also log if > the wheels are slipping > slipL = abs(frontL - backL) > slipR = abs(frontR - backR) > file.write('Speeds: ('+frontL+', '+frontR+', '+backL+', > '+backR+'), \n Slip: '+slipL+', '+slipR) > counter = 0 > elif average([frontL, frontR, backL, backR]) > 60: # Also log if > we're going really fast > slipL = abs(frontL - backL) > slipR = abs(frontR - backR) > file.write('Speeds: ('+frontL+', '+frontR+', '+backL+', > '+backR+'), \n Slip: '+slipL+', '+slipR) > counter = 0 > time.sleep(.1) > def main(): file = open('file.txt') counter = 0 while True: counter += 1 frontL, frontR, backL, backR = getWheelSpeeds() slipL = abs(frontL - backL) slipR = abs(frontR - backR) if (counter > 100 or abs(frontR-backR) > 1 or abs(frontL-backL) > 1 or average([frontL, frontR, backL, backR]) > 60): counter = 0 file.write('Speeds: ('+frontL+', '+frontR+', '+backL+', '+backR+'), \n Slip: '+slipL+', '+slipR) time.sleep(.1) Oscar From skip at pobox.com Wed Feb 5 15:49:34 2014 From: skip at pobox.com (Skip Montanaro) Date: Wed, 5 Feb 2014 08:49:34 -0600 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: I'm not convinced this particular example needs an inline function. Just write saveLine() as a normal function and pass in your locals: def f(): a = 1 b = 7 compute(**locals()) def g(): a = 3 b = 0.9 compute(**locals()) def compute(**args): print args["a"] + args["b"] f() g() That's not to say inline functions might not be handy. It's possible that you can implement them without modifying the language either. A peephole optimizer could inline small functions which are used a lot. Skip From lemiant at hotmail.com Wed Feb 5 16:15:08 2014 From: lemiant at hotmail.com (Alex Rodrigues) Date: Wed, 5 Feb 2014 08:15:08 -0700 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: , Message-ID: Hi guys, A couple more thoughtsAs to why I didn't use an "or" in the original example. I am aware that the example doesn't actually need an inline function, but it demonstrates the concept in a simple way (so that it would fit in an email). Most use-cases are more complicated, often the spots where you want to execute the same code are buried inside long, branching logic statements where other things are going on, making restructuring the code much less trivial, if not impossible.I had never really considered using closures that way (I haven't really used them much at all), that is extremely cool! It does exactly what I was envisioning as long as you only need it in one spot. If inline function got implemented I'm sure they would steal a lot of code from closures.The remaining use cases for inline functions are times when you would need the same capabilities in multiple functions (calling in multiple methods of a class for example), or when you wish to define what is essentially a closure at the first level of execution, since closures are not possible when you are not in a function.I always knew you could do it the way skip showed, but I feel like that is a bit clunky. This would allow you to do that in a more straight-forward way. -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Wed Feb 5 16:21:35 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 5 Feb 2014 15:21:35 +0000 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: On 5 February 2014 15:15, Alex Rodrigues wrote: > As to why I didn't use an "or" in the original example. I am aware that the > example doesn't actually need an inline function, but it demonstrates the > concept in a simple way (so that it would fit in an email). Most use-cases > are more complicated, often the spots where you want to execute the same > code are buried inside long, branching logic statements where other things > are going on, making restructuring the code much less trivial, if not > impossible. That sounds to me like your function is so complex that it should probably have been refactored long before you got to the point of needing your inline function. Possibly by factoring out the conditional testing to a separate function. Of course, that sort of thing is always easier said than done, I know :-) Paul From ned at nedbatchelder.com Wed Feb 5 16:22:01 2014 From: ned at nedbatchelder.com (Ned Batchelder) Date: Wed, 05 Feb 2014 10:22:01 -0500 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: <52F25719.7010500@nedbatchelder.com> On 2/5/14 9:32 AM, Alex Rodrigues wrote: > Hi everyone, > > This is my first time on the Python mailing lists. I've been learning > a lot about how python is run recently and today I thought I ran > across an idea which might create in interesting discussion. > Today I was writing a piece of software where I had a block of code > that would take a bunch of local variables, apply some transformations > to them and then output them as a a string to a log. Then I realized > that I actually wanted to reuse this code in multiple locations - > there were multiple cases where I might need to do this. My natural > inclination was to write a function in order to maintain DRY > programming. This was prohibitively challenging, however, since the > code itself interacted with lots of variables in the namespace. The > number of arguments to the function would have to be very large and > possibly would change on a fairly regular basis. > This seems like a fairly common problem in programming, having a piece > of code which is both reused and heavily integrated with the namespace > making it necessary to use copy-paste. As a solution to this I propose > the idea of an inline function. An inline function would run in it's > parent's namespace instead of creating a new one. This would allow you > to avoid passing back and forth tons of values while still maintaining > DRY code. It might look something like this if implemented in a log > for a traction control system: > > # First without an inline function > def main(): > file = open('file.txt') > counter = 0 > while True: > counter += 1 > frontL, frontR, backL, backR = getWheelSpeeds() > if counter > 100: # Log at least every 10 seconds > slipL = abs(frontL - backL) > slipR = abs(frontR - backR) > file.write('Speeds: ('+frontL+', '+frontR+', '+backL+', '+backR+'), \n > Slip: '+slipL+', '+slipR) > counter = 0 > elif abs(frontR-backR) > 1 or abs(frontL-backL) > 1: # Also log if > the wheels are slipping > slipL = abs(frontL - backL) > slipR = abs(frontR - backR) > file.write('Speeds: ('+frontL+', '+frontR+', '+backL+', '+backR+'), \n > Slip: '+slipL+', '+slipR) > counter = 0 > elif average([frontL, frontR, backL, backR]) > 60: # Also log if > we're going really fast > slipL = abs(frontL - backL) > slipR = abs(frontR - backR) > file.write('Speeds: ('+frontL+', '+frontR+', '+backL+', '+backR+'), \n > Slip: '+slipL+', '+slipR) > counter = 0 > time.sleep(.1) > > > # And now with an inline function > def main(): > file = open('file.txt') > counter = 0 > while True: > counter += 1 > frontL, frontR, backL, backR = getWheelSpeeds() > if counter > 100: # Log every 10 seconds no matter what > saveLine() > elif abs(frontR-backR) > 1 or abs(frontL-backL) > 1: # Also log if > the wheels are slipping > saveLine() > elif average([frontL, frontR, backL, backR]) > 60: # Also log if > we're going really fast > saveLine() > time.sleep(.1) > > inline def saveLine(): > slipL = abs(frontL - backL) > slipR = abs(frontR - backR) > file.write('Speeds: ('+frontL+', '+frontR+', '+backL+', > '+backR+'), \n Slip: '+slipL+', '+slipR) > counter = 0 > > What do you think? > In Python 3: def main(): def saveLine(): nonlocal counter slipL = abs(frontL - backL) slipR = abs(frontR - backR) file.write('Speeds: ('+frontL+', '+frontR+', '+backL+', '+backR+'), \n Slip: '+slipL+', '+slipR) counter = 0 file = open('file.txt') counter = 0 while True: counter += 1 frontL, frontR, backL, backR = getWheelSpeeds() if counter > 100: # Log every 10 seconds no matter what saveLine() elif abs(frontR-backR) > 1 or abs(frontL-backL) > 1: # Also log if the wheels are slipping saveLine() elif average([frontL, frontR, backL, backR]) > 60: # Also log if we're going really fast saveLine() time.sleep(.1) > - Alex > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Wed Feb 5 16:28:48 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 6 Feb 2014 02:28:48 +1100 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: On Thu, Feb 6, 2014 at 2:15 AM, Alex Rodrigues wrote: > I had never really considered using closures that way (I haven't really used > them much at all), that is extremely cool! It does exactly what I was > envisioning as long as you only need it in one spot. If inline function got > implemented I'm sure they would steal a lot of code from closures. Or, maintain compatibility with older versions of Python by simply using closures :) Remember, something discussed here has to then be coded, be accepted into trunk, be tested, and eventually be distributed in a new version. You can't possibly get something like this added to Python 3.3 or earlier; 3.4 is already too close to add something like this to; so you'd be looking at 3.5 at the absolute soonest. Maybe that's not a problem, if your code's for you and nobody else (you can run a Python alpha or even a modified Python with additional code in it), but if you want to distribute your code and have people install Python and run your program, you'd be waiting an absolute minimum of 2-3 years for this to happen. And the consequence of that is that a proposed new feature really has to be all that awesome; it has to be worth waiting several years for (and probably putting in some coding work to create, unless you can excite someone else enough to have him/her write it for you). That's a pretty steep hump to get over. > The remaining use cases for inline functions are times when you would need > the same capabilities in multiple functions (calling in multiple methods of > a class for example), or when you wish to define what is essentially a > closure at the first level of execution, since closures are not possible > when you are not in a function. The nearest concept I can think of to your proposed inline functions is a preprocessor macro. Effectively, you create a simple token that expands to a block of source code, and it's handled prior to the parsing and compilation of the code. There are times when that sort of thing is really useful, but it's oh so easy to make horribly unreadable code. (Plus, it'd be hard to get indentation right, if the different contexts are indented to different levels.) There's usually some other way to do what you want. Python's pretty amazingly dynamic (often at the cost of performance); offer a specific use-case (maybe on python-list rather than here) and there's a reasonable chance someone can suggest a way to refactor it. Post actual working code (in the verbose form), highlighting the duplicated bits, and see what people offer. You'd be amazed what the language already has! > I always knew you could do it the way skip showed, but I feel like that is a > bit clunky. This would allow you to do that in a more straight-forward way. Yes, that proposal is VERY clunky. Passing locals() around is almost always a baaaad idea :) ChrisA From ericsnowcurrently at gmail.com Wed Feb 5 16:57:35 2014 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Wed, 5 Feb 2014 08:57:35 -0700 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: On Feb 5, 2014 7:34 AM, "Alex Rodrigues" wrote: > > Hi everyone, > > This is my first time on the Python mailing lists. Welcome! > I've been learning a lot about how python is run recently and today I thought I ran across an idea which might create in interesting discussion. > Today I was writing a piece of software where I had a block of code that would take a bunch of local variables, apply some transformations to them and then output them as a a string to a log. Then I realized that I actually wanted to reuse this code in multiple locations - there were multiple cases where I might need to do this. My natural inclination was to write a function in order to maintain DRY programming. This was prohibitively challenging, however, since the code itself interacted with lots of variables in the namespace. The number of arguments to the function would have to be very large and possibly would change on a fairly regular basis. > This seems like a fairly common problem in programming, having a piece of code which is both reused and heavily integrated with the namespace making it necessary to use copy-paste. As a solution to this I propose the idea of an inline function. An inline function would run in it's parent's namespace instead of creating a new one. This sounds like one aspect of Ruby's blocks. Nick Coghlan (one of the more active Python committers) wrote up a nice block post that describes them: http://www.curiousefficiency.org/posts/2011/10/correcting-ignorance-learning-bit-about.html > What do you think? You've described an interesting idea. I think Paul is right that it's a bit of a code smell that the function is so complex as to make "inlining" desirable. Yet, it seems like there is a practical idea hiding in there. Keep at the core idea and see if anything more focused, including a better use case, pops out. To be honest the end result will likely be no changes to Python (see this list's archives and the list of deferred/rejected PEPs). However, you and others will probably learn a thing or two in the process. Unlike some traffic on this list, your idea and the discussion of it are well within the realm of not wasting people's time. -eric [1] Though not exactly applicable to your "inline" functions, aother Ruby blocks-like idea is statement local namespaces. See PEPs 403 and 3150 (at http://www.python.org/dev/peps/). From mertz at gnosis.cx Wed Feb 5 19:17:09 2014 From: mertz at gnosis.cx (David Mertz) Date: Wed, 5 Feb 2014 10:17:09 -0800 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: It's hard for me to see the use case for this "inline function" idea. The example given is just some code that is repeated within branches of an 'if'; but the obvious thing is to move that repetition outside the if/elif clauses. I understand that the example is simplified to present it, but it's hard for me easily to imagine a case where either using a closure or passing in 'locals()' directly fails to cover everything you are asking for. I mean, you did say that passing locals() is "clunky", but it's hard to see how any function that dealt with the availability of different variables on an ad hoc basis could really be non-clunky (and that what every inline function would have to do). On Wed, Feb 5, 2014 at 7:15 AM, Alex Rodrigues wrote: > Hi guys, > > A couple more thoughts > > 1. As to why I didn't use an "or" in the original example. I am aware > that the example doesn't actually need an inline function, but it > demonstrates the concept in a simple way (so that it would fit in an > email). Most use-cases are more complicated, often the spots where you want > to execute the same code are buried inside long, branching logic statements > where other things are going on, making restructuring the code much less > trivial, if not impossible. > 2. I had never really considered using closures that way (I haven't > really used them much at all), that is extremely cool! It does exactly what > I was envisioning as long as you only need it in one spot. If inline > function got implemented I'm sure they would steal a lot of code from > closures. > - The remaining use cases for inline functions are times when you > would need the same capabilities in multiple functions (calling in > multiple methods of a class for example), or when you wish to define what > is essentially a closure at the first level of execution, since closures > are not possible when you are not in a function. > 3. I always knew you could do it the way skip showed, but I feel like > that is a bit clunky. This would allow you to do that in a more > straight-forward way. > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: From yoavglazner at gmail.com Wed Feb 5 19:38:35 2014 From: yoavglazner at gmail.com (yoav glazner) Date: Wed, 5 Feb 2014 20:38:35 +0200 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: On Feb 5, 2014 8:18 PM, "David Mertz" wrote: > > It's hard for me to see the use case for this "inline function" idea. The example given is just some code that is repeated within branches of an 'if'; but the obvious thing is to move that repetition outside the if/elif clauses. > > I understand that the example is simplified to present it, but it's hard for me easily to imagine a case where either using a closure or passing in 'locals()' directly fails to cover everything you are asking for. I mean, you did say that passing locals() is "clunky", but it's hard to see how any function that dealt with the availability of different variables on an ad hoc basis could really be non-clunky (and that what every inline function would have to do) How about: Def somefun(): a = 5 b = 7 return fmt('{a} is not {b}') inline def fmt(fmtstr): return fmtstr.format_map(locals()) -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemiant at hotmail.com Wed Feb 5 20:48:52 2014 From: lemiant at hotmail.com (Alex Rodrigues) Date: Wed, 5 Feb 2014 12:48:52 -0700 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: , Message-ID: I'm not under any illusions about how hard it is to actually get something added to standard python . But I thought this would be a fun and informative undertaking that possibly even helps people. Thank you for your kind words. After some consideration I have come up with another use case that demonstrates a little more of how this might be useful:Let's say that I am a weather forecaster and I maintain a large library of different simulation techniques. All the techniques need to get their initial data from some database. Let's take a look at some way of writing this code and their advantages/disadvantages:Hard code the startup code into each simulation - This works, but it will lead to lots of code re-use and all the problems that come with that (i.e. if I later need to change how the startup values are downloaded, now I have to change every single function)Use a closure - This isn't really an option, both because there are multiple functions and because closure cannot create variables in their containing scopeReturn the values from a function - This will add lots of unnecessary typing at the top of each function to list out all of the variables and if I wish to add a new input to one simulation (maybe UV just became important) I must now modify all of the functions to have that value passed in.Classes - This is probably the best option currently available. Make a simulation class that has a initialize function and have each simulation inherit from it. This also has some drawbacks though, since I must now instantiate a copy of the object to find the output of the simulation, which makes calling the function harder (especially on a one time basis).Use an inline def - This is very modular and maintainable with an absolute minimum of boilerplate. The main downside being that it is not explicit what is available in the namespace once you initialize the simulation.Here's how it might look: inline def start_simulation(): date = datetime.utcnow() date.replace(tzinfo=utc) start_data = urllib2.urlopen('http://www.source.com/seed_sims').read().split() num_clouds = int(start_data[0]) temp = float(start_data[1]) grid = [] for a in range(int(start_data[2])): grid.append([]) for b in range(int(start_data[3])): grid[a].append(0) def sim_basic(): start_simulation() return temp def sim_iterative(): start_simulation() for row in grid: for cell in row: cell = temp+random.random() for i in range(10): for row in grid for cell in row: cell += random.random()-0.5 return average([cell for row in grid for cell in row]) def sim_clouds(): start_simulation() return temp - numclouds/10. > Date: Wed, 5 Feb 2014 08:57:35 -0700 > Subject: Re: [Python-ideas] Inline Functions - idea > From: ericsnowcurrently at gmail.com > To: lemiant at hotmail.com > CC: python-ideas at python.org > > On Feb 5, 2014 7:34 AM, "Alex Rodrigues" wrote: > > > > Hi everyone, > > > > This is my first time on the Python mailing lists. > > Welcome! > > > I've been learning a lot about how python is run recently and today I thought I ran across an idea which might create in interesting discussion. > > Today I was writing a piece of software where I had a block of code that would take a bunch of local variables, apply some transformations to them and then output them as a a string to a log. Then I realized that I actually wanted to reuse this code in multiple locations - there were multiple cases where I might need to do this. My natural inclination was to write a function in order to maintain DRY programming. This was prohibitively challenging, however, since the code itself interacted with lots of variables in the namespace. The number of arguments to the function would have to be very large and possibly would change on a fairly regular basis. > > This seems like a fairly common problem in programming, having a piece of code which is both reused and heavily integrated with the namespace making it necessary to use copy-paste. As a solution to this I propose the idea of an inline function. An inline function would run in it's parent's namespace instead of creating a new one. > > This sounds like one aspect of Ruby's blocks. Nick Coghlan (one of > the more active Python committers) wrote up a nice block post that > describes them: > > http://www.curiousefficiency.org/posts/2011/10/correcting-ignorance-learning-bit-about.html > > > What do you think? > > You've described an interesting idea. I think Paul is right that it's > a bit of a code smell that the function is so complex as to make > "inlining" desirable. Yet, it seems like there is a practical idea > hiding in there. Keep at the core idea and see if anything more > focused, including a better use case, pops out. > > To be honest the end result will likely be no changes to Python (see > this list's archives and the list of deferred/rejected PEPs). > However, you and others will probably learn a thing or two in the > process. Unlike some traffic on this list, your idea and the > discussion of it are well within the realm of not wasting people's > time. > > -eric > > > [1] Though not exactly applicable to your "inline" functions, aother > Ruby blocks-like idea is statement local namespaces. See PEPs 403 and > 3150 (at http://www.python.org/dev/peps/). -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Wed Feb 5 21:03:46 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 05 Feb 2014 12:03:46 -0800 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: , Message-ID: <52F29922.1000509@stoneleaf.us> On 02/05/2014 07:15 AM, Alex Rodrigues wrote: > > 3. I always knew you could do it the way skip showed, but I feel like that is a bit clunky. This would allow you to do > that in a more straight-forward way. I disagree. saveline(**locals()) is very clear about what it's doing, while saveline() is not. One has to go find `saveline` in the source to see that it's a magical 'in-line' function. This is definitely a case of "explicit is better than implicit". Nice discussion, though. Thanks for bringing it up. -- ~Ethan~ From edk141 at gmail.com Wed Feb 5 21:45:13 2014 From: edk141 at gmail.com (Ed Kellett) Date: Wed, 5 Feb 2014 20:45:13 +0000 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: On 5 February 2014 18:38, yoav glazner wrote: > > How about: > > Def somefun(): > a = 5 > b = 7 > return fmt('{a} is not {b}') > > inline def fmt(fmtstr): > return fmtstr.format_map(locals()) > I wrote a tiny module that did almost exactly what you've just described using stack inspection. I hardly ever use it - not because it's too hacky (although it is) - it's just not something I commonly need to do. Format strings are about the only case I can think of where this makes sense, and since I've had the option to do it, I haven't found it useful. When I wrote the module I ran into a slight problem with closures. Given: def foo(): a = 5 b = 7 def bar(): return fmt('{a} is not {b}') return bar() print foo() you'll an error. bar() doesn't refer to 'a' anywhere, so it's not closed over and not accessible from bar's scope. This will be a problem with inline functions too; closures will need to refer to *everything* in their enclosing scope, in case it calls an inline function at some point during its life. Maybe that's not a problem, but having looked at a few bits and pieces using closures that I've written recently, I'm concerned that a significant amount of garbage would end up sticking around in memory for a long time. On 5 February 2014 19:48, Alex Rodrigues wrote: > Use an inline def - This is very modular and maintainable with an absolute > minimum of boilerplate. The main downside being that it is not explicit what > is available in the namespace once you initialize the simulation. > I think the existence of inline functions would significantly diminish maintainability. If you do: a = 4 b = frobnicate("xyz") you have a pretty good idea of the value of 'a' after the call to frobnicate(). If frobnicate's an inline function, who knows what a might be? Maybe frobnicate() deliberately sets it to something, or accidentally leaks a temporary variable named 'a' into the calling scope. One solution to both problems might be a new operator for inline function calls - closures wouldn't need to be bloaty unless they included at least one such operation, and it would be immediately obvious to a maintainer that they couldn't make any assumptions about the state of the namespace where it appeared. On 5 February 2014 19:48, Alex Rodrigues wrote: > I'm not under any illusions about how hard it is to actually get something > added to standard python . But I thought this would be a fun and informative > undertaking that possibly even helps people. Thank you for your kind words. > After some consideration I have come up with another use case that > demonstrates a little more of how this might be useful: > Let's say that I am a weather forecaster and I maintain a large library of > different simulation techniques. All the techniques need to get their > initial data from some database. Let's take a look at some way of writing > this code and their advantages/disadvantages: > > Hard code the startup code into each simulation - This works, but it will > lead to lots of code re-use and all the problems that come with that (i.e. > if I later need to change how the startup values are downloaded, now I have > to change every single function) > Use a closure - This isn't really an option, both because there are multiple > functions and because closure cannot create variables in their containing > scope > Return the values from a function - This will add lots of unnecessary typing > at the top of each function to list out all of the variables and if I wish > to add a new input to one simulation (maybe UV just became important) I must > now modify all of the functions to have that value passed in. > Classes - This is probably the best option currently available. Make a > simulation class that has a initialize function and have each simulation > inherit from it. This also has some drawbacks though, since I must now > instantiate a copy of the object to find the output of the simulation, which > makes calling the function harder (especially on a one time basis). > Use an inline def - This is very modular and maintainable with an absolute > minimum of boilerplate. The main downside being that it is not explicit what > is available in the namespace once you initialize the simulation. > > Here's how it might look: > > inline def start_simulation(): > date = datetime.utcnow() > date.replace(tzinfo=utc) > start_data = > urllib2.urlopen('http://www.source.com/seed_sims').read().split() > num_clouds = int(start_data[0]) > temp = float(start_data[1]) > grid = [] > for a in range(int(start_data[2])): > grid.append([]) > for b in range(int(start_data[3])): > grid[a].append(0) > > def sim_basic(): > start_simulation() > return temp > > def sim_iterative(): > start_simulation() > for row in grid: > for cell in row: > cell = temp+random.random() > for i in range(10): > for row in grid > for cell in row: > cell += random.random()-0.5 > return average([cell for row in grid for cell in row]) > > def sim_clouds(): > start_simulation() > return temp - numclouds/10. > > >> Date: Wed, 5 Feb 2014 08:57:35 -0700 >> Subject: Re: [Python-ideas] Inline Functions - idea >> From: ericsnowcurrently at gmail.com >> To: lemiant at hotmail.com >> CC: python-ideas at python.org > >> >> On Feb 5, 2014 7:34 AM, "Alex Rodrigues" wrote: >> > >> > Hi everyone, >> > >> > This is my first time on the Python mailing lists. >> >> Welcome! >> >> > I've been learning a lot about how python is run recently and today I >> > thought I ran across an idea which might create in interesting discussion. >> > Today I was writing a piece of software where I had a block of code that >> > would take a bunch of local variables, apply some transformations to them >> > and then output them as a a string to a log. Then I realized that I actually >> > wanted to reuse this code in multiple locations - there were multiple cases >> > where I might need to do this. My natural inclination was to write a >> > function in order to maintain DRY programming. This was prohibitively >> > challenging, however, since the code itself interacted with lots of >> > variables in the namespace. The number of arguments to the function would >> > have to be very large and possibly would change on a fairly regular basis. >> > This seems like a fairly common problem in programming, having a piece >> > of code which is both reused and heavily integrated with the namespace >> > making it necessary to use copy-paste. As a solution to this I propose the >> > idea of an inline function. An inline function would run in it's parent's >> > namespace instead of creating a new one. >> >> This sounds like one aspect of Ruby's blocks. Nick Coghlan (one of >> the more active Python committers) wrote up a nice block post that >> describes them: >> >> >> http://www.curiousefficiency.org/posts/2011/10/correcting-ignorance-learning-bit-about.html >> >> > What do you think? >> >> You've described an interesting idea. I think Paul is right that it's >> a bit of a code smell that the function is so complex as to make >> "inlining" desirable. Yet, it seems like there is a practical idea >> hiding in there. Keep at the core idea and see if anything more >> focused, including a better use case, pops out. >> >> To be honest the end result will likely be no changes to Python (see >> this list's archives and the list of deferred/rejected PEPs). >> However, you and others will probably learn a thing or two in the >> process. Unlike some traffic on this list, your idea and the >> discussion of it are well within the realm of not wasting people's >> time. >> >> -eric >> >> >> [1] Though not exactly applicable to your "inline" functions, aother >> Ruby blocks-like idea is statement local namespaces. See PEPs 403 and >> 3150 (at http://www.python.org/dev/peps/). > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From lemiant at hotmail.com Wed Feb 5 21:45:02 2014 From: lemiant at hotmail.com (Alex Rodrigues) Date: Wed, 5 Feb 2014 13:45:02 -0700 Subject: [Python-ideas] Inline Functions - idea Message-ID: I like the idea of requiring that inline functions be called in a special way that denotes them. Otherwise it does add a much bigger area that a maintainer must look for possible bugs. Also on the topic of locals(), note this from the python docs: Note The contents of this dictionary [the locals() dict] should not be modified; changes may not affect the values of local and free variables used by the interpreter. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Wed Feb 5 22:16:17 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 6 Feb 2014 08:16:17 +1100 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: On Thu, Feb 6, 2014 at 6:48 AM, Alex Rodrigues wrote: > I'm not under any illusions about how hard it is to actually get something > added to standard python . But I thought this would be a fun and informative > undertaking that possibly even helps people. Thank you for your kind words. A good discussion is seldom a waste of time. :) > After some consideration I have come up with another use case that > demonstrates a little more of how this might be useful: > Let's say that I am a weather forecaster and I maintain a large library of > different simulation techniques. All the techniques need to get their > initial data from some database. Let's take a look at some way of writing > this code and their advantages/disadvantages: > > Classes - This is probably the best option currently available. Make a > simulation class that has a initialize function and have each simulation > inherit from it. This also has some drawbacks though, since I must now > instantiate a copy of the object to find the output of the simulation, which > makes calling the function harder (especially on a one time basis). This was my immediate thought on reading your description. Either that, or your start_simulation function is really a single simple class. Try this: class simulation(object): # the explicit inherit is optional in Py3 def __init__(self): self.date = datetime.utcnow() self.date.replace(tzinfo=utc) self.start_data = urllib2.urlopen('http://www.source.com/seed_sims').read().split() self.num_clouds = int(start_data[0]) self.temp = float(self.start_data[1]) # List comprehension is simpler than the nested loop self.grid = [[0]*int(self.start_data[3]) for _ in range(int(start_data[2]))] def sim_basic(): sim = simulation() return sim.temp def sim_iterative(): sim = simulation() for row in sim.grid: # You had this, which won't work anyway: # for cell in row: # cell = random.random() for i in range(len(row)): row[i] = random.random() for i in range(10): for row in sim.grid # Again, modifying cell wouldn't work for i in range(len(row)): row[i] += random.random()-0.5 return average([cell for row in sim.grid for cell in row]) def sim_clouds(): sim = simulation() return sim.temp - sim.numclouds/10. As you see, it's a very straight-forward translation (apart from the bits that I had to change because Python's iteration isn't done with writable references). It's easy to add more to the namespace - and there's no way that it can accidentally collide with a variable name used elsewhere in the simulation. With your model, suppose you wanted to have start_simulation return a thing called 'row'... oops, it's been overwritten by sim_iterative, that's not good. With my model, the new thing would be called self.row or sim.row, so it won't collide. Yes, it's slightly more typing; and when you refactor something (say you find that 95% of your sim_*() functions are deriving the value temp_per_cloud from self.temp and self.num_clouds - yeah, I know, shows how much I know about weather), you have to stick 'sim.' in front of it. But it's guaranteed to be safe. It gives a pretty big benefit in readability, since you know where to go looking for something. Remember, assigning to a name inside a function makes it local; anything you _don't_ assign to is automatically global. That means that you could be happily referencing a global, but then start_simulation gets updated to assign to that name... now your code's broken. This will be very confusing. I'm sure there are situations where inline blocks are extremely useful. I'm just concerned that, in a language where scoping is defined by fairly free rules based on assignment, it'd be way too easy to have the inline block break stuff all over the place. ChrisA From zuo at chopin.edu.pl Wed Feb 5 22:22:59 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Wed, 05 Feb 2014 22:22:59 +0100 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: , Message-ID: <8ec3c4988539c09f49249a59b2291e8b@chopin.edu.pl> 05.02.2014 20:48, Alex Rodrigues wrote: > Let's say that I am a weather forecaster and I maintain a large > library of different simulation techniques. All the techniques need > to > get their initial data from some database. Let's take a look at some > way of writing this code and their advantages/disadvantages: > > * Hard code the startup code into each simulation - This works, but > it will lead to lots of code re-use and all the problems that come > with that (i.e. if I later need to change how the startup values are > downloaded, now I have to change every single function) > * Use a closure - This isn't really an option, both because there > are multiple functions and because closure cannot create variables in > their containing scope > * Return the values from a function - This will add lots of > unnecessary typing at the top of each function to list out all of the > variables and if I wish to add a new input to one simulation (maybe > UV > just became important) I must now modify all of the functions to have > that value passed in. > * Classes - This is probably the best option currently available. > Make a simulation class that has a initialize function and have each > simulation inherit from it. This also has some drawbacks though, > since > I must now instantiate a copy of the object to find the output of the > simulation, which makes calling the function harder (especially on a > one time basis). > * Use an inline def - This is very modular and maintainable with an > absolute minimum of boilerplate. The main downside being that it is > not explicit what is available in the namespace once you initialize > the simulation. > > Here's how it might look: > > inline def start_simulation(): > date = datetime.utcnow() > date.replace(tzinfo=utc) > start_data = > urllib2.urlopen('http://www.source.com/seed_sims').read().split() > num_clouds = int(start_data[0]) > temp = float(start_data[1]) > grid = [] > for a in range(int(start_data[2])): > grid.append([]) > for b in range(int(start_data[3])): > grid[a].append(0) > > def sim_basic(): > start_simulation() > return temp > > def sim_iterative(): > start_simulation() > for row in grid: > for cell in row: > cell = temp+random.random() > for i in range(10): > for row in grid > for cell in row: > cell += random.random()-0.5 > return average([cell for row in grid for cell in row]) > > def sim_clouds(): > start_simulation() > return temp - numclouds/10. Please note, that it can already be implemented in the following way: class Simulation: def __init__(self): self.date = datetime.utcnow().replace(tzinfo=utc) self.start_data = urllib2.urlopen( 'http://www.source.com/seed_sims').read().split() self.num_clouds = int(self.start_data[0]) self.temp = float(self.start_data[1]) self.grid = [] for a in range(int(self.start_data[2])): self.grid.append([]) for b in range(int(self.start_data[3])): self.grid[a].append(0) def sim_basic(): sim = Simulation() return sim.temp def sim_iterative(): sim = Simulation() for row in sim.grid: for cell in row: cell = sim.temp + random.random() for i in range(10): for row in sim.grid: for cell in row: cell += random.random()-0.5 return average(cell for row in sim.grid for cell in row) def sim_clouds(): sim = Simulation() return sim.temp - sim.numclouds/10. I believe it's better as here simulation state variables are explicitly differentiated from actual local variables, so the code is much more readable. Cheers. *j From ethan at stoneleaf.us Wed Feb 5 21:59:53 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 05 Feb 2014 12:59:53 -0800 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: <52F2A649.1030508@stoneleaf.us> On 02/05/2014 12:45 PM, Alex Rodrigues wrote: > > I like the idea of requiring that inline functions be called > in a special way that denotes them. I don't. ;) > Otherwise it does add a much bigger area that a maintainer must > look for possible bugs. Also on the topic of locals(), note this > from the python docs: > >> Note >> >> The contents of this dictionary [the locals() dict] should not >> be modified; changes may not affect the values of local and free >> variables used by the interpreter. Irrelevant. `**locals()` expands the locals dict to be keywords and values, which are then gathered into a new dict: --> def print_me(**args): ... print(id(args)) ... for k, v in args.items(): ... print(k, v) --> def tester(): ... a = 1 ... b = 2 ... c = 3 ... d = locals() ... print(id(d)) ... print_me(**d) --> tester() 33842528 # locals() dict 33842240 # args dict ('a', 1) ('c', 3) ('b', 2) As you can see, the two dicts are different. Further, you could have print_me be more specific about what it uses: --> def print_me(a, b, **not_used): ... print(a, b) and still call it with **locals(). -- ~Ethan~ P.S. Tested using Python 2.7. From steve at pearwood.info Wed Feb 5 23:15:28 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 6 Feb 2014 09:15:28 +1100 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: <52F29922.1000509@stoneleaf.us> References: <52F29922.1000509@stoneleaf.us> Message-ID: <20140205221528.GX3799@ando> On Wed, Feb 05, 2014 at 12:03:46PM -0800, Ethan Furman wrote: > On 02/05/2014 07:15 AM, Alex Rodrigues wrote: > > > > 3. I always knew you could do it the way skip showed, but I feel like > > that is a bit clunky. This would allow you to do > > that in a more straight-forward way. > > I disagree. > > saveline(**locals()) > > is very clear about what it's doing, while > > saveline() > > is not. This is true, and there's no need to write this: def saveline(**kwargs): # Trivial one-liner implementation used for brevity. return kwargs['a'] + kwargs['b'] This ought to work and be much nicer: def saveline(a, b, **kwargs): # kwargs accumulates all the other, unused, locals from the caller. return a + b I think what Alex is describing is a type of function which operates using *dynamic scoping*, rather than static/lexical scoping like Python functions normally do. This means that the function can see the variables from where it is called, not where it is defined. https://en.wikipedia.org/wiki/Scope_%28computer_science%29 http://c2.com/cgi/wiki?DynamicScoping My understanding is that most languages prefer static scoping because it is safer and less error-prone for the programmer, and more easily understood when reading code. But a few languages, such as Perl, offer dynamic scoping as a option. -- Steven From ethan at stoneleaf.us Wed Feb 5 23:26:54 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 05 Feb 2014 14:26:54 -0800 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: <20140205221528.GX3799@ando> References: <52F29922.1000509@stoneleaf.us> <20140205221528.GX3799@ando> Message-ID: <52F2BAAE.6090809@stoneleaf.us> On 02/05/2014 02:15 PM, Steven D'Aprano wrote: > > This ought to work and be much nicer: > > def saveline(a, b, **kwargs): > # kwargs accumulates all the other, unused, locals from the caller. > return a + b Yup, and that's what I proposed in my other email. ;) > I think what Alex is describing is a type of function which operates > using *dynamic scoping*, rather than static/lexical scoping like Python > functions normally do. Possibly, but it's easy enough to simulate (at least in a read-only fashion) with **locals(). -- ~Ethan~ From guido at python.org Thu Feb 6 01:15:45 2014 From: guido at python.org (Guido van Rossum) Date: Wed, 5 Feb 2014 16:15:45 -0800 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: <52F2BAAE.6090809@stoneleaf.us> References: <52F29922.1000509@stoneleaf.us> <20140205221528.GX3799@ando> <52F2BAAE.6090809@stoneleaf.us> Message-ID: On Wed, Feb 5, 2014 at 2:26 PM, Ethan Furman wrote: > Possibly, but it's easy enough to simulate (at least in a read-only > fashion) with **locals(). > Eew. Use of locals() for any purpose other than debugging should be discouraged. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Thu Feb 6 01:26:59 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 5 Feb 2014 16:26:59 -0800 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: On Feb 5, 2014, at 7:28, Chris Angelico wrote: > On Thu, Feb 6, 2014 at 2:15 AM, Alex Rodrigues wrote: >> The remaining use cases for inline functions are times when you would need >> the same capabilities in multiple functions (calling in multiple methods of >> a class for example), or when you wish to define what is essentially a >> closure at the first level of execution, since closures are not possible >> when you are not in a function. > > The nearest concept I can think of to your proposed inline functions > is a preprocessor macro. Effectively, you create a simple token that > expands to a block of source code, and it's handled prior to the > parsing and compilation of the code. There are times when that sort of > thing is really useful, but it's oh so easy to make horribly > unreadable code. (Plus, it'd be hard to get indentation right, if the > different contexts are indented to different levels.) Higher-level macros are a much better match. In lisp terms, what you want is a non-hygienic (or optionally-hygienic) syntactic macro. In Python terms, you can do that by working on ASTs rather than source code. Which has already been implemented very nicely by Li Haoyi--see MacroPy on PyPI. From rosuav at gmail.com Thu Feb 6 01:36:52 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 6 Feb 2014 11:36:52 +1100 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: On Thu, Feb 6, 2014 at 11:26 AM, Andrew Barnert wrote: > On Feb 5, 2014, at 7:28, Chris Angelico wrote: > >> On Thu, Feb 6, 2014 at 2:15 AM, Alex Rodrigues wrote: > >>> The remaining use cases for inline functions are times when you would need >>> the same capabilities in multiple functions (calling in multiple methods of >>> a class for example), or when you wish to define what is essentially a >>> closure at the first level of execution, since closures are not possible >>> when you are not in a function. >> >> The nearest concept I can think of to your proposed inline functions >> is a preprocessor macro. Effectively, you create a simple token that >> expands to a block of source code, and it's handled prior to the >> parsing and compilation of the code. There are times when that sort of >> thing is really useful, but it's oh so easy to make horribly >> unreadable code. (Plus, it'd be hard to get indentation right, if the >> different contexts are indented to different levels.) > > Higher-level macros are a much better match. In lisp terms, what you want is a non-hygienic (or optionally-hygienic) syntactic macro. In Python terms, you can do that by working on ASTs rather than source code. Which has already been implemented very nicely by Li Haoyi--see MacroPy on PyPI. Ah, possibly. I've never dug into that side of things, but that sounds reasonable. It'd solve the indentation issues, at least. In any case, it wouldn't be a function any more; it'd have to be a compile-time alteration. You can't, at run time, choose between several such macros - imagine if they differed in what names they made local. ChrisA From python at mrabarnett.plus.com Thu Feb 6 01:37:35 2014 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 06 Feb 2014 00:37:35 +0000 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: <52F29922.1000509@stoneleaf.us> <20140205221528.GX3799@ando> <52F2BAAE.6090809@stoneleaf.us> Message-ID: <52F2D94F.2000608@mrabarnett.plus.com> On 2014-02-06 00:15, Guido van Rossum wrote: > On Wed, Feb 5, 2014 at 2:26 PM, Ethan Furman > wrote: > > Possibly, but it's easy enough to simulate (at least in a read-only > fashion) with **locals(). > > > Eew. Use of locals() for any purpose other than debugging should be > discouraged. > Even when you're using "format_map"? From guido at python.org Thu Feb 6 01:49:02 2014 From: guido at python.org (Guido van Rossum) Date: Wed, 5 Feb 2014 16:49:02 -0800 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: <52F2D94F.2000608@mrabarnett.plus.com> References: <52F29922.1000509@stoneleaf.us> <20140205221528.GX3799@ando> <52F2BAAE.6090809@stoneleaf.us> <52F2D94F.2000608@mrabarnett.plus.com> Message-ID: On Wed, Feb 5, 2014 at 4:37 PM, MRAB wrote: > On 2014-02-06 00:15, Guido van Rossum wrote: > >> On Wed, Feb 5, 2014 at 2:26 PM, Ethan Furman > > wrote: >> >> Possibly, but it's easy enough to simulate (at least in a read-only >> fashion) with **locals(). >> >> >> Eew. Use of locals() for any purpose other than debugging should be >> discouraged. >> >> Even when you're using "format_map"? > Yes. Explicit is better than implicit. locals() is hard for the compiler to get right -- I wouldn't be surprised if some compilers would have to generate less optimal code if locals() is used. (Similar to sys._getframe().) -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From haoyi.sg at gmail.com Thu Feb 6 02:06:19 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Wed, 5 Feb 2014 17:06:19 -0800 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: <52F29922.1000509@stoneleaf.us> <20140205221528.GX3799@ando> <52F2BAAE.6090809@stoneleaf.us> <52F2D94F.2000608@mrabarnett.plus.com> Message-ID: > Which has already been implemented very nicely by Li Haoyi--see MacroPy on PyPI. In fact, my *trace* and *require* macrosdo exactly what we're describing here: they expand and grab whatever *log* function is present in the scope they're called and use it to do the logging. That way you can control what you're using for logging without having to pass it everywhere. > You can't, at run time, choose between several such macros - imagine if they differed in what names they made local. You probably don't want to! We're talking about functions that grab *different things from their enclosing scope* *depending on how they feel like when you run them*. This is the kind of thing that makes your colleagues go to your house and splash paint on your car. > Even when you're using "format_map"? With macros, you get string interpolationby grabbing them statically at load-time without having to do all this load time magic. As Guido said, dynamic locals() is hard to do for the compiler. If you ever find yourself writing a JIT like PyPy, you will curse the presence of locals() for the headache they cause you when you're trying to optimize away all the silly dictionary lookups. In general, an unhygienic macro is exactly an inline function. It takes its arguments at load-time and splices in the resultant AST into the call-site. You should go have a go with them =) On Wed, Feb 5, 2014 at 4:49 PM, Guido van Rossum wrote: > On Wed, Feb 5, 2014 at 4:37 PM, MRAB wrote: > >> On 2014-02-06 00:15, Guido van Rossum wrote: >> >>> On Wed, Feb 5, 2014 at 2:26 PM, Ethan Furman >> > wrote: >>> >>> Possibly, but it's easy enough to simulate (at least in a read-only >>> fashion) with **locals(). >>> >>> >>> Eew. Use of locals() for any purpose other than debugging should be >>> discouraged. >>> >>> Even when you're using "format_map"? >> > > Yes. Explicit is better than implicit. locals() is hard for the compiler > to get right -- I wouldn't be surprised if some compilers would have to > generate less optimal code if locals() is used. (Similar to > sys._getframe().) > > -- > --Guido van Rossum (python.org/~guido) > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Thu Feb 6 01:53:56 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 05 Feb 2014 16:53:56 -0800 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: <52F29922.1000509@stoneleaf.us> <20140205221528.GX3799@ando> <52F2BAAE.6090809@stoneleaf.us> Message-ID: <52F2DD24.50603@stoneleaf.us> On 02/05/2014 04:15 PM, Guido van Rossum wrote: > On Wed, Feb 5, 2014 at 2:26 PM, Ethan Furman wrote: >> >> Possibly, but it's easy enough to simulate (at least in >> a read-only fashion) with **locals(). > > Eew. Use of locals() for any purpose other than debugging should be discouraged. Well, as I recall (although I can't find it now) there was discussion about auto-filling a functions parameters with local values when the names matched, which would be much nicer is this situation (as well as extending situations). For example: class MyString(str): def find(sub, start, end): ... # manipulate sub and/or start and/or end ... return super().find(#blah#) where #blah# means "pick out sub, start, and end from locals and use them". -- ~Ethan~ From ethan at stoneleaf.us Thu Feb 6 02:10:51 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 05 Feb 2014 17:10:51 -0800 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: <52F2DD24.50603@stoneleaf.us> References: <52F29922.1000509@stoneleaf.us> <20140205221528.GX3799@ando> <52F2BAAE.6090809@stoneleaf.us> <52F2DD24.50603@stoneleaf.us> Message-ID: <52F2E11B.8010709@stoneleaf.us> On 02/05/2014 04:53 PM, Ethan Furman wrote: > > Well, as I recall (although I can't find it now) Found it: https://mail.python.org/pipermail/python-ideas/2013-June/021466.html -- ~Ethan~ From guido at python.org Thu Feb 6 03:47:35 2014 From: guido at python.org (Guido van Rossum) Date: Wed, 5 Feb 2014 18:47:35 -0800 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: <52F2DD24.50603@stoneleaf.us> References: <52F29922.1000509@stoneleaf.us> <20140205221528.GX3799@ando> <52F2BAAE.6090809@stoneleaf.us> <52F2DD24.50603@stoneleaf.us> Message-ID: On Wed, Feb 5, 2014 at 4:53 PM, Ethan Furman wrote: > On 02/05/2014 04:15 PM, Guido van Rossum wrote: > > On Wed, Feb 5, 2014 at 2:26 PM, Ethan Furman wrote: >> >>> >>> Possibly, but it's easy enough to simulate (at least in >>> a read-only fashion) with **locals(). >>> >> >> Eew. Use of locals() for any purpose other than debugging should be >> discouraged. >> > > Well, as I recall (although I can't find it now) there was discussion > about auto-filling a functions parameters with local values when the names > matched, which would be much nicer is this situation (as well as extending > situations). For example: > > class MyString(str): > def find(sub, start, end): > ... > # manipulate sub and/or start and/or end > ... > return super().find(#blah#) > > where #blah# means "pick out sub, start, and end from locals and use them". > And how on earth would the compiler know the argument names? -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron3200 at gmail.com Thu Feb 6 11:34:16 2014 From: ron3200 at gmail.com (Ron Adam) Date: Thu, 06 Feb 2014 04:34:16 -0600 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: On 02/05/2014 08:49 AM, Skip Montanaro wrote: > I'm not convinced this particular example needs an inline function. > Just write saveLine() as a normal function and pass in your locals: > > def f(): > a = 1 > b = 7 > compute(**locals()) > > def g(): > a = 3 > b = 0.9 > compute(**locals()) > > def compute(**args): > print args["a"] + args["b"] > > f() > g() > > That's not to say inline functions might not be handy. It's possible > that you can implement them without modifying the language either. A > peephole optimizer could inline small functions which are used a lot. > > Skip The other day I was trying to figure out how to do lamba calculus without using closures (or other tricks). >>> def I (x): return x >>> def K (x): ... return lambda y: x >>> def S (x): ... return lambda y: lambda z: x(z)(y(z)) >>> I("A") 'A' >>> K("A")("B") 'A' >>> S(K)(K)("A") 'A' To do it without closures isn't easy. But it did give me an interesting idea that could simplify a lot of problems. Basically... # S := ?x.?y.?z.x z (y z) def continue S: :from(x) :from(y) :from(z) return x(z)(y(z)) And it would be used exactly like the lambda example above. S(K)(K)("A") --> 'A' That's equation is supposed to give the same result back out.. (* That is if I have it right?) Each call returns the updated frame but we only have one code object which makes it easier to write and think about. In this example.. g1 = S(K) g2 = g1(K) result = g2('A') --> also S(K)(K)('A') It depends on how it's implemented weather g1 and g2 are the same object, or new objects with different states. Each ":from()" can use a full function signature, So these could be used as decorators. def continue foo: :from(func) :from(*args, **kwds) # do something with the function arguments return func(*args, **kwds) @foo def bar(...): ... Or probably also be used as with statement context managers. There is very little duplication. The :from(...) arguments get pulled directly into locals, (the same as a function call updates them).. so there is no assignment needed and no need to play with locals(). You could also think of this as an explicit closure turned inside out. The closure become locals, and return that with each function signature.... but there is only one code object shared between calls, it makes writing a lot of things easier. About the funny grammar... I think it needs something different to say these are functions signatures. Normally, the body is on the right of the colon, I used the reverse in this case because it is presenting the function signature from the inside, rather than the outside. (Well, it makes sense to me anyway.) I'm sure there are lots of other colours for this bike. Cheers, Ron From steve at pearwood.info Thu Feb 6 11:56:02 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 6 Feb 2014 21:56:02 +1100 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: <52F29922.1000509@stoneleaf.us> <20140205221528.GX3799@ando> <52F2BAAE.6090809@stoneleaf.us> <52F2DD24.50603@stoneleaf.us> Message-ID: <20140206105601.GA3799@ando> On Wed, Feb 05, 2014 at 06:47:35PM -0800, Guido van Rossum wrote: > On Wed, Feb 5, 2014 at 4:53 PM, Ethan Furman wrote: > > class MyString(str): > > def find(sub, start, end): > > ... > > # manipulate sub and/or start and/or end > > ... > > return super().find(#blah#) > > > > where #blah# means "pick out sub, start, and end from locals and use them". > > > > And how on earth would the compiler know the argument names? This is not a defence of the idea, which I find rather nasty, but I suppose you might compile the call x.find(#blah#) into code like this: # pseudo-code func = getattr(x, 'find') arg_list = inspect.getargspec(func).args l = locals() args = {arg: l[arg] for arg in arg_list if arg != "self"} func(**args) I think this idea would be inefficient, hard to understand, and brittle, just to save the programmer from having to explicitly write out the arguments to a function. I'm sure it would be really popular in PHP circles :-) -- Steven From ncoghlan at gmail.com Thu Feb 6 12:14:41 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 6 Feb 2014 21:14:41 +1000 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: On 6 February 2014 20:34, Ron Adam wrote: > On 02/05/2014 08:49 AM, Skip Montanaro wrote: >> >> I'm not convinced this particular example needs an inline function. >> Just write saveLine() as a normal function and pass in your locals: >> >> def f(): >> a = 1 >> b = 7 >> compute(**locals()) >> >> def g(): >> a = 3 >> b = 0.9 >> compute(**locals()) >> >> def compute(**args): >> print args["a"] + args["b"] >> >> f() >> g() >> >> That's not to say inline functions might not be handy. It's possible >> that you can implement them without modifying the language either. A >> peephole optimizer could inline small functions which are used a lot. >> >> Skip > > > The other day I was trying to figure out how to do lamba calculus without > using closures (or other tricks). > >>>> def I (x): return x >>>> def K (x): > ... return lambda y: x This returns a closure. >>>> def S (x): > ... return lambda y: lambda z: x(z)(y(z)) This returns a closure that returns a closure. > To do it without closures isn't easy. Lambda expressions aren't special from a scoping point of view, they're just a different way to write: def (*args, **kwds): return expr They're only special from a structural point of view (as expressions, you can put them in places where full functions aren't allowed). As far as what you're proposing goes, is it essentially a way to declare a function like (spelling out the lambdas fully): def S(x): def _second(y): def _third(z): return x(z)(y(z)) return _third return _second As something much shorter like this: def S (x)(y)(z): return x(z)(y(z)) The main potential benefit I could see to a construct like that is that it may allow the more consistent creation of closures that support pickling, since the outer functions are guaranteed not to have any side effects and to have argument capture as their *only* significant state. This means that you could take the inner function, pickle it along with its closure variables and reconstruct that at the far end, only relying on the name of the outer function. Such a construct could also make decorator factories easier to write. def decorator(f): # Do something with f def decorator_factory(some, args, here)(f): # Do something with f, but have access to the bound args. There's an argument to be made that the extra parens in the function header are too easy to miss, but I still see "make it easier to write side-effect free closures" as an idea worth discussing further. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ram.rachum at gmail.com Thu Feb 6 11:24:12 2014 From: ram.rachum at gmail.com (Ram Rachum) Date: Thu, 6 Feb 2014 02:24:12 -0800 (PST) Subject: [Python-ideas] int('0x3241fca1') Message-ID: What do you think about letting the `int` constructor automatically understand the number type without specifying base if given a prefix, so int('0x3414fa') would immediately work without specifying a base of 16? -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Thu Feb 6 13:11:09 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 6 Feb 2014 04:11:09 -0800 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: On Feb 6, 2014, at 3:14, Nick Coghlan wrote: > On 6 February 2014 20:34, Ron Adam wrote: >> On 02/05/2014 08:49 AM, Skip Montanaro wrote: >>> >>> I'm not convinced this particular example needs an inline function. >>> Just write saveLine() as a normal function and pass in your locals: >>> >>> def f(): >>> a = 1 >>> b = 7 >>> compute(**locals()) >>> >>> def g(): >>> a = 3 >>> b = 0.9 >>> compute(**locals()) >>> >>> def compute(**args): >>> print args["a"] + args["b"] >>> >>> f() >>> g() >>> >>> That's not to say inline functions might not be handy. It's possible >>> that you can implement them without modifying the language either. A >>> peephole optimizer could inline small functions which are used a lot. >>> >>> Skip >> >> >> The other day I was trying to figure out how to do lamba calculus without >> using closures (or other tricks). >> >>>>> def I (x): return x >>>>> def K (x): >> ... return lambda y: x > > This returns a closure. > >>>>> def S (x): >> ... return lambda y: lambda z: x(z)(y(z)) > > This returns a closure that returns a closure. > >> To do it without closures isn't easy. > > Lambda expressions aren't special from a scoping point of view, > they're just a different way to write: > > def (*args, **kwds): > return expr > > They're only special from a structural point of view (as expressions, > you can put them in places where full functions aren't allowed). > > As far as what you're proposing goes, is it essentially a way to > declare a function like (spelling out the lambdas fully): > > def S(x): > def _second(y): > def _third(z): > return x(z)(y(z)) > return _third > return _second > > As something much shorter like this: > > def S (x)(y)(z): > return x(z)(y(z)) I think his point is that he wants it to be implemented _without_ closures--or at least as a single closure, with the same scope shared by all of the functions instead of three scopes and closure cells linking them together. Personally, I'm not sure what that adds; I'm just raising what I think is someone else's point. There could conceivably be performance benefits, but then again, there could also be a cost to keeping alive an entire scope when you don't need everything. Other than that, the visible effect would be exactly the same as with your explicit closure version. > The main potential benefit I think the arguable brevity and readability benefit is the main potential benefit here, as your decorator example. If the abbreviated examples are clearer, the idea is a win even without the pickling; if it's more of an attractive nuisance that makes it easier to write impenetrable code more than it makes it easier to write simpler code, the idea isn't worth doing even with the pickling. > I could see to a construct like that is > that it may allow the more consistent creation of closures that > support pickling, since the outer functions are guaranteed not to have > any side effects and to have argument capture as their *only* > significant state. This means that you could take the inner function, > pickle it along with its closure variables and reconstruct that at the > far end, only relying on the name of the outer function. Out of curiosity, how would you implement that? Right now, functions don't reduce, and pickle just stores their global name. Would you make pickle detect side-effect-free closures (by way of a flag exposed via inspect or something?), dig out the arguments to the outer function, and pickle them as calls with a new opcode similar to new-style class instances? Or change functions so they can reduce themselves in appropriate circumstances? > Such a construct could also make decorator factories easier to write. > > def decorator(f): > # Do something with f > > def decorator_factory(some, args, here)(f): > # Do something with f, but have access to the bound args. > > There's an argument to be made that the extra parens in the function > header are too easy to miss, but I still see "make it easier to write > side-effect free closures" as an idea worth discussing further. I like this idea, and your syntax. With or without collapsing frames or doing anything special (like pickling) with the results. From denis.spir at gmail.com Thu Feb 6 13:28:55 2014 From: denis.spir at gmail.com (spir) Date: Thu, 06 Feb 2014 13:28:55 +0100 Subject: [Python-ideas] int('0x3241fca1') In-Reply-To: References: Message-ID: <52F38007.80809@gmail.com> On 02/06/2014 11:24 AM, Ram Rachum wrote: > What do you think about letting the `int` constructor automatically > understand the number type without specifying base if given a prefix, so > int('0x3414fa') would immediately work without specifying a base of 16? Do you mean int(numeral), where numeral is a *variable* string, with a python base prefix? (Else, just type in the constant 0x3414fa ;-) If yes, then I find it a good idea. When int() is used to decode variable numerals, it could/should/would decode all correct python numeral notations. Note that int() also does not decode 'e' postfixes: Python 3.3.2+ (default, Oct 9 2013, 14:50:09) [GCC 4.8.1] on linux Type "help", "copyright", "credits" or "license" for more information. >>> int(123e2) 12300 >>> int("123e2") Traceback (most recent call last): File "", line 1, in ValueError: invalid literal for int() with base 10: '123e2' But float() does: >>> float(-1.23e4) -12300.0 >>> float("-1.23e4") -12300.0 ! After all, it's just a question of practical notational conventions (we don't use "hundred and twenty-three" or "CXXIII" or "v^^^^^v^^"). Python's own decoding builtins should be consistent with its own choice of notations. d From ram at rachum.com Thu Feb 6 13:30:30 2014 From: ram at rachum.com (Ram Rachum) Date: Thu, 6 Feb 2014 14:30:30 +0200 Subject: [Python-ideas] int('0x3241fca1') In-Reply-To: <52F38007.80809@gmail.com> References: <52F38007.80809@gmail.com> Message-ID: Yep, that's what I meant :) On Thu, Feb 6, 2014 at 2:28 PM, spir wrote: > On 02/06/2014 11:24 AM, Ram Rachum wrote: > >> What do you think about letting the `int` constructor automatically >> understand the number type without specifying base if given a prefix, so >> int('0x3414fa') would immediately work without specifying a base of 16? >> > > Do you mean int(numeral), where numeral is a *variable* string, with a > python base prefix? (Else, just type in the constant 0x3414fa ;-) If yes, > then I find it a good idea. When int() is used to decode variable numerals, > it could/should/would decode all correct python numeral notations. > > Note that int() also does not decode 'e' postfixes: > > Python 3.3.2+ (default, Oct 9 2013, 14:50:09) > [GCC 4.8.1] on linux > Type "help", "copyright", "credits" or "license" for more information. > >> int(123e2) >>>> >>> 12300 > >> int("123e2") >>>> >>> Traceback (most recent call last): > File "", line 1, in > ValueError: invalid literal for int() with base 10: '123e2' > > But float() does: > > float(-1.23e4) >>>> >>> -12300.0 > >> float("-1.23e4") >>>> >>> -12300.0 > > ! > > After all, it's just a question of practical notational conventions (we > don't use "hundred and twenty-three" or "CXXIII" or "v^^^^^v^^"). Python's > own decoding builtins should be consistent with its own choice of notations. > > d > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > -- > > --- You received this message because you are subscribed to a topic in the > Google Groups "python-ideas" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/python-ideas/RKQviWz9BYk/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > python-ideas+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From graffatcolmingov at gmail.com Thu Feb 6 13:36:54 2014 From: graffatcolmingov at gmail.com (Ian Cordasco) Date: Thu, 6 Feb 2014 06:36:54 -0600 Subject: [Python-ideas] int('0x3241fca1') In-Reply-To: References: <52F38007.80809@gmail.com> Message-ID: For point of reference, other languages are similarly broken (or perhaps they just choose to not guess). Ruby for example exhibits the following behaviour: '0x123'.to_i # => 0 '0x123'.to_i 16 # => 291 '123'.to_i # => 123 And for what it is worth, int has a default parameter base with the value of 10. If you look at the documentation that is present: | int(x, base=10) -> int or long In other words, int is always expecting a base 10 number unless otherwise specified. Guessing at a number's base to save you from having to call int('0x123', 16) is not a good thing. Regarding internal consistency (int vs float), I would guess (but I don't know) that this has to do with how floats are represented typically. But I'll let someone with more specific knowledge of that difference answer that concern. On Thu, Feb 6, 2014 at 6:30 AM, Ram Rachum wrote: > Yep, that's what I meant :) > > > On Thu, Feb 6, 2014 at 2:28 PM, spir wrote: > >> On 02/06/2014 11:24 AM, Ram Rachum wrote: >> >>> What do you think about letting the `int` constructor automatically >>> understand the number type without specifying base if given a prefix, so >>> int('0x3414fa') would immediately work without specifying a base of 16? >>> >> >> Do you mean int(numeral), where numeral is a *variable* string, with a >> python base prefix? (Else, just type in the constant 0x3414fa ;-) If yes, >> then I find it a good idea. When int() is used to decode variable numerals, >> it could/should/would decode all correct python numeral notations. >> >> Note that int() also does not decode 'e' postfixes: >> >> Python 3.3.2+ (default, Oct 9 2013, 14:50:09) >> [GCC 4.8.1] on linux >> Type "help", "copyright", "credits" or "license" for more information. >> >>> int(123e2) >>>>> >>>> 12300 >> >>> int("123e2") >>>>> >>>> Traceback (most recent call last): >> File "", line 1, in >> ValueError: invalid literal for int() with base 10: '123e2' >> >> But float() does: >> >> float(-1.23e4) >>>>> >>>> -12300.0 >> >>> float("-1.23e4") >>>>> >>>> -12300.0 >> >> ! >> >> After all, it's just a question of practical notational conventions (we >> don't use "hundred and twenty-three" or "CXXIII" or "v^^^^^v^^"). Python's >> own decoding builtins should be consistent with its own choice of notations. >> >> d >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> >> >> -- >> >> --- You received this message because you are subscribed to a topic in >> the Google Groups "python-ideas" group. >> To unsubscribe from this topic, visit https://groups.google.com/d/ >> topic/python-ideas/RKQviWz9BYk/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> python-ideas+unsubscribe at googlegroups.com. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Thu Feb 6 13:39:49 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 6 Feb 2014 12:39:49 +0000 Subject: [Python-ideas] int('0x3241fca1') In-Reply-To: <52F38007.80809@gmail.com> References: <52F38007.80809@gmail.com> Message-ID: On 6 February 2014 12:28, spir wrote: > Do you mean int(numeral), where numeral is a *variable* string, with a > python base prefix? (Else, just type in the constant 0x3414fa ;-) If yes, > then I find it a good idea. When int() is used to decode variable numerals, > it could/should/would decode all correct python numeral notations. Having a function like that might be useful, but I also find the current int() behaviour useful - so maybe just have a separate function. And actually, I think it's easy enough to build something like that based on ast.literal_eval, so maybe having a builtin for it isn't worth it...? Paul From abarnert at yahoo.com Thu Feb 6 13:37:32 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 6 Feb 2014 04:37:32 -0800 Subject: [Python-ideas] int('0x3241fca1') In-Reply-To: <52F38007.80809@gmail.com> References: <52F38007.80809@gmail.com> Message-ID: On Feb 6, 2014, at 4:28, spir wrote: > On 02/06/2014 11:24 AM, Ram Rachum wrote: >> What do you think about letting the `int` constructor automatically >> understand the number type without specifying base if given a prefix, so >> int('0x3414fa') would immediately work without specifying a base of 16? > > Do you mean int(numeral), where numeral is a *variable* string, with a python base prefix? (Else, just type in the constant 0x3414fa ;-) If yes, then I find it a good idea. When int() is used to decode variable numerals, it could/should/would decode all correct python numeral notations. > > Note that int() also does not decode 'e' postfixes: Of course it doesn't. A number with an e postfix is a float, not an int, so why should int parse it? You also can't use int to parse 123.45, or even 123.0, or a quoted string like "123", or "1+0j". > Python 3.3.2+ (default, Oct 9 2013, 14:50:09) > [GCC 4.8.1] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> int(123e2) > 12300 >>>> int("123e2") > Traceback (most recent call last): > File "", line 1, in > ValueError: invalid literal for int() with base 10: '123e2' > > But float() does: > >>>> float(-1.23e4) > -12300.0 >>>> float("-1.23e4") > -12300.0 > > ! > > After all, it's just a question of practical notational conventions (we don't use "hundred and twenty-three" or "CXXIII" or "v^^^^^v^^"). Python's own decoding builtins should be consistent with its own choice of notations. In that case, int('-3') shouldn't work, because -3 is not a valid int literal in Python. (It's a unary expression with the '-' operator followed by the literal 3.) From abarnert at yahoo.com Thu Feb 6 13:47:54 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 6 Feb 2014 04:47:54 -0800 Subject: [Python-ideas] int('0x3241fca1') In-Reply-To: References: <9DCCBB69-EB26-491A-85BA-51C08DDBC643@yahoo.com> <952DC233-1F8D-432C-BA54-DE6D707DBDD9@yahoo.com> Message-ID: <47FD681B-C962-4C48-9AF4-8C7E6F3AE95D@yahoo.com> On Feb 6, 2014, at 4:44, Ram Rachum wrote: > Ah, you're talking about non-programmers. I can't imagine though a non-programmer seriously complaining that "0x0" is a number while "0z" isn't. Can you please read the whole sentence before replying to it? I specifically said "Would a newbie--or a non-programmer using a program--understand..." And you can't imagine a non-programmer complaining that "0123" is either not a number, or the number 83? > On Thu, Feb 6, 2014 at 2:41 PM, Andrew Barnert wrote: >> On Feb 6, 2014, at 4:35, Ram Rachum wrote: >> >>> On Thu, Feb 6, 2014 at 2:31 PM, Andrew Barnert wrote: >>>> On Feb 6, 2014, at 4:19, Ram Rachum wrote: >>>> >>>>> I don't understand... The newbie will pass '0x13412' to the int constructor by mistake and be surprised when it's parsed as hex? Doesn't make sense does it? >>>> >>>> Would a newbie--or a non-programmer using a program--understand why, say, "0X0" counts as a valid number but "0Z" doesn't? >>> >>> Since they're likely to get the same confusion while feeding literals to the Python shell, I think this is acceptable. >> >> You think it's acceptable that a non-programmer should have to understand the python shell to use any program written in Python? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ram.rachum at gmail.com Thu Feb 6 13:51:08 2014 From: ram.rachum at gmail.com (Ram Rachum) Date: Thu, 6 Feb 2014 14:51:08 +0200 Subject: [Python-ideas] int('0x3241fca1') In-Reply-To: <47FD681B-C962-4C48-9AF4-8C7E6F3AE95D@yahoo.com> References: <9DCCBB69-EB26-491A-85BA-51C08DDBC643@yahoo.com> <952DC233-1F8D-432C-BA54-DE6D707DBDD9@yahoo.com> <47FD681B-C962-4C48-9AF4-8C7E6F3AE95D@yahoo.com> Message-ID: On Thu, Feb 6, 2014 at 2:47 PM, Andrew Barnert wrote: > On Feb 6, 2014, at 4:44, Ram Rachum wrote: > > Ah, you're talking about non-programmers. I can't imagine though a > non-programmer seriously complaining that "0x0" is a number while "0z" > isn't. > > > Can you please read the whole sentence before replying to it? I > specifically said "Would a newbie--or a non-programmer using a > program--understand..." > > And you can't imagine a non-programmer complaining that "0123" is either > not a number, or the number 83? > You're right. I suggest int would just throw away the leading zeroes. In Python 3.x there's no meaning to them without 0o or 0x or 0b anyway, right? > > On Thu, Feb 6, 2014 at 2:41 PM, Andrew Barnert wrote: > >> On Feb 6, 2014, at 4:35, Ram Rachum wrote: >> >> On Thu, Feb 6, 2014 at 2:31 PM, Andrew Barnert wrote: >> >>> On Feb 6, 2014, at 4:19, Ram Rachum wrote: >>> >>> I don't understand... The newbie will pass '0x13412' to the int >>> constructor by mistake and be surprised when it's parsed as hex? Doesn't >>> make sense does it? >>> >>> >>> Would a newbie--or a non-programmer using a program--understand why, >>> say, "0X0" counts as a valid number but "0Z" doesn't? >>> >> >> Since they're likely to get the same confusion while feeding literals to >> the Python shell, I think this is acceptable. >> >> >> You think it's acceptable that a non-programmer should have to understand >> the python shell to use any program written in Python? >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Thu Feb 6 14:00:49 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 6 Feb 2014 13:00:49 +0000 Subject: [Python-ideas] int('0x3241fca1') In-Reply-To: References: <9DCCBB69-EB26-491A-85BA-51C08DDBC643@yahoo.com> <952DC233-1F8D-432C-BA54-DE6D707DBDD9@yahoo.com> <47FD681B-C962-4C48-9AF4-8C7E6F3AE95D@yahoo.com> Message-ID: On 6 February 2014 12:51, Ram Rachum wrote: >> And you can't imagine a non-programmer complaining that "0123" is either >> not a number, or the number 83? > > You're right. I suggest int would just throw away the leading zeroes. In > Python 3.x there's no meaning to them without 0o or 0x or 0b anyway, right? Once you get to this point, you're writing your own spec for what input you want to allow, and you're probably better off writing your own code that meets that spec. Sanitising the string how you want then calling ast.literal_eval is easy enough. -1 on changing the behaviour of int. -0.5 on adding a new builtin (I see no need for it, but if others do, it won't cause me any issues if it exists...). +1 on adding documentation examples showing how to use ast.literal_eval to write your own specialised evaluators. Paul. From abarnert at yahoo.com Thu Feb 6 14:03:44 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 6 Feb 2014 05:03:44 -0800 Subject: [Python-ideas] int('0x3241fca1') In-Reply-To: References: <9DCCBB69-EB26-491A-85BA-51C08DDBC643@yahoo.com> <952DC233-1F8D-432C-BA54-DE6D707DBDD9@yahoo.com> <47FD681B-C962-4C48-9AF4-8C7E6F3AE95D@yahoo.com> Message-ID: On Feb 6, 2014, at 4:51, Ram Rachum wrote: > On Thu, Feb 6, 2014 at 2:47 PM, Andrew Barnert wrote: >> On Feb 6, 2014, at 4:44, Ram Rachum wrote: >> >>> Ah, you're talking about non-programmers. I can't imagine though a non-programmer seriously complaining that "0x0" is a number while "0z" isn't. >> >> Can you please read the whole sentence before replying to it? I specifically said "Would a newbie--or a non-programmer using a program--understand..." >> >> And you can't imagine a non-programmer complaining that "0123" is either not a number, or the number 83? > > You're right. I suggest int would just throw away the leading zeroes. In Python 3.x there's no meaning to them without 0o or 0x or 0b anyway, right? As a literal in Python source, there are good reasons for it to be an error--because in Python 2.x, in many other languages, in various shell command invocations that you might want to convert from bash scripts to Python, etc., it means octal. Someone who types "0123" into Python source code could just as easily mean 83 as 123, and that's why it's an error. As input from an end user, on the other hand, "0123" clearly means 123, not 83. And that's a perfect example of why int(s) should not always do the same thing as the interactive interpreter. When you _want_ the same behavior as the interpreter, it's trivial to get it. When you want to treat end user data--or data from some file format or protocol, or whatever--as a string representation of an integer, rather than as a string representation of a Python integer literal--why shouldn't that be spelled int(s)? > >> >>> On Thu, Feb 6, 2014 at 2:41 PM, Andrew Barnert wrote: >>>> On Feb 6, 2014, at 4:35, Ram Rachum wrote: >>>> >>>>> On Thu, Feb 6, 2014 at 2:31 PM, Andrew Barnert wrote: >>>>>> On Feb 6, 2014, at 4:19, Ram Rachum wrote: >>>>>> >>>>>>> I don't understand... The newbie will pass '0x13412' to the int constructor by mistake and be surprised when it's parsed as hex? Doesn't make sense does it? >>>>>> >>>>>> Would a newbie--or a non-programmer using a program--understand why, say, "0X0" counts as a valid number but "0Z" doesn't? >>>>> >>>>> Since they're likely to get the same confusion while feeding literals to the Python shell, I think this is acceptable. >>>> >>>> You think it's acceptable that a non-programmer should have to understand the python shell to use any program written in Python? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From denis.spir at gmail.com Thu Feb 6 14:07:41 2014 From: denis.spir at gmail.com (spir) Date: Thu, 06 Feb 2014 14:07:41 +0100 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: <52F3891D.3060106@gmail.com> On 02/05/2014 03:32 PM, Alex Rodrigues wrote: > Hi everyone, > This is my first time on the Python mailing lists. I've been learning a lot about how python is run recently and today I thought I ran across an idea which might create in interesting discussion.Today I was writing a piece of software where I had a block of code that would take a bunch of local variables, apply some transformations to them and then output them as a a string to a log. Then I realized that I actually wanted to reuse this code in multiple locations - there were multiple cases where I might need to do this. My natural inclination was to write a function in order to maintain DRY programming. This was prohibitively challenging, however, since the code itself interacted with lots of variables in the namespace. The number of arguments to the function would have to be very large and possibly would change on a fairly regular basis.This seems like a fairly common problem in programming, having a piece of code which is both reused and heavily integrated with the namespace makin g it necessary to use copy-paste. As a solution to this I propose the idea of an inline function. An inline function would run in it's parent's namespace instead of creating a new one. This would allow you to avoid passing back and forth tons of values while still maintaining DRY code. Do you really mean "parent", or instead "caller". If parent, then it's a closure function. If caller, I guess you are proposing to introduce dynamic scoping to python, which uses lexical scoping [*]. Dynamic scoping is (as I understand it) when variables of the caller scope are used in a called procedure; which seems to be what you need, unless I misunderstand and you really mean parent scope (but this does not seem to match your use case explanations). d [*] see: https://en.wikipedia.org/wiki/Lexical_scoping#Lexical_scoping_vs._dynamic_scoping From python at mrabarnett.plus.com Thu Feb 6 14:22:21 2014 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 06 Feb 2014 13:22:21 +0000 Subject: [Python-ideas] int('0x3241fca1') In-Reply-To: References: Message-ID: <52F38C8D.7020501@mrabarnett.plus.com> On 2014-02-06 10:24, Ram Rachum wrote: > What do you think about letting the `int` constructor automatically > understand the number type without specifying base if given a prefix, so > int('0x3414fa') would immediately work without specifying a base of 16? > The base defaults to 10, but: >>> int('0x3241fca1', 0) 843185313 From ned at nedbatchelder.com Thu Feb 6 16:13:17 2014 From: ned at nedbatchelder.com (Ned Batchelder) Date: Thu, 06 Feb 2014 10:13:17 -0500 Subject: [Python-ideas] int('0x3241fca1') In-Reply-To: <52F38C8D.7020501@mrabarnett.plus.com> References: <52F38C8D.7020501@mrabarnett.plus.com> Message-ID: <52F3A68D.6020405@nedbatchelder.com> On 2/6/14 8:22 AM, MRAB wrote: > On 2014-02-06 10:24, Ram Rachum wrote: >> What do you think about letting the `int` constructor automatically >> understand the number type without specifying base if given a prefix, so >> int('0x3414fa') would immediately work without specifying a base of 16? >> > The base defaults to 10, but: > > >>> int('0x3241fca1', 0) > 843185313 > I can't believe how many replies this thread got that overlooked that the built-in int() already does exactly what the OP wanted. Is it too much to ask that before you suggest changing the behavior of something, that you read *both* paragraphs of the documentation? http://docs.python.org/2/library/functions.html#int --Ned. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From ethan at stoneleaf.us Thu Feb 6 16:16:39 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 06 Feb 2014 07:16:39 -0800 Subject: [Python-ideas] int('0x3241fca1') In-Reply-To: References: <52F38007.80809@gmail.com> Message-ID: <52F3A757.4040006@stoneleaf.us> On 02/06/2014 04:37 AM, Andrew Barnert wrote: > > Of course it doesn't. A number with an e postfix is a float, not an > int, so why should int parse it? You also can't use int to parse > 123.45, or even 123.0, or a quoted string like "123", or "1+0j". Python 3.4.0b3+ (default:7d0a4f89c6ce, Feb 5 2014, 17:04:36) [GCC 4.7.3] on linux Type "help", "copyright", "credits" or "license" for more information. --> int("123") 123 -- ~Etahn~ From python at mrabarnett.plus.com Thu Feb 6 17:37:42 2014 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 06 Feb 2014 16:37:42 +0000 Subject: [Python-ideas] int('0x3241fca1') In-Reply-To: <52F3A757.4040006@stoneleaf.us> References: <52F38007.80809@gmail.com> <52F3A757.4040006@stoneleaf.us> Message-ID: <52F3BA56.7040107@mrabarnett.plus.com> On 2014-02-06 15:16, Ethan Furman wrote: > On 02/06/2014 04:37 AM, Andrew Barnert wrote: >> >> Of course it doesn't. A number with an e postfix is a float, not an >> int, so why should int parse it? You also can't use int to parse >> 123.45, or even 123.0, or a quoted string like "123", or "1+0j". > > Python 3.4.0b3+ (default:7d0a4f89c6ce, Feb 5 2014, 17:04:36) > [GCC 4.7.3] on linux > Type "help", "copyright", "credits" or "license" for more information. > --> int("123") > 123 > By "quoted string" he means '"123"'. From bruce at leapyear.org Thu Feb 6 19:39:33 2014 From: bruce at leapyear.org (Bruce Leban) Date: Thu, 6 Feb 2014 10:39:33 -0800 Subject: [Python-ideas] Fwd: int('0x3241fca1') In-Reply-To: References: Message-ID: On Thu, Feb 6, 2014 at 2:24 AM, Ram Rachum wrote: > What do you think about letting the `int` constructor automatically > understand the number type without specifying base if given a prefix, so > int('0x3414fa') would immediately work without specifying a base of 16? > What's the use case? Do you expect prefixes in user input? In data you're reading from files and parsing where you don't know the base? I don't see it. This would change the behavior of current code and potentially introduce bugs. For example, this allows attackers to trick programs into accepting aliases of numbers they would not otherwise accept. Search [overlong utf-8 sequences] for an example of this kind of attack. And it would be quite strange that a base of 10 means sometimes 10 and sometimes look at a prefix while all other bases mean exactly what they say. >>> int('0xa', 16) 10 >>> int('0xa', 36) 1198 >>> int('1e1', 16) 481 EIBTI: If there's a compelling use case, a better idea IMHO would be to allow int(x, None) to mean to use prefixes or even a different function, like parse_int. I think e suffixes have long implied float values and I see little benefit in allowing them here. ASIDE: int accepts a prefix when it matches the base you're parsing and rejects invalid prefixes. Most of the time. >>> int('0x111', 16) 273 >>> int('0b111', 2) 7 >>> int('0x111', 2) Traceback (most recent call last): File "", line 1, in ValueError: invalid literal for int() with base 2: '0x111' >>> int('0b111', 16) 45329 The last case might be surprising for a moment but hardly wrong, and it's a good illustration of the problem of trying to guess what things mean. --- Bruce -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron3200 at gmail.com Thu Feb 6 21:09:34 2014 From: ron3200 at gmail.com (Ron Adam) Date: Thu, 06 Feb 2014 14:09:34 -0600 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: On 02/06/2014 05:14 AM, Nick Coghlan wrote: > As far as what you're proposing goes, is it essentially a way to > declare a function like (spelling out the lambdas fully): > > def S(x): > def _second(y): > def _third(z): > return x(z)(y(z)) > return _third > return _second > > As something much shorter like this: > > def S (x)(y)(z): > return x(z)(y(z)) The examples just had all the inner calls at the top. I don't see a requirement for that. By being able to place the capture/continuation statements anywhere, it opens up a lot of other (and new) possibilities. > The main potential benefit I could see to a construct like that is > that it may allow the more consistent creation of closures that > support pickling, since the outer functions are guaranteed not to have > any side effects and to have argument capture as their*only* > significant state. This means that you could take the inner function, > pickle it along with its closure variables and reconstruct that at the > far end, only relying on the name of the outer function. Would the calls need to be at the top for that? Another use is as auto_start generators that don't need a next() call to start. def continue auto_gen: # do pre-setup stuff. ... :from(a, b, c) # get initial values. # rest of generator with normal yields. ... Or maybe something like... def continue select: :from(pick_mode) if (mode == mode_a): :from(cmd, *args) ... do stuff return cmd(*args) else: :from(cmd, a, b, c) ... do other stuff return cmd(a, b, c) I think the ability to put them where they are needed is an important part of the suggestions. Cheers, Ron From antony.lee at berkeley.edu Thu Feb 6 22:37:50 2014 From: antony.lee at berkeley.edu (Antony Lee) Date: Thu, 6 Feb 2014 13:37:50 -0800 Subject: [Python-ideas] Making sys.std* thread-local Message-ID: The recent discussion about lexical vs. dynamic scoping reminded me of the following issue, that plagues e.g. the new contextlib.redirect_stdout context manager: the standard file-like objects sys.std* are global instead of thread-local, making their manipulation unsafe in multithreaded programs (this issue is related to lexical vs dynamic scoping because in a language with (optional) dynamic scoping, these objects can be made dynamically scoped, thus solving the issue). Of course, changing sys.std* to being thread-local would be backwards incompatible, but perhaps some new objects, e.g. sys.thread_local_std*, could be added to sys, with print() and related functions using sys.thread_local_std* if it is set and sys.std* otherwise. Antony -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Thu Feb 6 22:41:45 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 06 Feb 2014 16:41:45 -0500 Subject: [Python-ideas] int('0x3241fca1') In-Reply-To: References: Message-ID: On 2/6/2014 5:24 AM, Ram Rachum wrote: > What do you think about letting the `int` constructor automatically > understand the number type without specifying base if given a prefix, so > int('0x3414fa') would immediately work without specifying a base of 16? In addition to int(string, 0): >>> number = '0x3414fa' >>> eval(number) 3413242 -- Terry Jan Reedy From python at mrabarnett.plus.com Thu Feb 6 23:04:13 2014 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 06 Feb 2014 22:04:13 +0000 Subject: [Python-ideas] int('0x3241fca1') In-Reply-To: References: Message-ID: <52F406DD.6060405@mrabarnett.plus.com> On 2014-02-06 21:41, Terry Reedy wrote: > On 2/6/2014 5:24 AM, Ram Rachum wrote: >> What do you think about letting the `int` constructor automatically >> understand the number type without specifying base if given a prefix, so >> int('0x3414fa') would immediately work without specifying a base of 16? > > In addition to int(string, 0): > > >>> number = '0x3414fa' > >>> eval(number) > 3413242 > The disadvantage is that it'll evaluate (run) anything, so it's unsafe in the general case. From ram.rachum at gmail.com Fri Feb 7 00:58:05 2014 From: ram.rachum at gmail.com (Ram Rachum) Date: Thu, 6 Feb 2014 15:58:05 -0800 (PST) Subject: [Python-ideas] Question about `list.insert` Message-ID: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> Hi, I'm curious. If I append an item to a list from the left using `list.insert`, will Python always move the entire list one item to the right (which can be super-slow) or will it check first to see whether it can just allocate more memory to the left of the list and put the item there, saving a lot of resources? Thanks, Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ram.rachum at gmail.com Fri Feb 7 00:58:49 2014 From: ram.rachum at gmail.com (Ram Rachum) Date: Thu, 6 Feb 2014 15:58:49 -0800 (PST) Subject: [Python-ideas] Question about `list.insert` In-Reply-To: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> References: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> Message-ID: <49931276-ddee-44b2-9de3-992110a1e841@googlegroups.com> Oops, my mistake, I meant to post in python-list but accidentally posted here because it was in my favorites. Sorry, I'll post there. On Friday, February 7, 2014 1:58:05 AM UTC+2, Ram Rachum wrote: > > Hi, > > I'm curious. If I append an item to a list from the left using > `list.insert`, will Python always move the entire list one item to the > right (which can be super-slow) or will it check first to see whether it > can just allocate more memory to the left of the list and put the item > there, saving a lot of resources? > > > Thanks, > Ram. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.hoelzl at posteo.de Fri Feb 7 00:41:11 2014 From: robert.hoelzl at posteo.de (Robert =?utf-8?b?SMO2bHps?=) Date: Thu, 6 Feb 2014 23:41:11 +0000 (UTC) Subject: [Python-ideas] Simplifying .format() based string interpolation Message-ID: Hello All, I REALLY like str.format() from PEP3101 since it is simple to use, powerful and extendable. Especially I like the possibility to use named fields for two reasons: readability AND it is less error prone. i.E. compare "{name} is {value} ({value:X} in hex)" to "{} is {} ({1:X} in hex)". In a lot of cases all fields are identical with variables from the local/global name space. This means in the above example one had to write: "{name} is {value} ({value:X} in hex)".format(name=name, value=value) But such statements are violating the DRY principles! How about introducing new string tokens beginning with 'f', which tells the parser to take over that job. In this case one simply had to write: f"{name} is {value} ({value:X} in hex)" and the parser would replace this by? "{name} is {value} ({value:X} in hex)".format( **collections.ChainMap(locals(), globals()) ) Would be glad to hear your thoughts, Robert From ben+python at benfinney.id.au Fri Feb 7 01:04:01 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 07 Feb 2014 11:04:01 +1100 Subject: [Python-ideas] Question about `list.insert` References: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> Message-ID: <85d2izeqku.fsf@benfinney.id.au> Ram Rachum writes: > I'm curious. That kind of question is better on the general Python discussion forum . The ?python-ideas? forum is for discussing ideas to *change* Python behaviour in future versions. -- \ ?Read not to contradict and confute, nor to believe and take | `\ for granted ? but to weigh and consider.? ?Francis Bacon | _o__) | Ben Finney From ram.rachum at gmail.com Fri Feb 7 01:10:02 2014 From: ram.rachum at gmail.com (Ram Rachum) Date: Thu, 6 Feb 2014 16:10:02 -0800 (PST) Subject: [Python-ideas] sentinel_exception argument to `iter` Message-ID: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> Hi, This time I'm posting on the right list :) Sorry for the mistake in the last thread. `iter` has a very cool `sentinel` argument. I suggest an additional argument `sentinel_exception`; when it's supplied, instead of waiting for a sentinel value, we wait for a sentinel exception to be raised, and then the iteration is finished. This'll be useful to construct things like this: my_iterator = iter(my_deque.popleft, IndexError) What do you think? Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From yselivanov.ml at gmail.com Fri Feb 7 01:14:26 2014 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Thu, 06 Feb 2014 19:14:26 -0500 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> Message-ID: <52F42562.9050207@gmail.com> On 2/6/2014, 7:10 PM, Ram Rachum wrote: > Hi, > > This time I'm posting on the right list :) Sorry for the mistake in the > last thread. > > `iter` has a very cool `sentinel` argument. I suggest an additional > argument `sentinel_exception`; when it's supplied, instead of waiting for a > sentinel value, we wait for a sentinel exception to be raised, and then the > iteration is finished. > > This'll be useful to construct things like this: > > my_iterator = iter(my_deque.popleft, IndexError) > Before starting discussion about the new parameter: how passing 'my_deque.popleft' is supposed to work? Yury From rosuav at gmail.com Fri Feb 7 01:11:09 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Feb 2014 11:11:09 +1100 Subject: [Python-ideas] Simplifying .format() based string interpolation In-Reply-To: References: Message-ID: On Fri, Feb 7, 2014 at 10:41 AM, Robert H?lzl wrote: > How about introducing new string tokens beginning with 'f', > which tells the parser to take over that job. > In this case one simply had to write: > > f"{name} is {value} ({value:X} in hex)" > > and the parser would replace this by? > > "{name} is {value} ({value:X} in hex)".format( > **collections.ChainMap(locals(), globals()) ) PHP programmers would love it. Subsequent maintainers would hate it. It's way too magical for my liking. ChrisA From ram at rachum.com Fri Feb 7 01:17:43 2014 From: ram at rachum.com (Ram Rachum) Date: Fri, 7 Feb 2014 02:17:43 +0200 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: <52F42562.9050207@gmail.com> References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <52F42562.9050207@gmail.com> Message-ID: Looks like someone needs to read the `iter` docs :) On Fri, Feb 7, 2014 at 2:14 AM, Yury Selivanov wrote: > > On 2/6/2014, 7:10 PM, Ram Rachum wrote: > >> Hi, >> >> This time I'm posting on the right list :) Sorry for the mistake in the >> last thread. >> >> `iter` has a very cool `sentinel` argument. I suggest an additional >> argument `sentinel_exception`; when it's supplied, instead of waiting for >> a >> sentinel value, we wait for a sentinel exception to be raised, and then >> the >> iteration is finished. >> >> This'll be useful to construct things like this: >> >> my_iterator = iter(my_deque.popleft, IndexError) >> >> Before starting discussion about the new parameter: > how passing 'my_deque.popleft' is supposed to work? > > Yury > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > -- > > --- You received this message because you are subscribed to a topic in the > Google Groups "python-ideas" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/python-ideas/UCaNfAHkBlQ/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > python-ideas+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yselivanov.ml at gmail.com Fri Feb 7 01:24:47 2014 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Thu, 06 Feb 2014 19:24:47 -0500 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <52F42562.9050207@gmail.com> Message-ID: <52F427CF.2080701@gmail.com> My bad, never used it that way. Since your propose a new parameter, it needs to be a keyword-only one, like this: iter(callback, sentinel_exception=IndexError) Yury On 2/6/2014, 7:17 PM, Ram Rachum wrote: > Looks like someone needs to read the `iter` docs :) > > > On Fri, Feb 7, 2014 at 2:14 AM, Yury Selivanov wrote: > >> On 2/6/2014, 7:10 PM, Ram Rachum wrote: >> >>> Hi, >>> >>> This time I'm posting on the right list :) Sorry for the mistake in the >>> last thread. >>> >>> `iter` has a very cool `sentinel` argument. I suggest an additional >>> argument `sentinel_exception`; when it's supplied, instead of waiting for >>> a >>> sentinel value, we wait for a sentinel exception to be raised, and then >>> the >>> iteration is finished. >>> >>> This'll be useful to construct things like this: >>> >>> my_iterator = iter(my_deque.popleft, IndexError) >>> >>> Before starting discussion about the new parameter: >> how passing 'my_deque.popleft' is supposed to work? >> >> Yury >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> >> -- >> >> --- You received this message because you are subscribed to a topic in the >> Google Groups "python-ideas" group. >> To unsubscribe from this topic, visit https://groups.google.com/d/ >> topic/python-ideas/UCaNfAHkBlQ/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> python-ideas+unsubscribe at googlegroups.com. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From yselivanov.ml at gmail.com Fri Feb 7 01:32:30 2014 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Thu, 06 Feb 2014 19:32:30 -0500 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <52F42562.9050207@gmail.com> Message-ID: <52F4299E.1060802@gmail.com> And, FWIW, take a look at itertools.takewhile. Your particular example would look like itertools.takewhile(lambda el:my_deque, my_deque) On 2/6/2014, 7:17 PM, Ram Rachum wrote: > Looks like someone needs to read the `iter` docs :) > > > On Fri, Feb 7, 2014 at 2:14 AM, Yury Selivanov wrote: > >> On 2/6/2014, 7:10 PM, Ram Rachum wrote: >> >>> Hi, >>> >>> This time I'm posting on the right list :) Sorry for the mistake in the >>> last thread. >>> >>> `iter` has a very cool `sentinel` argument. I suggest an additional >>> argument `sentinel_exception`; when it's supplied, instead of waiting for >>> a >>> sentinel value, we wait for a sentinel exception to be raised, and then >>> the >>> iteration is finished. >>> >>> This'll be useful to construct things like this: >>> >>> my_iterator = iter(my_deque.popleft, IndexError) >>> >>> Before starting discussion about the new parameter: >> how passing 'my_deque.popleft' is supposed to work? >> >> Yury >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> >> -- >> >> --- You received this message because you are subscribed to a topic in the >> Google Groups "python-ideas" group. >> To unsubscribe from this topic, visit https://groups.google.com/d/ >> topic/python-ideas/UCaNfAHkBlQ/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> python-ideas+unsubscribe at googlegroups.com. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From haoyi.sg at gmail.com Fri Feb 7 01:34:10 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Thu, 6 Feb 2014 16:34:10 -0800 Subject: [Python-ideas] Simplifying .format() based string interpolation In-Reply-To: References: Message-ID: It already works! from macropy.string_interp import macros, sA = 10B = 5print s["{A} + {B} = {A + B}"]# 10 + 5 = 15 a, b = 1, 2print s["{a} apple and {b} bananas"]# 1 apple and 2 bananas On Thu, Feb 6, 2014 at 4:11 PM, Chris Angelico wrote: > On Fri, Feb 7, 2014 at 10:41 AM, Robert H?lzl > wrote: > > How about introducing new string tokens beginning with 'f', > > which tells the parser to take over that job. > > In this case one simply had to write: > > > > f"{name} is {value} ({value:X} in hex)" > > > > and the parser would replace this by? > > > > "{name} is {value} ({value:X} in hex)".format( > > **collections.ChainMap(locals(), globals()) ) > > PHP programmers would love it. Subsequent maintainers would hate it. > > It's way too magical for my liking. > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Fri Feb 7 02:00:13 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 06 Feb 2014 20:00:13 -0500 Subject: [Python-ideas] int('0x3241fca1') In-Reply-To: <52F406DD.6060405@mrabarnett.plus.com> References: <52F406DD.6060405@mrabarnett.plus.com> Message-ID: On 2/6/2014 5:04 PM, MRAB wrote: > On 2014-02-06 21:41, Terry Reedy wrote: >> On 2/6/2014 5:24 AM, Ram Rachum wrote: >>> What do you think about letting the `int` constructor automatically >>> understand the number type without specifying base if given a prefix, so >>> int('0x3414fa') would immediately work without specifying a base of 16? >> >> In addition to int(string, 0): >> >> >>> number = '0x3414fa' >> >>> eval(number) >> 3413242 >> > The disadvantage is that it'll evaluate (run) anything, so it's unsafe > in the general case. To evaluate any number expression, but only number expressions, but be safe against untrusted input, one should use ast.literal_eval, and then test that the result is a number (either with numbers.Number or adding 0 import ast from numbers import Number def get_number(prompt = "Input a number: "): s = input(prompt) try: x = ast.literal_eval(s) # raise ValueError on failure if not isinstance(x, Number): raise ValueError() except ValueError: raise ValueError("'{}' is not a Python number literal".format(s)) from None return x -- Terry Jan Reedy From robert.hoelzl at posteo.de Fri Feb 7 02:24:32 2014 From: robert.hoelzl at posteo.de (Robert =?utf-8?b?SMO2bHps?=) Date: Fri, 7 Feb 2014 01:24:32 +0000 (UTC) Subject: [Python-ideas] Simplifying .format() based string interpolation References: Message-ID: Chris Angelico writes: > On Fri, Feb 7, 2014 at 10:41 AM, Robert H?lzl posteo.de> wrote: > > How about introducing new string tokens beginning with 'f', > > which tells the parser to take over that job. > > In this case one simply had to write: > > > > f"{name} is {value} ({value:X} in hex)" > > > > and the parser would replace this by? > > > > "{name} is {value} ({value:X} in hex)".format( > > **collections.ChainMap(locals(), globals()) ) > > PHP programmers would love it. Subsequent maintainers would hate it. In my Optinion the bad thing with PHP is that you can create fields that contain complex expressions (even whole programs). But the nice thing of .format are its restrictions: it allows only getattr() and getitem() operations... > > It's way too magical for my liking. mmh; why magical? - it is explicit (the string is marked by 'f') - it is side effect free - does not change the behaviour of an existing feature From robert.hoelzl at posteo.de Fri Feb 7 02:32:44 2014 From: robert.hoelzl at posteo.de (Robert =?utf-8?b?SMO2bHps?=) Date: Fri, 7 Feb 2014 01:32:44 +0000 (UTC) Subject: [Python-ideas] Simplifying .format() based string interpolation References: Message-ID: Haoyi Li writes: > > It already works! > > > from macropy.string_interp import macros, s > A = 10 > B = 5 > print s["{A} + {B} = {A + B}"] > # 10 + 5 = 15 I checked out macropy and seems to be really funny. It is kind a bringing LISP (which all its power to modify the language by itself) to Python. But nevertheless, this solution is not pythonic (as least as long as the python community did not decide to include macropy and use it as inherent as macros are used in LISP) Furthermore I do NOT want {A+B} to work, since this would be PHP. I would like to use the pythonic .format() Robert From rosuav at gmail.com Fri Feb 7 02:46:28 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Feb 2014 12:46:28 +1100 Subject: [Python-ideas] Simplifying .format() based string interpolation In-Reply-To: References: Message-ID: On Fri, Feb 7, 2014 at 12:24 PM, Robert H?lzl wrote: > mmh; why magical? > - it is explicit (the string is marked by 'f') > - it is side effect free > - does not change the behaviour of an existing feature It looks like a new form of literal, but it's actually a run-time expression evaluator. All the other types of string literal produce, ultimately, the same thing: a string. (Okay, the u"" and b"" prefixes choose between one of two things, but you still simply get one single literal object.) There's no such thing as a "raw string" or a "multi-line string"; there's just a "raw string literal" and a "multi-line string literal". >>> r"asdf\qwer" == """asdf\\qwer""" True But now, what you're suggesting is that f"some string" is a thing that does interpolation. You have something that looks like a literal, but whose value depends majorly on context. No other form of literal is like that, unless you count the way [a,b,c] will depend on the values of a, b, and c - and that's not so much a literal as a piece of syntax that constructs something, as can be seen in dis.dis: >>> def foo(x): a = "string literal" b = r"raw string literal" c = ["list","literal"] d = [a,b,c] return d >>> dis.dis(foo) 2 0 LOAD_CONST 1 ('string literal') 3 STORE_FAST 1 (a) 3 6 LOAD_CONST 2 ('raw string literal') 9 STORE_FAST 2 (b) 4 12 LOAD_CONST 3 ('list') 15 LOAD_CONST 4 ('literal') 18 BUILD_LIST 2 21 STORE_FAST 3 (c) 5 24 LOAD_FAST 1 (a) 27 LOAD_FAST 2 (b) 30 LOAD_FAST 3 (c) 33 BUILD_LIST 3 36 STORE_FAST 4 (d) 6 39 LOAD_FAST 4 (d) 42 RETURN_VALUE Technically there's no "list literal" syntax, only a convenient form for constructing a list at run-time. What you're doing here looks like a string literal, but isn't a literal at all. (Note that tuples do kinda have a literal syntax. If you replace the square brackets with round ones, you'll find that the all-literals tuple gets stored as a const, same as the strings are. But that's an optimization that works only when everything's literals, which - kinda by definition - your f'' string won't be.) ChrisA From ethan at stoneleaf.us Fri Feb 7 01:56:02 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 06 Feb 2014 16:56:02 -0800 Subject: [Python-ideas] Simplifying .format() based string interpolation In-Reply-To: References: Message-ID: <52F42F22.7000806@stoneleaf.us> On 02/06/2014 03:41 PM, Robert H?lzl wrote: > > How about introducing new string tokens beginning with 'f', > which tells the parser to take over that job. See https://mail.python.org/pipermail/python-ideas/2013-June/021466.html for a similar discussion. -- ~Ethan~ From tjreedy at udel.edu Fri Feb 7 03:36:19 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 06 Feb 2014 21:36:19 -0500 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> Message-ID: On 2/6/2014 7:10 PM, Ram Rachum wrote: > `iter` has a very cool `sentinel` argument. I suggest an additional > argument `sentinel_exception`; when it's supplied, instead of waiting > for a sentinel value, we wait for a sentinel exception to be raised, and > then the iteration is finished. > > This'll be useful to construct things like this: > > my_iterator = iter(my_deque.popleft, IndexError) > > What do you think? I think this would be a great idea if simplified to reuse the current parameter. It can work in Python because exceptions are objects like anything else and can be passed as arguments. No new parameter is needed. We only need to add 'or raises' to the two-parameter iter definition "In the second form, the callable is called until it returns the sentinel." to get "In the second form, the callable is called until it returns or raises the sentinel." With this version of the proposal, your pop example would work as written. The two-parameter form return a 'callable_iterator' object. Its __next__ method in Python might look like (untested) def __next__(self): x = self.func() if x == self.sentinel: raise StopIteration else: return x The only change needed for the added behavior is to wrap the function call and change an exception matching the sentinel to StopIteration. def __next__(self): try: x = self.func() except Exception as exc: if isinstance(exc, self.sentinel): raise StopIteration from None if x == self.sentinel: raise StopIteration else: return x I do something similar to this in my custom function tester and it is really handy to be able to pass in an expected exception as the expected 'output'. For example, my 'test table' for an int divide function might contain these input and expected output pairs: (2,0), ZeroDevisionError (2,1), 2 (2,2), 1. I consider the threat to backward compatibility, because of the added test for exceptions, theoretical rather than actual. It is very rare to write a function that returns an exception, rarer still to write one that can also raise an instance of the same exception class. Also, using an exception-returning function in two-parameter iter is tricky because exceptions inherit the default equality comparison of only equal by identity. >>> ValueError('high') == ValueError('high') False >>> e=ValueError('high') >>> e == e True The following tested example of code that would be broken by the proposal could well be the first ever written. Warning: it is so twisted and awful that it might to read it. --- from random import random e=ValueError('high value') def crazy(): x = random() if x > .999: return e elif x < .001: raise ValueError('low value') else: return x try: for x in iter(crazy, e): pass else: print(e.args[0]) except ValueError as exc: print(exc.args[0]) # prints 'high value' or 'low value' in less than a second -- Terry Jan Reedy From rosuav at gmail.com Fri Feb 7 03:45:50 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Feb 2014 13:45:50 +1100 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> Message-ID: On Fri, Feb 7, 2014 at 1:36 PM, Terry Reedy wrote: > def __next__(self): > try: > x = self.func() > except Exception as exc: > if isinstance(exc, self.sentinel): > raise StopIteration from None > if x == self.sentinel: > raise StopIteration > else: > return x Forgive me if this is a stupid question, but wouldn't this suppress any other thrown exception? I'm looking for a bare 'raise' inside the except block, such that any exception is guaranteed to raise something, but if it's a subclass of self.sentinel, it raises StopIteration instead. ChrisA From yselivanov.ml at gmail.com Fri Feb 7 04:01:41 2014 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Thu, 06 Feb 2014 22:01:41 -0500 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> Message-ID: <52F44C95.4010104@gmail.com> On 2/6/2014, 9:36 PM, Terry Reedy wrote: > I think this would be a great idea if simplified to reuse the current > parameter. It can work in Python because exceptions are objects like > anything else and can be passed as arguments. No new parameter is needed. What will the following code print: d = deque((42, IndexError, 'spam')) print(list(iter(d.popleft, IndexError))) ? Yury From rosuav at gmail.com Fri Feb 7 04:11:05 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Feb 2014 14:11:05 +1100 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: <52F44C95.4010104@gmail.com> References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <52F44C95.4010104@gmail.com> Message-ID: On Fri, Feb 7, 2014 at 2:01 PM, Yury Selivanov wrote: > > On 2/6/2014, 9:36 PM, Terry Reedy wrote: >> >> I think this would be a great idea if simplified to reuse the current >> parameter. It can work in Python because exceptions are objects like >> anything else and can be passed as arguments. No new parameter is needed. > > What will the following code print: > > d = deque((42, IndexError, 'spam')) > print(list(iter(d.popleft, IndexError))) > > ? Presumably it would stop once it reaches the IndexError, exactly per current behaviour, and print [42]. So you can't depend on it actually having caught IndexError, any more than you can depend on it actually having found the element: >>> def foo(): global idx idx+=1 if idx==3: raise StopIteration() return idx*10 >>> idx=0 >>> print(list(iter(foo, 20))) [10] >>> idx=0 >>> print(list(iter(foo, 50))) [10, 20] It'll stop at any of three conditions: a raised StopIteration from the function, a returned value equal to the second argument, or a raised exception that's a subclass of the second argument. All three will be folded into the same result: StopIteration. ChrisA From tjreedy at udel.edu Fri Feb 7 05:09:20 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 06 Feb 2014 23:09:20 -0500 Subject: [Python-ideas] Simplifying .format() based string interpolation In-Reply-To: References: Message-ID: On 2/6/2014 8:46 PM, Chris Angelico wrote: > Technically there's no "list literal" syntax, only a convenient form > for constructing a list at run-time. There is literally no 'list literal' syntax. Number, string, and byte literals are discussed under Lexical Analysis in http://docs.python.org/3/reference/lexical_analysis.html#literals List, set, and dict displays are discussed under Expressions in http://docs.python.org/3/reference/expressions.html#displays-for-lists-sets-and-dictionaries and tuples in http://docs.python.org/3/reference/expressions.html#expression-lists -- Terry Jan Reedy From tjreedy at udel.edu Fri Feb 7 05:15:14 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 06 Feb 2014 23:15:14 -0500 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> Message-ID: On 2/6/2014 9:45 PM, Chris Angelico wrote: > On Fri, Feb 7, 2014 at 1:36 PM, Terry Reedy wrote: >> def __next__(self): >> try: >> x = self.func() >> except Exception as exc: >> if isinstance(exc, self.sentinel): >> raise StopIteration from None else: raise >> if x == self.sentinel: >> raise StopIteration >> else: >> return x > > Forgive me if this is a stupid question, but wouldn't this suppress > any other thrown exception? I'm looking for a bare 'raise' inside the > except block, which should be there. As I said, but you snipped, '(untested)' > such that any exception is guaranteed to raise > something, but if it's a subclass of self.sentinel, it raises > StopIteration instead. -- Terry Jan Reedy From rosuav at gmail.com Fri Feb 7 05:47:48 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Feb 2014 15:47:48 +1100 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> Message-ID: On Fri, Feb 7, 2014 at 3:15 PM, Terry Reedy wrote: >> Forgive me if this is a stupid question, but wouldn't this suppress >> any other thrown exception? I'm looking for a bare 'raise' inside the >> except block, > > > which should be there. As I said, but you snipped, '(untested)' Ah okay. Makes sense :) Wasn't sure if it was intentional, or if there was some other magic that would do what was wanted. ChrisA From tjreedy at udel.edu Fri Feb 7 07:03:16 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 07 Feb 2014 01:03:16 -0500 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> Message-ID: On 2/6/2014 11:15 PM, Terry Reedy wrote: >> On Fri, Feb 7, 2014 at 1:36 PM, Terry Reedy >>> def __next__(self): >>> try: >>> x = self.func() >>> except Exception as exc: >>> if isinstance(exc, self.sentinel): >>> raise StopIteration from None > else: > raise I just realized that the above is unnecessarily complicated because the expression that follows 'except' is not limited to a builtin exception class name or tuple thereof. (I have never before had reason to dynamically determine the exception to be caught.) So, using a third parameter, replace the 5 lines with 2. except self.stop_exception: raise StopIteration from None >>> if x == self.sentinel: >>> raise StopIteration >>> else: >>> return x -- Terry Jan Reedy From tjreedy at udel.edu Fri Feb 7 07:05:33 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 07 Feb 2014 01:05:33 -0500 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: <52F44C95.4010104@gmail.com> References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <52F44C95.4010104@gmail.com> Message-ID: On 2/6/2014 10:01 PM, Yury Selivanov wrote: > > On 2/6/2014, 9:36 PM, Terry Reedy wrote: >> I think this would be a great idea if simplified to reuse the current >> parameter. It can work in Python because exceptions are objects like >> anything else and can be passed as arguments. No new parameter is needed. > What will the following code print: > > d = deque((42, IndexError, 'spam')) > print(list(iter(d.popleft, IndexError))) As Chris said, 42. To change current behavior before the function raises an exception, the comparison of each function return to the sentinel would have to be changed (or eliminated). My proposal does not do that. It only changes behavior when there is an exception and iter has been told that an exception is to be treated as having the same 'I am finished' meaning as StopIteration. As you showed, it is easy to construct callables that might return an exception before raising it from finite collections with a prev or next method (whether destructive or not). >>> list(iter(['spam', IndexError, 42].pop, IndexError)) [42] >>> list(iter({'spam', KeyError, 42}.pop, KeyError)) [42, 'spam'] # or [42] or ['spam'], depending on hashing For these example, I guess Ram is right in suggesting a 3rd parameter. I would, however, just call it something like 'exception' or 'stop_iter'. That would make the description "In the second form, the callable is called until it returns the sentinel or raises an instance of stop_iter" If sentinel is omitted, then the callable is iterated until 'completion' The signature is a bit awkward because 'sentinel' is positional-only and optional without a default. The C code must use the equivalent of *args and switch on the number of args passed. So the new param would probably have to be keyword-only. I remain in favor of the proposal. -- Terry Jan Reedy From rosuav at gmail.com Fri Feb 7 07:41:32 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Feb 2014 17:41:32 +1100 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <52F44C95.4010104@gmail.com> Message-ID: On Fri, Feb 7, 2014 at 5:05 PM, Terry Reedy wrote: > As you showed, it is easy to construct callables that might return an > exception before raising it from finite collections with a prev or next > method (whether destructive or not). > >>>> list(iter(['spam', IndexError, 42].pop, IndexError)) > [42] >>>> list(iter({'spam', KeyError, 42}.pop, KeyError)) > [42, 'spam'] # or [42] or ['spam'], depending on hashing > > For these example, I guess Ram is right in suggesting a 3rd parameter. I > would, however, just call it something like 'exception' or 'stop_iter'. > That would make the description I honestly wouldn't worry. How often, in production code, will you iterate over something that might return an exception, AND be testing for the raising of that same exception? Or the converse - how often would you be in a position to raise the thing you might want to return, and be annoyed that the raised exception gets squashed into StopIteration? Don't complicate a nice simple API for the sake of that. If you're concerned about security implications of someone manipulating the return values and chopping something off, then just write your own generator: def safe_iter(func, sentinel): try: while True: yield func() except sentinel: pass This will stop when the exception's raised, but not when it's returned. The unusual case can be covered with so little code that the API can afford to ignore it, imo. ChrisA From abarnert at yahoo.com Fri Feb 7 07:52:00 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 6 Feb 2014 22:52:00 -0800 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> Message-ID: <36ABE145-12ED-4A87-B3C0-5FBDEAE36699@yahoo.com> On Feb 6, 2014, at 22:03, Terry Reedy wrote: > On 2/6/2014 11:15 PM, Terry Reedy wrote: >>> On Fri, Feb 7, 2014 at 1:36 PM, Terry Reedy > >>>> def __next__(self): >>>> try: >>>> x = self.func() >>>> except Exception as exc: >>>> if isinstance(exc, self.sentinel): >>>> raise StopIteration from None >> else: >> raise > > I just realized that the above is unnecessarily complicated because the expression that follows 'except' is not limited to a builtin exception class name or tuple thereof. (I have never before had reason to dynamically determine the exception to be caught.) So, using a third parameter, replace the 5 lines with 2. > > except self.stop_exception: > raise StopIteration from None Except that you don't have a stop_exception, you have a sentinel, which can be either an object or an exception type. I'm actually not sure whether it's legal to use, say, 0 or "" as the except expression. In recent 3.4 builds, it seems to be accepted, and to never catch anything. So, if that's guaranteed by the language, it's just a simple typo to fix and your simplified implementation works perfectly. > >>>> if x == self.sentinel: >>>> raise StopIteration >>>> else: >>>> return x > > -- > Terry Jan Reedy > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From abarnert at yahoo.com Fri Feb 7 07:55:34 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 6 Feb 2014 22:55:34 -0800 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: <36ABE145-12ED-4A87-B3C0-5FBDEAE36699@yahoo.com> References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <36ABE145-12ED-4A87-B3C0-5FBDEAE36699@yahoo.com> Message-ID: On Feb 6, 2014, at 22:52, Andrew Barnert wrote: > On Feb 6, 2014, at 22:03, Terry Reedy wrote: > >> On 2/6/2014 11:15 PM, Terry Reedy wrote: >>>> On Fri, Feb 7, 2014 at 1:36 PM, Terry Reedy >> >>>>> def __next__(self): >>>>> try: >>>>> x = self.func() >>>>> except Exception as exc: >>>>> if isinstance(exc, self.sentinel): >>>>> raise StopIteration from None >>> else: >>> raise >> >> I just realized that the above is unnecessarily complicated because the expression that follows 'except' is not limited to a builtin exception class name or tuple thereof. (I have never before had reason to dynamically determine the exception to be caught.) So, using a third parameter, replace the 5 lines with 2. >> >> except self.stop_exception: >> raise StopIteration from None > > Except that you don't have a stop_exception, you have a sentinel, which can be either an object or an exception type. > > I'm actually not sure whether it's legal to use, say, 0 or "" as the except expression. In recent 3.4 builds, it seems to be accepted, and to never catch anything. So, if that's guaranteed by the language, it's just a simple typo to fix and your simplified implementation works perfectly. Reading the docs, it seems like it ought to be ok. In 8.4, it just says: 'For an except clause with an expression, the expression is evaluated, and the clause matches the exception if the resulting object is "compatible" with the exception. An object is compatible with an exception if it is the class or a base class of the exception object or a tuple containing an item compatible with the exception.' So, it seems like 0 is a perfectly valid except expression, which can be checked for compatibility with any exception and will never match. Which is perfect. From rosuav at gmail.com Fri Feb 7 07:59:50 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Feb 2014 17:59:50 +1100 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: <36ABE145-12ED-4A87-B3C0-5FBDEAE36699@yahoo.com> References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <36ABE145-12ED-4A87-B3C0-5FBDEAE36699@yahoo.com> Message-ID: On Fri, Feb 7, 2014 at 5:52 PM, Andrew Barnert wrote: > I'm actually not sure whether it's legal to use, say, 0 or "" as the except expression. In recent 3.4 builds, it seems to be accepted, and to never catch anything. So, if that's guaranteed by the language, it's just a simple typo to fix and your simplified implementation works perfectly. > In 3.4b2: >>> def f(): raise StopIteration >>> try: f() except "": print("Blank exception caught") Traceback (most recent call last): File "", line 2, in f() File "", line 2, in f raise StopIteration StopIteration During handling of the above exception, another exception occurred: Traceback (most recent call last): File "", line 3, in except "": TypeError: catching classes that do not inherit from BaseException is not allowed It doesn't bomb until something gets raised. Is that changed in a newer build? (This is the newest I have on here.) ChrisA From denis.spir at gmail.com Fri Feb 7 08:28:04 2014 From: denis.spir at gmail.com (spir) Date: Fri, 07 Feb 2014 08:28:04 +0100 Subject: [Python-ideas] Simplifying .format() based string interpolation In-Reply-To: References: Message-ID: <52F48B04.6050105@gmail.com> On 02/07/2014 02:46 AM, Chris Angelico wrote: > On Fri, Feb 7, 2014 at 12:24 PM, Robert H?lzl wrote: >> mmh; why magical? >> - it is explicit (the string is marked by 'f') >> - it is side effect free >> - does not change the behaviour of an existing feature > > It looks like a new form of literal, but it's actually a run-time > expression evaluator. All the other types of string literal produce, > ultimately, the same thing: a string. (Okay, the u"" and b"" prefixes > choose between one of two things, but you still simply get one single > literal object.) There's no such thing as a "raw string" or a > "multi-line string"; there's just a "raw string literal" and a > "multi-line string literal". > >>>> r"asdf\qwer" == """asdf\\qwer""" > True > > But now, what you're suggesting is that f"some string" is a thing that > does interpolation. You have something that looks like a literal, but > whose value depends majorly on context. No other form of literal is > like that, unless you count the way [a,b,c] will depend on the values > of a, b, and c - and that's not so much a literal as a piece of syntax > that constructs something, as can be seen in dis.dis: > >>>> def foo(x): > a = "string literal" > b = r"raw string literal" > c = ["list","literal"] > d = [a,b,c] > return d > >>>> dis.dis(foo) > 2 0 LOAD_CONST 1 ('string literal') > 3 STORE_FAST 1 (a) > > 3 6 LOAD_CONST 2 ('raw string literal') > 9 STORE_FAST 2 (b) > > 4 12 LOAD_CONST 3 ('list') > 15 LOAD_CONST 4 ('literal') > 18 BUILD_LIST 2 > 21 STORE_FAST 3 (c) > > 5 24 LOAD_FAST 1 (a) > 27 LOAD_FAST 2 (b) > 30 LOAD_FAST 3 (c) > 33 BUILD_LIST 3 > 36 STORE_FAST 4 (d) > > 6 39 LOAD_FAST 4 (d) > 42 RETURN_VALUE > > Technically there's no "list literal" syntax, only a convenient form > for constructing a list at run-time. What you're doing here looks like > a string literal, but isn't a literal at all. > > (Note that tuples do kinda have a literal syntax. If you replace the > square brackets with round ones, you'll find that the all-literals > tuple gets stored as a const, same as the strings are. But that's an > optimization that works only when everything's literals, which - kinda > by definition - your f'' string won't be.) Well, in fact all your argumentation just supports the proposal, doesn't it? There would be string constants and string variables: "Hello, world!" "Hello, {username}!" just like list constants and list variables: [1,2,3] [a,b,c] So, where's the *actual* problem? It's not as if it were implicit or otherwise hidden. And we would not even need an 'f' prefix if it were not for backward compatibility (or maybe the parser could deal with that?); various languages just use this format (among others, Cobra, highly influenced by Python) and have just abandoned all this complicated variable string syntax noise. Would be very pythonic, in my view (but maybe I don't get what pythonic means for this matter; but surely it does not mean unneeded complication). d From steve at pearwood.info Fri Feb 7 09:28:54 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 7 Feb 2014 19:28:54 +1100 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> Message-ID: <20140207082853.GA1313@ando> On Thu, Feb 06, 2014 at 09:36:19PM -0500, Terry Reedy wrote: > On 2/6/2014 7:10 PM, Ram Rachum wrote: > > >`iter` has a very cool `sentinel` argument. I suggest an additional > >argument `sentinel_exception`; when it's supplied, instead of waiting > >for a sentinel value, we wait for a sentinel exception to be raised, and > >then the iteration is finished. > > > >This'll be useful to construct things like this: > > > > my_iterator = iter(my_deque.popleft, IndexError) > > > >What do you think? > > I think this would be a great idea if simplified to reuse the current > parameter. That would be a backwards-incompatible change, for exactly the reason you give below: > It can work in Python because exceptions are objects like > anything else and can be passed as arguments. Right. And there is a big difference between *returning* an exception and *raising* an exception, which is why a new parameter (or a new function) is required. A function might legitimately return exception objects for some reason: exceptions_to_be_tested = iter( [IndexError(msg), ValueError, StopIteration, TypeError] ) def func(): # pre- or post-processing might happen return next(it) for exception in iter(func, StopIteration): # assume the exceptions are caught elsewhere raise exception With the current behaviour, that will raise IndexError and ValueError, then stop. With the suggested change in behaviour, it will raise all four exceptions. We cannot assume that an exception is never a legitimate return result from the callable. "Iterate until this exception is raised" and "iterate until this value is returned" are very different things and it is folly to treat them as the same. [...] > I consider the threat to backward compatibility, because of the added > test for exceptions, theoretical rather than actual. It is very rare to > write a function that returns an exception, Rare or not, I've done it, it's allowed by the language, and it is inappropriate to conflate returning a class or instance with raising an exception. It doesn't matter whether it is rare. It is rare to write: iter(func, ({}, {})) nevertheless it would be poor design to have iter treat tuples of exactly two dicts as a special case. Exceptions are first-class values like strings, ints, and tuples containing exactly two dicts. They should be treated exactly the same as any other first-class value. -- Steven From tjreedy at udel.edu Fri Feb 7 09:36:51 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 07 Feb 2014 03:36:51 -0500 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: <36ABE145-12ED-4A87-B3C0-5FBDEAE36699@yahoo.com> References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <36ABE145-12ED-4A87-B3C0-5FBDEAE36699@yahoo.com> Message-ID: On 2/7/2014 1:52 AM, Andrew Barnert wrote: > On Feb 6, 2014, at 22:03, Terry Reedy wrote: > >> On 2/6/2014 11:15 PM, Terry Reedy wrote: >>>> On Fri, Feb 7, 2014 at 1:36 PM, Terry Reedy >> >>>>> def __next__(self): >>>>> try: >>>>> x = self.func() >>>>> except Exception as exc: >>>>> if isinstance(exc, self.sentinel): >>>>> raise StopIteration from None >>> else: >>> raise >> >> I just realized that the above is unnecessarily complicated because the expression that follows 'except' is not limited to a builtin exception class name or tuple thereof. (I have never before had reason to dynamically determine the exception to be caught.) So, using a third parameter, replace the 5 lines with 2. >> >> except self.stop_exception: >> raise StopIteration from None > > Except that you don't have a stop_exception, you have a sentinel, which can be either an object or an exception type. I wrote the above with the idea that there would be a third parameter to provide an exception. It would have to have a private default like class _NoException(Exception): pass > I'm actually not sure whether it's legal to use, say, 0 or "" as the except expression. In recent 3.4 builds, it seems to be accepted, and to never catch anything. So, if that's guaranteed by the language, it's just a simple typo to fix and your simplified implementation works perfectly. Easy to test. The result is that a non-exception expression is syntactically legal, but not when an exception occurs. >>> try: 3 except 1: 4 3 >>> try: 1/0 except 1: 4 Traceback (most recent call last): File "", line 1, in try: 1/0 ZeroDivisionError: division by zero During handling of the above exception, another exception occurred: Traceback (most recent call last): File "", line 2, in except 1: 4 TypeError: catching classes that do not inherit from BaseException is not allowed -- Terry Jan Reedy From steve at pearwood.info Fri Feb 7 09:39:19 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 7 Feb 2014 19:39:19 +1100 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <52F44C95.4010104@gmail.com> Message-ID: <20140207083919.GB1313@ando> On Fri, Feb 07, 2014 at 05:41:32PM +1100, Chris Angelico wrote: > I honestly wouldn't worry. How often, in production code, will you > iterate over something that might return an exception, AND be testing > for the raising of that same exception? Or the converse - how often > would you be in a position to raise the thing you might want to > return, and be annoyed that the raised exception gets squashed into > StopIteration? Doesn't matter how rare it is. Doesn't matter if nobody has ever written production that does it. Or for that matter, if nobody has ever written *non-production* code that does it. "Special cases aren't special enough to break the rules." Exceptions are values just like strings and lists and floats. Probably nobody has ever written this particular piece of code: iter(func, "NOBODY expects the Spanish Inquisition!!!") but would you think it is okay to treat that particular sentinel value differently from every other sentinel value? > Don't complicate a nice simple API for the sake of > that. Your idea of simple is not the same as mine. The current behaviour "the callable is called until it returns the sentinel" is simpler than the proposed behaviour: "the callable is called until it returns the sentinel, unless the sentinel is an exception instance or class, in which case the callable is called until it raises that exception, or one which is compatible with it" This is an ugly API that violates the Zen of Python. I think that is why Raymond has listed this as a recipe rather than modify the function to behave as suggested: http://code.activestate.com/recipes/577155 -- Steven From tjreedy at udel.edu Fri Feb 7 09:41:57 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 07 Feb 2014 03:41:57 -0500 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <36ABE145-12ED-4A87-B3C0-5FBDEAE36699@yahoo.com> Message-ID: On 2/7/2014 1:59 AM, Chris Angelico wrote: > On Fri, Feb 7, 2014 at 5:52 PM, Andrew Barnert wrote: >> I'm actually not sure whether it's legal to use, say, 0 or "" as the except expression. In recent 3.4 builds, it seems to be accepted, and to never catch anything. So, if that's guaranteed by the language, it's just a simple typo to fix and your simplified implementation works perfectly. >> > > In 3.4b2: > >>>> def f(): > raise StopIteration > >>>> try: > f() > except "": > print("Blank exception caught") > > Traceback (most recent call last): > File "", line 2, in > f() > File "", line 2, in f > raise StopIteration > StopIteration > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): > File "", line 3, in > except "": > TypeError: catching classes that do not inherit from BaseException is > not allowed > > > It doesn't bomb until something gets raised. Is that changed in a > newer build? (This is the newest I have on here.) As of a week ago, no. -- Terry Jan Reedy From tjreedy at udel.edu Fri Feb 7 10:28:14 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 07 Feb 2014 04:28:14 -0500 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: <20140207082853.GA1313@ando> References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <20140207082853.GA1313@ando> Message-ID: On 2/7/2014 3:28 AM, Steven D'Aprano wrote: > On Thu, Feb 06, 2014 at 09:36:19PM -0500, Terry Reedy wrote: >> On 2/6/2014 7:10 PM, Ram Rachum wrote: >> >>> `iter` has a very cool `sentinel` argument. I suggest an additional >>> argument `sentinel_exception`; when it's supplied, instead of waiting >>> for a sentinel value, we wait for a sentinel exception to be raised, and >>> then the iteration is finished. >>> >>> This'll be useful to construct things like this: >>> >>> my_iterator = iter(my_deque.popleft, IndexError) >>> >>> What do you think? >> >> I think this would be a great idea if simplified to reuse the current >> parameter. > > That would be a backwards-incompatible change, for exactly the reason > you give below: In a later message, I reversed myself, in spite of the actual C signature making it a bit messy. Although help says 'iter(callable, sentinel)', the actual signature is iter(*args), with any attempt to pass an arg by keyword raising TypeError: iter() takes no keyword arguments Let n = len(args). Then the code switches as follows: n=0: TypeError: iter expected at least 1 arguments, got 0 n>2: TypeError: iter expected at most 2 arguments, got 3 n=1: Return __iter__ or __getitem__ wrapper. n=2: if callable(args[0]): return callable_iterator(*args), else: TypeError: iter(v, w): v must be callable If the current signature were merely extended, then I believe the new signature would have to be (if possible) iter(*args, *, stop_iter=) But having parameter args[1] (sentinel) be position-only, with no default, while added parameter stop_iter is keyword only, with a (private) default, would be a bit weird. So instead I would suggest making the new signature be iter(iter_or_call, sentinel=, stop_iter=). If sentinel and stop_iter are both default, use current n=1 code, else pass all 3 args to modified callable_iterator that compares sentinel to return values and catches stop_iter exceptions. Either way, the user could choose to only stop on a return value, only stop on an exception, or stop on either with the two values not having to be the same. The only thing that would break is code that depends on a TypeError, but we allow ourselves to do that to extend functions. >> It can work in Python because exceptions are objects like >> anything else and can be passed as arguments. > > Right. And there is a big difference between *returning* an exception > and *raising* an exception, which is why a new parameter (or a new > function) is required. A function might legitimately return exception > objects for some reason: > > exceptions_to_be_tested = iter( > [IndexError(msg), ValueError, StopIteration, TypeError] > ) > > def func(): > # pre- or post-processing might happen > return next(it) Did you mean next(exceptions_to_be_tested) > for exception in iter(func, StopIteration): > # assume the exceptions are caught elsewhere > raise exception > > > With the current behaviour, that will raise IndexError and ValueError, > then stop. With the suggested change in behaviour, it will raise all > four exceptions. No, it would still only raise the first two as it would still stop with the return of StopIteration. But the certainty that people would be confused by the double use of one parameter is reason enough not to do it. > We cannot assume that an exception is never a legitimate return result > from the callable. "Iterate until this exception is raised" and "iterate > until this value is returned" are very different things and it is folly > to treat them as the same. >> [...] >> I consider the threat to backward compatibility, because of the added >> test for exceptions, theoretical rather than actual. It is very rare to >> write a function that returns an exception, While *writing* such a function might be very rare, Yuri showed how easy it is to create such a callable by binding a collection instance to a method. > Rare or not, I've done it, it's allowed by the language, and it is > inappropriate to conflate returning a class or instance with raising an > exception. > > It doesn't matter whether it is rare. It is rare to write: > > iter(func, ({}, {})) > > nevertheless it would be poor design to have iter treat tuples of > exactly two dicts as a special case. > > Exceptions are first-class values like strings, ints, and tuples > containing exactly two dicts. They should be treated exactly the same as > any other first-class value. Agreed. -- Terry Jan Reedy From ncoghlan at gmail.com Fri Feb 7 10:37:08 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 7 Feb 2014 19:37:08 +1000 Subject: [Python-ideas] int('0x3241fca1') In-Reply-To: References: <52F406DD.6060405@mrabarnett.plus.com> Message-ID: On 7 Feb 2014 11:01, "Terry Reedy" wrote: > > On 2/6/2014 5:04 PM, MRAB wrote: >> >> On 2014-02-06 21:41, Terry Reedy wrote: >>> >>> On 2/6/2014 5:24 AM, Ram Rachum wrote: >>>> >>>> What do you think about letting the `int` constructor automatically >>>> understand the number type without specifying base if given a prefix, so >>>> int('0x3414fa') would immediately work without specifying a base of 16? >>> >>> >>> In addition to int(string, 0): >>> >>> >>> number = '0x3414fa' >>> >>> eval(number) >>> 3413242 >>> >> The disadvantage is that it'll evaluate (run) anything, so it's unsafe >> in the general case. > > > To evaluate any number expression, but only number expressions, but be safe against untrusted input, one should use ast.literal_eval, and then test that the result is a number (either with numbers.Number or adding 0 If you want to support non-integers, yes. But in most cases where prefix interpretation is of interest, you will specifically want an integer (so base 0 is the answer), and in other cases, base 10 integer strings are also valid input for the float or Decimal constructor. Cheers, Nick. > > import ast > from numbers import Number > > def get_number(prompt = "Input a number: "): > s = input(prompt) > try: > x = ast.literal_eval(s) # raise ValueError on failure > if not isinstance(x, Number): > raise ValueError() > except ValueError: > raise ValueError("'{}' is not a Python number literal".format(s)) from None > return x > > > -- > Terry Jan Reedy > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Feb 7 10:47:43 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 7 Feb 2014 19:47:43 +1000 Subject: [Python-ideas] Simplifying .format() based string interpolation In-Reply-To: <52F48B04.6050105@gmail.com> References: <52F48B04.6050105@gmail.com> Message-ID: Note that the current spelling of this is just "msg.format_map(locals())". What the proposal is effectively asking for is a new non-string literal type that creates an implicit closure, allowing the names to be referenced without supplying them explicitly. That's a significant new (and hard to understand) feature to replace a fairly simple call. Cheers, Nick. -------------- next part -------------- An HTML attachment was scrubbed... URL: From turnbull at sk.tsukuba.ac.jp Fri Feb 7 10:49:12 2014 From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull) Date: Fri, 07 Feb 2014 18:49:12 +0900 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: <20140207083919.GB1313@ando> References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <52F44C95.4010104@gmail.com> <20140207083919.GB1313@ando> Message-ID: <87k3d7mew7.fsf@uwakimon.sk.tsukuba.ac.jp> Steven D'Aprano writes: > iter(func, "NOBODY expects the Spanish Inquisition!!!") +1 Code Snippet of the Week! -- ???????????????????? ???? ??????? ????? 81+(0)29-853-5091 turnbull at sk.tsukuba.ac.jp http://turnbull.sk.tsukuba.ac.jp/ From ram.rachum at gmail.com Fri Feb 7 11:01:44 2014 From: ram.rachum at gmail.com (Ram Rachum) Date: Fri, 7 Feb 2014 02:01:44 -0800 (PST) Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: <87k3d7mew7.fsf@uwakimon.sk.tsukuba.ac.jp> References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <52F44C95.4010104@gmail.com> <20140207083919.GB1313@ando> <87k3d7mew7.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Friday, February 7, 2014 11:49:12 AM UTC+2, Stephen J. Turnbull wrote: > > Steven D'Aprano writes: > > > iter(func, "NOBODY expects the Spanish Inquisition!!!") > > +1 Code Snippet of the Week! > A++, would run again :) Now back to serious discussion: I do prefer a separate keyword argument rather than reusing `sentinel`. If `sentinel_exception` is too verbose, then an alternative is `exception`, but just not `stop_iter` which is very undescriptive. Also, I'd support having three separate implementations for the `iter` function, one for just sentinel, one for exception, and one for both, so the iteration would be as quick as possible after the iterator was created. Thanks, Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ram.rachum at gmail.com Fri Feb 7 11:14:09 2014 From: ram.rachum at gmail.com (Ram Rachum) Date: Fri, 7 Feb 2014 02:14:09 -0800 (PST) Subject: [Python-ideas] Question about `list.insert` In-Reply-To: <49931276-ddee-44b2-9de3-992110a1e841@googlegroups.com> References: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> <49931276-ddee-44b2-9de3-992110a1e841@googlegroups.com> Message-ID: <6cd218d1-a29e-4f0e-80f6-434142304c79@googlegroups.com> Okay, now that I've posted on python-list, and among the 80% of joke posts in my thread, there was a 20% of posts that assured me that Python's list has no such behavior and that I should use deque, I now ask on Python-ideas: Is there a good reason why `list.insert(whatever, 0)` doesn't opportunistically try to allocate more space at the left side of the list, so as to save the expensive operation of moving all the items? I'm not saying it should reserve space there, just check if that space is available, and if so use it. Is there any reason why not? Thanks, Ram. On Friday, February 7, 2014 1:58:49 AM UTC+2, Ram Rachum wrote: > > Oops, my mistake, I meant to post in python-list but accidentally posted > here because it was in my favorites. Sorry, I'll post there. > > On Friday, February 7, 2014 1:58:05 AM UTC+2, Ram Rachum wrote: >> >> Hi, >> >> I'm curious. If I append an item to a list from the left using >> `list.insert`, will Python always move the entire list one item to the >> right (which can be super-slow) or will it check first to see whether it >> can just allocate more memory to the left of the list and put the item >> there, saving a lot of resources? >> >> >> Thanks, >> Ram. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Fri Feb 7 11:17:23 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 07 Feb 2014 05:17:23 -0500 Subject: [Python-ideas] Simplifying .format() based string interpolation In-Reply-To: <52F48B04.6050105@gmail.com> References: <52F48B04.6050105@gmail.com> Message-ID: On 2/7/2014 2:28 AM, spir wrote: > On 02/07/2014 02:46 AM, Chris Angelico wrote: >> (Note that tuples do kinda have a literal syntax. 'Constant tuples are no different that other constant expressions for immutable objects. Like tuples, ints are immutable. If the fixed value of an int can be computed at compile time, then an implementation may opt to optimize runtime by doing the computation at compile time. This only really helps if it avoids computing the object multiple times. I suppose one could say that '1 + 1' is a kind of literal syntax for 2, but I am not sure this is helpful in this context. Number and string literals are *lexical* constants, as are keywords. They are all described in the chapter on *lexical* analysis of code. Literals are like the base cases of recursion and starting values of iteration. If you try to treat literals as Python runtime expressions, say 1 = int(1), then you get into infinite regression, as then int(1) = int(int(1)), etcetera. > Well, in fact all your argumentation just supports the proposal, doesn't > it? If the argumentation led you to confuse categories, it is defective. -- Terry Jan Reedy From denis.spir at gmail.com Fri Feb 7 11:37:06 2014 From: denis.spir at gmail.com (spir) Date: Fri, 07 Feb 2014 11:37:06 +0100 Subject: [Python-ideas] Question about `list.insert` In-Reply-To: <6cd218d1-a29e-4f0e-80f6-434142304c79@googlegroups.com> References: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> <49931276-ddee-44b2-9de3-992110a1e841@googlegroups.com> <6cd218d1-a29e-4f0e-80f6-434142304c79@googlegroups.com> Message-ID: <52F4B752.8090506@gmail.com> On 02/07/2014 11:14 AM, Ram Rachum wrote: > Okay, now that I've posted on python-list, and among the 80% of joke posts > in my thread, there was a 20% of posts that assured me that Python's list > has no such behavior and that I should use deque, I now ask on > Python-ideas: Is there a good reason why `list.insert(whatever, 0)` doesn't > opportunistically try to allocate more space at the left side of the list, > so as to save the expensive operation of moving all the items? I'm not > saying it should reserve space there, just check if that space is > available, and if so use it. Is there any reason why not? It is not possible, because it would change the pointer (to the actual "stock" of items), that must point to the first item, and be also known by the memory allocator as such. In principle, one can have a "fake" pointer with some part of the data actually positioned *before* the address it points to. This is in fact the way "Pascal arrays" (and strings) are commonly implemented, with their sizes prefixed to the first item. Bit this introduces some complication (eg one must recompute the original pointer for dealloc) and must be designed right from the start at the lowest level. The only practicle way would be to systematically reserve memory space before the start item [*], for such cases. It is not worth it for a very specific operation like like.insert_anywhere (even less list.insert_at_start), which is not (in my view) the proper point of lists (arrays, in fact). We should use proper collections whenever we need inserting (and removing) at arbitrary locations. 99% lists do not need that. As I see it, lists are for, in order of importance: * put new item (at the end) (also when used as a stack, =push) * iterate (in order) * read item (anywhere) * change item (anywhere) * take or remove last item (only when used only as a stack, =pop) The deal may possibly be different if arrays were python's only collection (like link lists in lisp or tables in lua). d [*] I sometimes wish strings would reserve place for exactly one more code at the _end_, for cases when one must ensure a string (often read from file) terminates with a newline ;-) From stephen at xemacs.org Fri Feb 7 12:05:41 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 07 Feb 2014 20:05:41 +0900 Subject: [Python-ideas] Question about `list.insert` In-Reply-To: <6cd218d1-a29e-4f0e-80f6-434142304c79@googlegroups.com> References: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> <49931276-ddee-44b2-9de3-992110a1e841@googlegroups.com> <6cd218d1-a29e-4f0e-80f6-434142304c79@googlegroups.com> Message-ID: <87eh3fmbcq.fsf@uwakimon.sk.tsukuba.ac.jp> Ram Rachum writes: > Okay, now that I've posted on python-list, and among the 80% of > joke posts in my thread, there was a 20% of posts that assured me that > Python's list has no such behavior and that I should use deque, For complexity of collections the best reference I know is https://wiki.python.org/moin/TimeComplexity > Is there a good reason why `list.insert(whatever, 0)` doesn't > opportunistically try to allocate more space at the left side of > the list, so as to save the expensive operation of moving all > the items? I'm not saying it should reserve space there, just check if > that space is available, and if so use it. The problem is that it would have to have unholy carnal knowledge of OS internals (eg, of malloc). First off, availability itself is non-portable, depending on a lot of things (eg, placement of malloc metadata and Python object metadata). I would imagine those things are placed at the beginning of a data structure, but your OS may vary. Even if somehow they were placed at the end of the allocation block, you'd have to ask malloc if there is an empty block before it, and then futz with the malloc metadata if you were to try to snarf that block. Sounds like a great way to crash Python to me. I'll take collections.deque, thankyouverymuch. A general note: implementations of Python builtins are generally quite well-tuned, but not at the expense of excessive complexity. Python's threshold for "excessive" is pretty low, but nonetheless most builtins are very efficient by now. P.S. ISTR posting via Googlegroups is deprecated. From ned at nedbatchelder.com Fri Feb 7 12:15:01 2014 From: ned at nedbatchelder.com (Ned Batchelder) Date: Fri, 07 Feb 2014 06:15:01 -0500 Subject: [Python-ideas] Question about `list.insert` In-Reply-To: <6cd218d1-a29e-4f0e-80f6-434142304c79@googlegroups.com> References: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> <49931276-ddee-44b2-9de3-992110a1e841@googlegroups.com> <6cd218d1-a29e-4f0e-80f6-434142304c79@googlegroups.com> Message-ID: <52F4C035.7090605@nedbatchelder.com> On 2/7/14 5:14 AM, Ram Rachum wrote: > Okay, now that I've posted on python-list, and among the 80% of joke > posts in my thread, there was a 20% of posts that assured me that > Python's list has no such behavior and that I should use deque, I now > ask on Python-ideas: Is there a good reason why `list.insert(whatever, > 0)` doesn't opportunistically try to allocate more space at the left > side of the list, so as to save the expensive operation of moving all > the items? I'm not saying it should reserve space there, just check if > that space is available, and if so use it. Is there any reason why not? > "check if that space is available": this is not a simple operation. Only the memory allocator knows what blocks of memory are allocated and which are not. Memory allocators typically don't support the operation of "extend my memory block downward if possible." Why is deque not the right answer for your problem? --Ned. From rosuav at gmail.com Fri Feb 7 12:45:33 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Feb 2014 22:45:33 +1100 Subject: [Python-ideas] Question about `list.insert` In-Reply-To: <52F4C035.7090605@nedbatchelder.com> References: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> <49931276-ddee-44b2-9de3-992110a1e841@googlegroups.com> <6cd218d1-a29e-4f0e-80f6-434142304c79@googlegroups.com> <52F4C035.7090605@nedbatchelder.com> Message-ID: On Fri, Feb 7, 2014 at 10:15 PM, Ned Batchelder wrote: > On 2/7/14 5:14 AM, Ram Rachum wrote: >> >> Okay, now that I've posted on python-list, and among the 80% of joke posts >> in my thread, there was a 20% of posts that assured me that Python's list >> has no such behavior and that I should use deque, I now ask on Python-ideas: >> Is there a good reason why `list.insert(whatever, 0)` doesn't >> opportunistically try to allocate more space at the left side of the list, >> so as to save the expensive operation of moving all the items? I'm not >> saying it should reserve space there, just check if that space is available, >> and if so use it. Is there any reason why not? >> > "check if that space is available": this is not a simple operation. Only the > memory allocator knows what blocks of memory are allocated and which are > not. Memory allocators typically don't support the operation of "extend my > memory block downward if possible." Checking if space is available could be done without any fiddling with malloc, though. All it requires is an optimization of pop(0) followed by insert(0,foo). That is, when you poop(0), the list just marks itself as having one junk entry before the first entry (presumably it already keeps track of spare space at the end, so this would be just one more integer), which can then be reused later. But apart from maybe reducing the memory copying (the same optimization could mean that repeated pop(0) calls would incur less copying, too), there's not a huge gain. ChrisA From ram at rachum.com Fri Feb 7 14:47:39 2014 From: ram at rachum.com (Ram Rachum) Date: Fri, 7 Feb 2014 15:47:39 +0200 Subject: [Python-ideas] Question about `list.insert` In-Reply-To: <52F4B752.8090506@gmail.com> References: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> <49931276-ddee-44b2-9de3-992110a1e841@googlegroups.com> <6cd218d1-a29e-4f0e-80f6-434142304c79@googlegroups.com> <52F4B752.8090506@gmail.com> Message-ID: Thanks for your answers everyone! It was interesting for me. On Feb 7, 2014 12:37 PM, "spir" wrote: > On 02/07/2014 11:14 AM, Ram Rachum wrote: > >> Okay, now that I've posted on python-list, and among the 80% of joke posts >> in my thread, there was a 20% of posts that assured me that Python's list >> has no such behavior and that I should use deque, I now ask on >> Python-ideas: Is there a good reason why `list.insert(whatever, 0)` >> doesn't >> opportunistically try to allocate more space at the left side of the list, >> so as to save the expensive operation of moving all the items? I'm not >> saying it should reserve space there, just check if that space is >> available, and if so use it. Is there any reason why not? >> > > It is not possible, because it would change the pointer (to the actual > "stock" of items), that must point to the first item, and be also known by > the memory allocator as such. In principle, one can have a "fake" pointer > with some part of the data actually positioned *before* the address it > points to. This is in fact the way "Pascal arrays" (and strings) are > commonly implemented, with their sizes prefixed to the first item. Bit this > introduces some complication (eg one must recompute the original pointer > for dealloc) and must be designed right from the start at the lowest level. > > The only practicle way would be to systematically reserve memory space > before the start item [*], for such cases. It is not worth it for a very > specific operation like like.insert_anywhere (even less > list.insert_at_start), which is not (in my view) the proper point of lists > (arrays, in fact). We should use proper collections whenever we need > inserting (and removing) at arbitrary locations. 99% lists do not need > that. As I see it, lists are for, in order of importance: > > * put new item (at the end) (also when used as a stack, =push) > * iterate (in order) > > * read item (anywhere) > * change item (anywhere) > > * take or remove last item (only when used only as a stack, =pop) > > The deal may possibly be different if arrays were python's only collection > (like link lists in lisp or tables in lua). > > d > > [*] I sometimes wish strings would reserve place for exactly one more code > at the _end_, for cases when one must ensure a string (often read from > file) terminates with a newline ;-) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > -- > > --- You received this message because you are subscribed to a topic in the > Google Groups "python-ideas" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/python-ideas/G0O5NN9DjSM/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > python-ideas+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From storchaka at gmail.com Fri Feb 7 15:28:38 2014 From: storchaka at gmail.com (Serhiy Storchaka) Date: Fri, 07 Feb 2014 16:28:38 +0200 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <36ABE145-12ED-4A87-B3C0-5FBDEAE36699@yahoo.com> Message-ID: 07.02.14 10:36, Terry Reedy ???????(??): > I wrote the above with the idea that there would be a third parameter to > provide an exception. It would have to have a private default like > > class _NoException(Exception): pass Natural default is StopIteration or (). From solipsis at pitrou.net Fri Feb 7 15:58:46 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 7 Feb 2014 15:58:46 +0100 Subject: [Python-ideas] Question about `list.insert` References: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> <49931276-ddee-44b2-9de3-992110a1e841@googlegroups.com> <6cd218d1-a29e-4f0e-80f6-434142304c79@googlegroups.com> <87eh3fmbcq.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20140207155846.1d0561bc@fsol> On Fri, 07 Feb 2014 20:05:41 +0900 "Stephen J. Turnbull" wrote: > > > Is there a good reason why `list.insert(whatever, 0)` doesn't > > opportunistically try to allocate more space at the left side of > > the list, so as to save the expensive operation of moving all > > the items? I'm not saying it should reserve space there, just check if > > that space is available, and if so use it. > > The problem is that it would have to have unholy carnal knowledge of > OS internals (eg, of malloc). First off, availability itself is > non-portable, depending on a lot of things (eg, placement of malloc > metadata and Python object metadata). ??? I don't understand what you're talking about. It is perfectly possible while being portable. The proof is that bytearray has a limited variant of that optimization (not for insertions, but for deletions at the front). Regards Antoine. From solipsis at pitrou.net Fri Feb 7 15:59:46 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 7 Feb 2014 15:59:46 +0100 Subject: [Python-ideas] Question about `list.insert` References: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> <49931276-ddee-44b2-9de3-992110a1e841@googlegroups.com> <6cd218d1-a29e-4f0e-80f6-434142304c79@googlegroups.com> <52F4C035.7090605@nedbatchelder.com> Message-ID: <20140207155946.2ec23a18@fsol> On Fri, 7 Feb 2014 22:45:33 +1100 Chris Angelico wrote: > But apart from > maybe reducing the memory copying (the same optimization could mean > that repeated pop(0) calls would incur less copying, too), there's not > a huge gain. If you think switching from O(n**2) to O(n) isn't a huge gain, then indeed :-) Regards Antoine. From techtonik at gmail.com Fri Feb 7 09:02:45 2014 From: techtonik at gmail.com (anatoly techtonik) Date: Fri, 7 Feb 2014 11:02:45 +0300 Subject: [Python-ideas] Iterative development In-Reply-To: References: <52E8BBD6.2010604@stoneleaf.us> Message-ID: On Thu, Jan 30, 2014 at 4:40 PM, Chris Angelico wrote: > On Fri, Jan 31, 2014 at 12:25 AM, anatoly techtonik wrote: >> Single dev hour is ok if you reached your goal. That's the point. >> You set the goals - you reach them. If you didn't reach them - you >> analyze and see what could be done better. It is all in relaxing and >> free manner, unlike the bloody corporation culture. You may invite >> other people to join the fun. People can find what are you working >> on and propose help. >> >> This is the process. > > Let's say you pick up something that's going to turn out to take you > three dev hours. You then put one hour of work into it, and the > two-week cut-off rolls around. What do you do? It *not cut-off*. It is *sync*. So, what do you do when two-week sync rolls around.. You list the status, say what was the problem with completing as you're planned, say if you need help, if there is something that could have helped to avoid hurdles you had that led to divergence from the plan, and you also communicate your vision about chances of this particular thing to see the light in the next two weeks. > In the current model, there is no cut-off, so you just keep your work > where it is until you find the time to finish it. Then you format it > as a patch, put it on the tracker issue, and move on. (Or, if you're a > core dev, I suppose you push it, see if the buildbots start looking > red and angry, and then move on. Either way.) It doesn't matter if > that took you one day, two weeks, or three months. It doesn't matter for you, because you work alone. But it still does worth for other people, so if you run out of time and have a clear understanding that over next three months you won't be able to work on this feature, and you've made it clear during sync, you at least can enable others to look into this problem and hack it for you, or you can *freeze* it properly with all work-in-the-process that is necessary for someone to pick it up later. > What you're suggesting is that people should conform to an arbitrary > number-of-days cutoff. That means that if the cut-off is getting > close, there's a *dis*incentive to pick up any job, because you won't > be able to finish it. There are *jobs*, yes. People choose what to do from existing backlog sorted by some criteria (wishes?) or may develop their own idea. But.. That doesn't mean they should *finish* the job in this time period. Jobs could be split into tasks for synchronization between people, and you are free to make the task that you *will* be able to complete in the time span that you've given. It is your own challenge. *research*, *triage*, *make a writeup* are all tasks that are not directly related to a problem, but if they yield some results for others to sync with, they worth to be mentioned. In my practice this sort of question came from people who undergo formal Scrum training and was told that at the end of each iteration you should deliver a complete feature set that was planned with all documentation and related materials. This keeps people motivated to complete the hard things. But Agile doesn't place this restriction - it is flexible to choose what you want to deliver at the end. The main benefit from the iterative process is that when people start to make goals that are public to everyone else, and then these goals fail, it is possible to make the analysis about the process - how to save time and make the whole collaboration stuff more fun. > Imagine if, when writing up a post for the > mailing list, you had to finish each sentence inside one minute as per > the clock. If it's currently showing hh:mm:49, you'd do better to not > start a sentence, because you probably can't finish it in eleven > seconds. Is that an advantage over "just write what you like, when you > like"? The advantage is that if I set a goal to reply to this thread in two weeks, I have better chances to finally find the time to do this in two weeks. From techtonik at gmail.com Fri Feb 7 11:32:47 2014 From: techtonik at gmail.com (anatoly techtonik) Date: Fri, 7 Feb 2014 13:32:47 +0300 Subject: [Python-ideas] Iterative development In-Reply-To: References: Message-ID: On Thu, Jan 30, 2014 at 7:17 PM, Zachary Ware wrote: > I haven't been following this thread very closely, but I have to > disagree with you here, Anatoly. That's ok. Kenneth Rietz converted me to fallibilism, and I hope to spread the virus further. > On Thu, Jan 30, 2014 at 5:24 AM, anatoly techtonik wrote: >> It is quite obvious from outside that Python has some kind of process, > > Which is well documented in several places. It can be tricky to > always find all of those places, but anyone who is interested can ask, > and will be quickly shown where to look. MIT Technology Review had an article about decreasing participation of Wikipedia. Here is the quote: "The main source of those problems is not mysterious. The loose collective running the site today, estimated to be 90 percent male, operates a crushing bureaucracy with an often abrasive atmosphere that deters newcomers who might increase participation in Wikipedia and broaden its coverage. In response, the Wikimedia Foundation, the 187-person nonprofit that pays for the legal and technical infrastructure supporting Wikipedia, is staging a kind of rescue mission. The foundation can?t order the volunteer community to change the way it operates. But by tweaking Wikipedia?s website and software, it hopes to steer the encyclopedia onto a more sustainable path." http://www.technologyreview.com/featuredstory/520446/the-decline-of-wikipedia/ Bureaucracy deters newcomers, and knowing that most people don't read, but scan, I believe that Python should also think about ways to tweak software and process for new generations instead of placing bureaucratic filters to allow only those who comfortable with old ways of doing things to pass by. Most people are lurkers and won't even dare to ask a question especially as (silly) as the one about the process seems. >> but it is quite hard to sync to it for people from outside, > > I'm not sure what you mean here. Every contributor starts from > "outside" of Python. I found no difficulty in getting started when I > did, and I've seen several people start contributing successfully > since then. It would be very hard to go from nothing to suddenly > contributing huge patches to the innermost details of Python at a > rapid pace, but that's not really what people (especially people new > to open source development, like I was) should be doing anyway. Start > slow and small, build from there, and it's an easy and painless > process. And if you just don't have a time for that? If you have a cancer and going to die in the next couple of years if technology won't be advance enough to save you? People have different goals in mind and you're trying to impose your own way on them, which won't suit them. That's why, again, I am not proposing to replace current development practice. I just want to see that people can see benefits in following a parallel route and try to think about those benefits, and not about intrusiveness about some idea in comfort zone of their established workflow. Which leads me the the interesting observation that if people used to things, they are less likely to discuss specific arguments, because there is no problem for them personally. Which divides the community into contributor and non-contributors and makes the communication between parties hard (if not impossible at all). >> because it is not open > > Here I must disagree emphatically. My entire Python experience shows > me that everything about Python is as open as possible. If you want > to know something, look for it. If you can't find it, ask for it. If > you can't be shown where it is, somebody (even yourself) will write it > down somewhere so the next person looking can find it. I agree that Python strives to be as open as possible, but there are projects that do it better. The notion of open is subjective, but the criteria that I use is if information is buried under piles of other information and is tricky to find - it is not truly open. In strictly technical terms of information theory - if information didn't reach the recipient - the information is unreachable. But.. it is my general point of view that Huxley wins over Orwell. In original mail there was the specific notion of open that you generalized. I stated reasons why the process is not open, one of which you've left below: >> - is not completely clear how the planning is made, > > I'm not sure what you mean here, what planning? Anything that could > be construed as "planning" is done via the PEP process, which is well > documented in PEP 1. I believe that question constitutes the answer. There is not planning, so it is not clear what's going on, so it is not possible to jump in and help. Yes, you can ask, but really - who'd do this? Internet has many other things to do than messing with all this mailing list communication stuff that people find "easy". >> which tasks are available for current sprint, what you can help with and how to track >> the progress. > > This is the very definition of a bug tracker, and Python's is quite > good for all of this. Ok. I opened the tracker. Who's doing what currently, what is being done for the next release and what can I help with? I don't see the answer there. I am pretty sure it is a documented somewhere on the internet, but I already look into internet I don't see it. > There could stand to be some upkeep done on > some of the older issues: it would be good for an impartial person to > pick through and see whether an issue is still a problem, update any > patches to apply to current branches, manage the 'easy' tag, add the > proper people to the nosy list, etc. This kind of thing would be a > great place for someone to contribute. Honestly, just bringing all > tracker issues up to date would be a worthwhile sprint task in my > opinion. The only problem that if you're not a committer, you don't have any of the privileges to do the stuff what you've mentioned above. But that's not the reason why the whole useful stuff is not done right now. The reason is that current process doesn't support it or make the whole process not fun. I am glad that we have a first *recurring* task that can be solved with the idea of iterative development process. It is "bug triaging" task which can be decomposed into more fun "cave hunting" task that addresses the most ancient issues to refactor, reclassify and deobfuscate them, e.g. push the progress in incremental way. For this to work, the progress should be measurable for two weeks period: [ ] entomology works for open species in range 1034-2000 (xx total) Which when completed is accompanied with analysis on the problems encountered and recommendation (consensus) what do to with each. specimen no.1034 - thread started from a patch, which was discussed and agreed in ML, LGTM required author to do nitpicks with user comments and more tests, which never happened. specimen filled in 2007-08-27 for Python ???, after 1 year 5 months reworked by core contributor for Python ???, missed the 2.7 deadline, planned for 3.2 and seems like missing it too. I don't know about you, but I see here a few very interesting problems in the process of dissecting this specimen that will be visible after the first iteration and what could and will be addressed in the next iterations by people who choose to work not on Python, but on bug tracker instead. That is the flexibility I am talking about. From techtonik at gmail.com Fri Feb 7 12:30:19 2014 From: techtonik at gmail.com (anatoly techtonik) Date: Fri, 7 Feb 2014 14:30:19 +0300 Subject: [Python-ideas] Iterative development In-Reply-To: <7C031855-8F04-442D-9843-63799B75E673@yahoo.com> References: <52E8BBD6.2010604@stoneleaf.us> <7C031855-8F04-442D-9843-63799B75E673@yahoo.com> Message-ID: On Thu, Jan 30, 2014 at 9:59 PM, Andrew Barnert wrote: > On Jan 30, 2014, at 5:25, anatoly techtonik wrote: > >> It may happen that resistance to change for open source projects may >> be bigger than in organizations. I just want to make sure that people >> aware that applying agile methodology to open source development is >> possible and I am inclined that it brings more positive improvements for >> the Python itself than de-facto development processes. > > Do you have any examples of an open source project (and not a company-driven one) that applied agile methodology and gained any benefits? Showing something concrete like that would make a far better argument than just rambling about what might be possible. Yes. A lot of if you read "agile methodology" as "flexible set of practices to organize a team work". SCons before migration from tigris.org [1] implemented a weekly bug triaging [2] and sync / planning sessions on IRC [3]. UFO:AI implemented roadmapping [4,5] and monthly progress reports on the front page [6]. Mercurial uses time-based release practice for "improving planning process" [7]. Subversion roadmap + status visibility [8]. 1. http://scons.tigris.org/issues/reports.cgi?state=Open+issues&x=Assigned+to&y=Milestone 2. http://scons.org/wiki/BugParty 3. http://scons.org/wiki/BugParty/IrcLog2011-04-10 4. http://ufoai.org/wiki/TODO/Roadmap 5. http://ufoai.org/wiki/TODO 6. http://ufoai.org/wiki/News 7. http://mercurial.selenic.com/wiki/TimeBasedReleasePlan 8. https://subversion.apache.org/roadmap.html Trac can be seen as managed by Edgewall, but many projects that use Trac implicitly implement roadmapping and iterations too http://trac.edgewall.org/roadmap These only those project that I had my own hands on experience with and sent code to, so I am certain that there are even more of these practices in other open source projects that combined make these even be better. Addressing your fear about that agile processes can not be sustained outside of company environment. I think that agile is meant to solve boredom and stress problems, to be flexible and to help people learn tools and methodologies to have fun from joint work. As a children we all enjoy playing games with interesting rules and challenges. This doesn't change with age. What changes is that other people and companies want to control the people and adopt otherwise good things in their awkward and unsuitable business environments. Imagine children play in business suits, and for me it is ridiculous and it doesn't change with age of those children. I think that open source communities are free from all this corporate bullsh*t and can do better at supporting fun, chaotic and decentralized team work. They just never tried to focus on this process. From techtonik at gmail.com Fri Feb 7 13:22:08 2014 From: techtonik at gmail.com (anatoly techtonik) Date: Fri, 7 Feb 2014 15:22:08 +0300 Subject: [Python-ideas] Iterative development In-Reply-To: References: Message-ID: On Thu, Jan 30, 2014 at 11:15 PM, Georg Brandl wrote: >>> - is not completely clear how the planning is made, >> >> I'm not sure what you mean here, what planning? Anything that could >> be construed as "planning" is done via the PEP process, which is well >> documented in PEP 1. > > We have tried quite a few times to make it clear to Anatoly that there is > no "planning" made apart from what you can read about in PEPs and mailing > lists. Apparently he thinks there's a secret agenda, when in reality there > often is no (shared) agenda at all -- that's in the nature of an open source > project. Of course individual developers may have private agendas. Let's pinpoint the conflicting point first - "nature of an open source" - s/a/the/ As I already pointed out, many open source project have planning, and before expanding the idea, let's say that I never had any conspiracy theory behind planning of Python development - I perfectly realize how the chaotic the process is, but that's not clear from outside, and what I am trying to say that not having any visible planning strategy (chaos is a strategy) is bad for any organized effort and more importantly - for an effort to organize. There is a big potential for self-organizing teams in Python community, but that's just didn't happen, because these team can't hold together. They can be organized around bugs (as in bug tracker), issues (as a "problem"), components (stdlib modules), releases (as it works now), tools (independent of core development at all), ideas (as we see in this thread). But to hold on, people from different time zones, cultures, need to sync. Iteration has one awesome and distinctive property - any iteration stripped of all the feature- creeped stuff is just a sync point in time. So, sync point is "subject, place and time". I tried to start with place for pydotorg (see thread few years ago), subject (attempt to split things by module in tracker) - https://bitbucket.org/techtonik/python-stdlib - good indicator of poor development of my skills and now I came to conclusion that the only thing that matters is time. Sync points are important, because they allow a project to be inclusive for people who are not interested in Python development at all, but may want to join later, because they want to make a change. You don't force people to read papers, but let them see hot it works. It is also more entertaining for the brain to watch the real thing and hack on ideas how to make the whole entrypoint more exciting. I can't name any reason why anyone should be excited with Python core developer when faced with a dusty tome of paperwork of ancient contributing wisdom written in largely uncommon English language. >>> which tasks are available for current sprint, what you can help with and how to track >>> the progress. >> >> This is the very definition of a bug tracker, and Python's is quite >> good for all of this. There could stand to be some upkeep done on >> some of the older issues: it would be good for an impartial person to >> pick through and see whether an issue is still a problem, update any >> patches to apply to current branches, manage the 'easy' tag, add the >> proper people to the nosy list, etc. This kind of thing would be a >> great place for someone to contribute. Honestly, just bringing all >> tracker issues up to date would be a worthwhile sprint task in my >> opinion. > > Few people have tried that because it's such a thankless task, but > there was definitely progress. Make it thankful. Make 'implementing a Twisted Highscore for b.p.o' a goal of the next iteration and let people contributions flow. If truck can not pass through gate, somebody get out and lift it. If chaotic effort is ineffective, then coordinated effort can help. There could be coordinator for the one or two iterations, but it can be possible for technology to help with it in the long run if it is worthy. -- anatoly t. From haoyi.sg at gmail.com Fri Feb 7 17:29:47 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Fri, 7 Feb 2014 08:29:47 -0800 Subject: [Python-ideas] Iterative development In-Reply-To: References: Message-ID: If you wrote as many patches as you did long emails, you would probably get all the inside-knowledge on the development process that you wanted *and* be in a better position to suggest changes =P On Fri, Feb 7, 2014 at 4:22 AM, anatoly techtonik wrote: > On Thu, Jan 30, 2014 at 11:15 PM, Georg Brandl wrote: > >>> - is not completely clear how the planning is made, > >> > >> I'm not sure what you mean here, what planning? Anything that could > >> be construed as "planning" is done via the PEP process, which is well > >> documented in PEP 1. > > > > We have tried quite a few times to make it clear to Anatoly that there is > > no "planning" made apart from what you can read about in PEPs and mailing > > lists. Apparently he thinks there's a secret agenda, when in reality > there > > often is no (shared) agenda at all -- that's in the nature of an open > source > > project. Of course individual developers may have private agendas. > > Let's pinpoint the conflicting point first - "nature of an open > source" - s/a/the/ > > As I already pointed out, many open source project have planning, and > before expanding the idea, let's say that I never had any conspiracy theory > behind planning of Python development - I perfectly realize how the chaotic > the process is, but that's not clear from outside, and what I am trying to > say > that not having any visible planning strategy (chaos is a strategy) is bad > for > any organized effort and more importantly - for an effort to organize. > > There is a big potential for self-organizing teams in Python community, but > that's just didn't happen, because these team can't hold together. They can > be organized around bugs (as in bug tracker), issues (as a "problem"), > components (stdlib modules), releases (as it works now), tools (independent > of core development at all), ideas (as we see in this thread). But to hold > on, > people from different time zones, cultures, need to sync. Iteration has one > awesome and distinctive property - any iteration stripped of all the > feature- > creeped stuff is just a sync point in time. > > So, sync point is "subject, place and time". I tried to start with place > for > pydotorg (see thread few years ago), subject (attempt to split things by > module in tracker) - https://bitbucket.org/techtonik/python-stdlib - good > indicator of poor development of my skills and now I came to conclusion > that the only thing that matters is time. > > Sync points are important, because they allow a project to be inclusive for > people who are not interested in Python development at all, but may want > to join later, because they want to make a change. > > You don't force people to read papers, but let them see hot it works. It is > also more entertaining for the brain to watch the real thing and hack on > ideas how to make the whole entrypoint more exciting. I can't name any > reason why anyone should be excited with Python core developer when > faced with a dusty tome of paperwork of ancient contributing wisdom > written in largely uncommon English language. > > >>> which tasks are available for current sprint, what you can help with > and how to track > >>> the progress. > >> > >> This is the very definition of a bug tracker, and Python's is quite > >> good for all of this. There could stand to be some upkeep done on > >> some of the older issues: it would be good for an impartial person to > >> pick through and see whether an issue is still a problem, update any > >> patches to apply to current branches, manage the 'easy' tag, add the > >> proper people to the nosy list, etc. This kind of thing would be a > >> great place for someone to contribute. Honestly, just bringing all > >> tracker issues up to date would be a worthwhile sprint task in my > >> opinion. > > > > Few people have tried that because it's such a thankless task, but > > there was definitely progress. > > Make it thankful. Make 'implementing a Twisted Highscore for b.p.o' > a goal of the next iteration and let people contributions flow. > > If truck can not pass through gate, somebody get out and lift it. If > chaotic effort is ineffective, then coordinated effort can help. There > could > be coordinator for the one or two iterations, but it can be possible for > technology to help with it in the long run if it is worthy. > -- > anatoly t. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Fri Feb 7 19:00:24 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 8 Feb 2014 05:00:24 +1100 Subject: [Python-ideas] Question about `list.insert` In-Reply-To: <20140207155946.2ec23a18@fsol> References: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> <49931276-ddee-44b2-9de3-992110a1e841@googlegroups.com> <6cd218d1-a29e-4f0e-80f6-434142304c79@googlegroups.com> <52F4C035.7090605@nedbatchelder.com> <20140207155946.2ec23a18@fsol> Message-ID: On Sat, Feb 8, 2014 at 1:59 AM, Antoine Pitrou wrote: > On Fri, 7 Feb 2014 22:45:33 +1100 > Chris Angelico wrote: >> But apart from >> maybe reducing the memory copying (the same optimization could mean >> that repeated pop(0) calls would incur less copying, too), there's not >> a huge gain. > > If you think switching from O(n**2) to O(n) isn't a huge gain, then > indeed :-) > Sure, that would be a huge gain. But anyone who's repeatedly calling pop(0) followed by insert(0,x) should be using deque rather than list. It's like optimizing str + str, only less common. Yes, there's an algorithmic complexity improvement, to be sure; but there's an alternative that has all the same improvement already there. A scheme such as I described would have a constant-time penalty for *every list*, so the benefit to a small number of lists is that much less likely to actually improve overall performance. ChrisA From solipsis at pitrou.net Fri Feb 7 19:13:28 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 7 Feb 2014 19:13:28 +0100 Subject: [Python-ideas] Question about `list.insert` References: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> <49931276-ddee-44b2-9de3-992110a1e841@googlegroups.com> <6cd218d1-a29e-4f0e-80f6-434142304c79@googlegroups.com> <52F4C035.7090605@nedbatchelder.com> <20140207155946.2ec23a18@fsol> Message-ID: <20140207191328.616c6a45@fsol> On Sat, 8 Feb 2014 05:00:24 +1100 Chris Angelico wrote: > On Sat, Feb 8, 2014 at 1:59 AM, Antoine Pitrou wrote: > > On Fri, 7 Feb 2014 22:45:33 +1100 > > Chris Angelico wrote: > >> But apart from > >> maybe reducing the memory copying (the same optimization could mean > >> that repeated pop(0) calls would incur less copying, too), there's not > >> a huge gain. > > > > If you think switching from O(n**2) to O(n) isn't a huge gain, then > > indeed :-) > > > > Sure, that would be a huge gain. But anyone who's repeatedly calling > pop(0) followed by insert(0,x) should be using deque rather than list. Indeed, since deque exists, it is the reasonable answer. This whole discussion happens in a hypothetical setting where deque wouldn't exist and there would be an opportunity to make list a one-size-fits-all sequence type. (note that my personal opinion is that built-in Python data types should be sufficiently powerful to cater for most use cases, rather than build a whole array of specialized collection types as in Java or C++) Regards Antoine. From rosuav at gmail.com Fri Feb 7 19:21:25 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 8 Feb 2014 05:21:25 +1100 Subject: [Python-ideas] Question about `list.insert` In-Reply-To: <20140207191328.616c6a45@fsol> References: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> <49931276-ddee-44b2-9de3-992110a1e841@googlegroups.com> <6cd218d1-a29e-4f0e-80f6-434142304c79@googlegroups.com> <52F4C035.7090605@nedbatchelder.com> <20140207155946.2ec23a18@fsol> <20140207191328.616c6a45@fsol> Message-ID: On Sat, Feb 8, 2014 at 5:13 AM, Antoine Pitrou wrote: >> Sure, that would be a huge gain. But anyone who's repeatedly calling >> pop(0) followed by insert(0,x) should be using deque rather than list. > > Indeed, since deque exists, it is the reasonable answer. > > This whole discussion happens in a hypothetical setting where deque > wouldn't exist and there would be an opportunity to make list a > one-size-fits-all sequence type. Well, yes. In the absence of deque, this would be a more plausible optimization; if nothing else, it would make destructive iteration far more efficient, even without the insert() optimization. It'd still require some deep analysis to figure out just how often the optimization would help, vs how often the complexity would add unnecessary cost, but I can imagine a way of doing it that wouldn't cost too much. (Just advance the base pointer and increment a "pre-list spare space" value. Then regular operations can ignore the spare space; only deallocation/resize need to worry about it (to get the real pointer for free()), and insert() can try to take advantage. It could even guess that it might be worth adding some wasted space, to cope with multiple insert()s.) And if someone proposed that, someone else would then say "It'd be so much more efficient if we explicitly tell the list constructor that it should do this, rather than have it do it all the time", and there you are, right back at deque. :) This, btw, is why I like Python's data type model ("have lots of 'em") rather than PHP's ("the Array will do everything, so use it"). If my code is slow because I used OrderedDict instead of list, I can fix my code. If my code is slow because I used Array instead of Array..... s'not a lot I can do about that. ChrisA From rymg19 at gmail.com Fri Feb 7 21:32:53 2014 From: rymg19 at gmail.com (Ryan) Date: Fri, 07 Feb 2014 14:32:53 -0600 Subject: [Python-ideas] Question about `list.insert` In-Reply-To: <20140207191328.616c6a45@fsol> References: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> <49931276-ddee-44b2-9de3-992110a1e841@googlegroups.com> <6cd218d1-a29e-4f0e-80f6-434142304c79@googlegroups.com> <52F4C035.7090605@nedbatchelder.com> <20140207155946.2ec23a18@fsol> <20140207191328.616c6a45@fsol> Message-ID: The reason C++ has so many data types is for performance reasons. If you have a constant list of items, you use a tuple. If you know the list isn't going to exceed a certain length, you use arrays. If you're mainly going to be iterating through it, you use list. Otherwise, you use vector. The reason Python doesn't need to worry about that is because it has entirely different goals. Antoine Pitrou wrote: >On Sat, 8 Feb 2014 05:00:24 +1100 >Chris Angelico wrote: >> On Sat, Feb 8, 2014 at 1:59 AM, Antoine Pitrou >wrote: >> > On Fri, 7 Feb 2014 22:45:33 +1100 >> > Chris Angelico wrote: >> >> But apart from >> >> maybe reducing the memory copying (the same optimization could >mean >> >> that repeated pop(0) calls would incur less copying, too), there's >not >> >> a huge gain. >> > >> > If you think switching from O(n**2) to O(n) isn't a huge gain, then >> > indeed :-) >> > >> >> Sure, that would be a huge gain. But anyone who's repeatedly calling >> pop(0) followed by insert(0,x) should be using deque rather than >list. > >Indeed, since deque exists, it is the reasonable answer. > >This whole discussion happens in a hypothetical setting where deque >wouldn't exist and there would be an opportunity to make list a >one-size-fits-all sequence type. > >(note that my personal opinion is that built-in Python data types >should >be sufficiently powerful to cater for most use cases, rather than build >a whole array of specialized collection types as in Java or C++) > >Regards > >Antoine. > > >_______________________________________________ >Python-ideas mailing list >Python-ideas at python.org >https://mail.python.org/mailman/listinfo/python-ideas >Code of Conduct: http://python.org/psf/codeofconduct/ -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Fri Feb 7 22:43:38 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 08 Feb 2014 06:43:38 +0900 Subject: [Python-ideas] Iterative development In-Reply-To: References: <52E8BBD6.2010604@stoneleaf.us> <7C031855-8F04-442D-9843-63799B75E673@yahoo.com> Message-ID: <87a9e2mwdx.fsf@uwakimon.sk.tsukuba.ac.jp> anatoly techtonik writes: > Yes. A lot of if you read "agile methodology" as "flexible set of > practices to organize a team work". You're missing the point. Agile methodology is about *reducing* flexibility in interaction and *increasing* organization, by imposing practices that increase productive contacts among developers without (hopefully) imposing much "bureaucracy." That works extremely well if the community needs more structured workflows, not least because the lack of hierarchical bureacracy appeals to programmers used to working in chaotic environments. However, Python long ago developed structured cooperation that seems to me to be working very well for the existing developer community. What you are suggesting is not just introducing an "option" to use agile methodologies, but a proposal to *change* the workflow. This will almost certainly be inconvenient and reduce participation by existing contributors. In other words, increasing bureaucracy. I really don't see any benefit to Python in adopting your proposals. The "bureaucracy" involved in Python processes is both minimal and useful -- it directly contributes to the extremely high quality of successive Python releases. From stephen at xemacs.org Fri Feb 7 22:50:29 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 08 Feb 2014 06:50:29 +0900 Subject: [Python-ideas] Question about `list.insert` In-Reply-To: <20140207155846.1d0561bc@fsol> References: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> <49931276-ddee-44b2-9de3-992110a1e841@googlegroups.com> <6cd218d1-a29e-4f0e-80f6-434142304c79@googlegroups.com> <87eh3fmbcq.fsf@uwakimon.sk.tsukuba.ac.jp> <20140207155846.1d0561bc@fsol> Message-ID: <878utmmw2i.fsf@uwakimon.sk.tsukuba.ac.jp> Antoine Pitrou writes: > On Fri, 07 Feb 2014 20:05:41 +0900 > "Stephen J. Turnbull" > wrote: > > > > > Is there a good reason why `list.insert(whatever, 0)` doesn't > > > opportunistically try to allocate more space at the left side of > > > the list, so as to save the expensive operation of moving all > > > the items? I'm not saying it should reserve space there, just check if > > > that space is available, and if so use it. > > > > The problem is that it would have to have unholy carnal knowledge of > > OS internals (eg, of malloc). First off, availability itself is > > non-portable, depending on a lot of things (eg, placement of malloc > > metadata and Python object metadata). > > ??? I don't understand what you're talking about. Obviously. > It is perfectly possible while being portable. The proof is that > bytearray has a limited variant of that optimization (not for > insertions, but for deletions at the front). What about "list.insert(whatever, 0)" looks like a deletion operation to you? Certainly, it would be possible to keep an additional start pointer, and advance that for deletions at position 0, then use that space for insertions at 0 (or perhaps "early" in the list) if available. But the OP refers to *allocation*, and specifically disallows "reserving space". So it's clear he's not talking about a feature like that, he's talking about finding *new* space. From tjreedy at udel.edu Fri Feb 7 22:54:04 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 07 Feb 2014 16:54:04 -0500 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: <20140207083919.GB1313@ando> References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <52F44C95.4010104@gmail.com> <20140207083919.GB1313@ando> Message-ID: On 2/7/2014 3:39 AM, Steven D'Aprano wrote: > is simpler than the proposed behaviour: > > "the callable is called until it returns the sentinel, unless the > sentinel is an exception instance or class, in which case the > callable is called until it raises that exception, or one which is > compatible with it" To repeat, for the 3rd or 4th time, this is not the behavior that either I or Ram proposed. I even provided code to be clear as to what I mean at the time. -- Terry Jan Reedy From solipsis at pitrou.net Fri Feb 7 23:33:15 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 7 Feb 2014 23:33:15 +0100 Subject: [Python-ideas] Question about `list.insert` References: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> <49931276-ddee-44b2-9de3-992110a1e841@googlegroups.com> <6cd218d1-a29e-4f0e-80f6-434142304c79@googlegroups.com> <87eh3fmbcq.fsf@uwakimon.sk.tsukuba.ac.jp> <20140207155846.1d0561bc@fsol> <878utmmw2i.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20140207233315.519849e6@fsol> On Sat, 08 Feb 2014 06:50:29 +0900 "Stephen J. Turnbull" wrote: > > Certainly, it would be possible to keep an additional start pointer, > and advance that for deletions at position 0, then use that space for > insertions at 0 (or perhaps "early" in the list) if available. Possible, and quite reasonable actually. > But > the OP refers to *allocation*, and specifically disallows "reserving > space". Ok, so you're arguing about a misunderstanding by the OP about how memory allocation works. That doesn't mean that overallocating at the front is any more difficult than overallocating at the end is (the latter being of course already implemented by the list datatype). Regards Antoine. From solipsis at pitrou.net Fri Feb 7 23:33:43 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 7 Feb 2014 23:33:43 +0100 Subject: [Python-ideas] Question about `list.insert` References: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> <49931276-ddee-44b2-9de3-992110a1e841@googlegroups.com> <6cd218d1-a29e-4f0e-80f6-434142304c79@googlegroups.com> <52F4C035.7090605@nedbatchelder.com> <20140207155946.2ec23a18@fsol> <20140207191328.616c6a45@fsol> Message-ID: <20140207233343.0bffc9b3@fsol> On Fri, 07 Feb 2014 14:32:53 -0600 Ryan wrote: > > The reason Python doesn't need to worry about that is because it has entirely different goals. Which was precisely my point. Regards Antoine. From ncoghlan at gmail.com Sat Feb 8 01:28:26 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 8 Feb 2014 10:28:26 +1000 Subject: [Python-ideas] Iterative development In-Reply-To: <87a9e2mwdx.fsf@uwakimon.sk.tsukuba.ac.jp> References: <52E8BBD6.2010604@stoneleaf.us> <7C031855-8F04-442D-9843-63799B75E673@yahoo.com> <87a9e2mwdx.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On 8 Feb 2014 07:44, "Stephen J. Turnbull" wrote: > > anatoly techtonik writes: > > > Yes. A lot of if you read "agile methodology" as "flexible set of > > practices to organize a team work". > > You're missing the point. Agile methodology is about *reducing* > flexibility in interaction and *increasing* organization, by imposing > practices that increase productive contacts among developers without > (hopefully) imposing much "bureaucracy." That works extremely well if > the community needs more structured workflows, not least because the > lack of hierarchical bureacracy appeals to programmers used to working > in chaotic environments. > > However, Python long ago developed structured cooperation that seems > to me to be working very well for the existing developer community. > What you are suggesting is not just introducing an "option" to use > agile methodologies, but a proposal to *change* the workflow. This > will almost certainly be inconvenient and reduce participation by > existing contributors. In other words, increasing bureaucracy. > > I really don't see any benefit to Python in adopting your proposals. > The "bureaucracy" involved in Python processes is both minimal and > useful -- it directly contributes to the extremely high quality of > successive Python releases. It's also deliberately structured to create *less* work for the core developers (and PEP 462 is a proposal to introduce even more automation to eliminate some of the current more tedious parts). The thing about the various Agile models are that they are a tool to improve communications flow between a development team and the rest of the company they work for. It's pretty good at that task, but it requires a highly cohesive team, working on short time frames to meet the needs of a particular business. This is the complete opposite of the way a relatively loose collaborative project like CPython works - each of the core developers is contributing for our own reasons, and while we do self-impose deadlines in order to actually get things shipped, that's largely voluntary - if a feature isn't ready, it isn't ready, and slips to the next release. The contributor documentation we provide is there not only for our own reference, but also to help people get involved if they want to, both because there's a natural attrition rate to open source development (so current developers move on and need to be replaced by new contributors), and also because the standard library is large and we can always use more help (and PEP 462 is largely about making more effective use of the help we *already* receive). The conflict between the core development team & Anatoly has always been that he wants *us* to change *our* practices to accommodate *him*. He first wanted us to move all development discussions to Google Wave because *he* preferred it to email. Then he wanted the PSF to drop the CLA relicensing terms, even though we need them to account for the CNRI licensing history, and even after a PSF director sat down with him at the PyCon US sprints to explain the terms and the need for them. Most people that refuse to sign the CLA are polite and either walk away entirely, or recognise that refusing to sign it will greatly restrict their ability to contribute constructively. Anatoly, by contrast, continues to try to interact with the core development team as if he was our direct manager, and then acts surprised when people react badly to his completely unjustified sense of being entitled to tell us what to do (as well as his making proud declarations of the fact that he has deliberately chosen not to read the existing contributor documentation, because he finds doing such a thing boring). Cheers, Nick. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Sat Feb 8 01:30:12 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 8 Feb 2014 11:30:12 +1100 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <52F44C95.4010104@gmail.com> <20140207083919.GB1313@ando> Message-ID: <20140208003012.GG3799@ando> On Fri, Feb 07, 2014 at 04:54:04PM -0500, Terry Reedy wrote: > On 2/7/2014 3:39 AM, Steven D'Aprano wrote: > > >is simpler than the proposed behaviour: > > > > "the callable is called until it returns the sentinel, unless the > > sentinel is an exception instance or class, in which case the > > callable is called until it raises that exception, or one which is > > compatible with it" > > To repeat, for the 3rd or 4th time, this is not the behavior that either > I or Ram proposed. I even provided code to be clear as to what I mean at > the time. You did write: No new parameter is needed. We only need to add 'or raises' to the two-parameter iter definition "In the second form, the callable is called until it returns the sentinel." to get "In the second form, the callable is called until it returns or raises the sentinel." I agree that what I described is not what Ram suggested, but I believe that it was you who first suggested overloading the meaning of the sentinel argument depending on whether it was an exception or not. If you've changed your mind, I'll be glad to hear it, and you can stop reading here. If you haven't, then I admit I'm still confused, and would be grateful for a short, simple, obvious example of what you are proposing and how it differs from the current behaviour. I must admit that I haven't studied your code snippets in detail, particularly since you descibed at least one of them as "twisted and awful". I don't believe that an iterable that returns exceptions (either exception classes, or exception instances) is either twisted or awful. However, I do believe that overloading the sentinel argument depending on the specific type of value provided *is* twisted and awful. That sort of behaviour should only be allowed when there is otherwise no possible reason for the function to return that kind of value. E.g. contrast iter() with (say) str.replace. The optional count argument, if given, must be an int. There is no valid reason for supplying (say) count='a string'. So we might, hypothetically, overload the count operator to say "if count is a string, then do this special behaviour instead". Even that is dubious, but at least it is backwards-compatible. With iter(), on the other hand, the callable can return values of any type, it is completely general. Likewise the sentinel can be of any type. We wouldn't even dream of overloading the behaviour of iter when sentinel happens to be a string. We shouldn't do so if it happens to be , no matter how unusual the type. -- Steven From tjreedy at udel.edu Sat Feb 8 08:32:27 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 08 Feb 2014 02:32:27 -0500 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: <20140208003012.GG3799@ando> References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <52F44C95.4010104@gmail.com> <20140207083919.GB1313@ando> <20140208003012.GG3799@ando> Message-ID: <52F5DD8B.20203@udel.edu> On 2/7/2014 7:30 PM, Steven D'Aprano wrote: > On Fri, Feb 07, 2014 at 04:54:04PM -0500, Terry Reedy wrote: >> On 2/7/2014 3:39 AM, Steven D'Aprano wrote: This >>> "the callable is called until it returns the sentinel, unless the >>> sentinel is an exception instance or class, in which case the >>> callable is called until it raises that exception, or one which is >>> compatible with it" >> >> To repeat, for the 3rd or 4th time, this is not the behavior that either >> I or Ram proposed. I even provided code to be clear as to what I mean at >> the time. is critically different from what I actually wrote for how to change the one-line docstring summary > You did write: > > No new parameter is needed. We only need to add 'or raises' to the > two-parameter iter definition "In the second form, the callable is > called until it returns the sentinel." to get "In the second form, > the callable is called until it returns or raises the sentinel." and the code that followed because it adds 'unless the sentinel is an exception instance or class'. Your addtion would change existing behavior before an exception is raised (if ever). I explicitly said in both code and text that I was not proposing to do that, but only to change what happened if and when an exception were raised (at which point there would be no more return values values to worry about. > If you've changed your mind, I'll be glad to hear it, As I already responded to you at 4:28 EST Friday, I withdrew my reuse-the-parameter proposal, in response to Yuri, 2 hours before the timestamp I have on your first response. In that response to you, I discussed in several paragraphs the actual existing parameter and how to add a third parameter. Since you did not read it, here it is again. ''' In a later message, I reversed myself, in spite of the actual C signature making it a bit messy. Although help says 'iter(callable, sentinel)', the actual signature is iter(*args), with any attempt to pass an arg by keyword raising TypeError: iter() takes no keyword arguments Let n = len(args). Then the code switches as follows: n=0: TypeError: iter expected at least 1 arguments, got 0 n>2: TypeError: iter expected at most 2 arguments, got 3 n=1: Return __iter__ or __getitem__ wrapper. n=2: if callable(args[0]): return callable_iterator(*args), else: TypeError: iter(v, w): v must be callable If the current signature were merely extended, then I believe the new signature would have to be (if possible) iter(*args, *, stop_iter=) But having parameter args[1] (sentinel) be position-only, with no default, while added parameter stop_iter is keyword only, with a (private) default, would be a bit weird. So instead I would suggest making the new signature be iter(iter_or_call, sentinel=, stop_iter=). If sentinel and stop_iter are both default, use current n=1 code, else pass all 3 args to modified callable_iterator that compares sentinel to return values and catches stop_iter exceptions. Either way, the user could choose to only stop on a return value, only stop on an exception, or stop on either with the two values not having to be the same. The only thing that would break is code that depends on a TypeError, but we allow ourselves to do that to extend functions. ''' iter(iter_or_call, sentinel=, stop_iter=) is still my current proposal. Serhiy suggested 'stop_iter=StopIteration' and I will explain separately why I think that this would not work. -- Terry Jan Reedy From stefan_ml at behnel.de Sat Feb 8 09:28:15 2014 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 08 Feb 2014 09:28:15 +0100 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: <52F5DD8B.20203@udel.edu> References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <52F44C95.4010104@gmail.com> <20140207083919.GB1313@ando> <20140208003012.GG3799@ando> <52F5DD8B.20203@udel.edu> Message-ID: Terry Reedy, 08.02.2014 08:32: > iter(iter_or_call, sentinel=, stop_iter=) > is still my current proposal. +1, the name "stop_iter" resembles StopIteration, which IMHO is enough of a hint at what it does. Also, +1 for the general proposal. While I'd rarely use either of the two, my guess is that there aren't even as many use cases for iterating up to a sentinel value (and I used that already) as for iterating up to an exception (and I'm sure I would have used that as well, if it had been there). > Serhiy suggested 'stop_iter=StopIteration' > and I will explain separately why I think that this would not work. It suggests that it swallows a raised StopIteration *instance* and raises its own, which it shouldn't. Stefan From tjreedy at udel.edu Sat Feb 8 09:32:56 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 08 Feb 2014 03:32:56 -0500 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <36ABE145-12ED-4A87-B3C0-5FBDEAE36699@yahoo.com> Message-ID: On 2/7/2014 9:28 AM, Serhiy Storchaka wrote: > 07.02.14 10:36, Terry Reedy ???????(??): >> I wrote the above with the idea that there would be a third parameter to >> provide an exception. It would have to have a private default like >> >> class _NoException(Exception): pass > > Natural default is StopIteration or (). That does not work because iter has to know if the user called iter with only one argument, an iterable, or with two or (with the proposal) three, with the first argument being a callable. def iter(iter_or_call, sentinel=, stop_iter=): if sentinel == and stop_iter == : # iter_or_call must be an iterable else: # iter_or_call must be a callable return callable_iterator(iter_or_call, sentinel, stop_iter) where callable_iterator has been modified to take the third parameter and raise StopIteration if iter_or_call raises stop_iter. class Callable_iterator: def __next__(self): try: val = self.func() except self.stop_iter: raise StopIteration from None if self.sentinel == value: raise StopIteration else: return val If a user passes sentinel but not stop_iter, the except clause should have no effect and the method should work as it does currently. A default of StopIteration would accomplish that but it does not work for the iter switch. A private exception class also works since no exception raised by self.func should be an instance of such a class (unless the user violates convention). If a user passes stop_iter but not sentinel, the if clause should have no effect and the iteration should continue until there is an exception. So the default sentinel should never compare equal to any value returned by self.func. A private instance of object on the left of == will invoke object.__eq__ which compares by identity. Unless a user violates convention by extracting and using the private instance, the function will never return it. Since value could compare equal to everything, the order for == matters. -- Terry Jan Reedy From ron3200 at gmail.com Sat Feb 8 18:55:56 2014 From: ron3200 at gmail.com (Ron Adam) Date: Sat, 08 Feb 2014 11:55:56 -0600 Subject: [Python-ideas] @partials = decorators Message-ID: On 02/06/2014 05:14 AM, Nick Coghlan wrote:> On 6 February 2014 20:34, Ron Adam wrote: (clipped unrelated discussion) > As far as what you're proposing goes, is it essentially a way to > declare a function like (spelling out the lambdas fully): > > def S(x): > def _second(y): > def _third(z): > return x(z)(y(z)) > return _third > return _second > > As something much shorter like this: > > def S (x)(y)(z): > return x(z)(y(z)) > > The main potential benefit I could see to a construct like that is > that it may allow the more consistent creation of closures that > support pickling, since the outer functions are guaranteed not to have > any side effects and to have argument capture as their*only* > significant state. This means that you could take the inner function, > pickle it along with its closure variables and reconstruct that at the > far end, only relying on the name of the outer function. > > Such a construct could also make decorator factories easier to write. > > def decorator(f): > # Do something with f > > def decorator_factory(some, args, here)(f): > # Do something with f, but have access to the bound args. > > There's an argument to be made that the extra parens in the function > header are too easy to miss, but I still see "make it easier to write > side-effect free closures" as an idea worth discussing further. It's a different feature than the one I was suggesting, which was a type of continuations... but the two concepts are related and compatible. I think the S(x)(y)(z): is adding more complexity to function signatures which are already more complex than I like, but there is an alternate option... What if these *calls* were equivalent... S(x, y, z) == S(x)(y)(z) * There might need to be some additional device or syntax needed to make it work. (* see '@' comments further down.) That is on the call side, so the definition would still be the same. def(x, y, z): ... Which is nice, because it gives us partials at a lower level which may have some advantages over a library function. And it gives us a better way to define decorators. def deco(func, *args, **kwds): ... deco(func, *args, **kwds) == deco(func)(*args, **kwds) or deco(func)(*args)(**kwds) In effect making a decorators into partials. @deco def foo(...): ... Hmmm... Could the @ syntax be generalised in this case? @foo(func) Partial waiting for rest... Then the more general case... a_op = @foo(op) # Partial waiting for rest. a_x = @a_op(x) a = a_x(y) That would be cool, unifies decorators and partials with decorator syntax! I think it's even back-words compatible if you allow this equivalency. def foo(x, y): ... @foo(x, y) == foo(x, y) # An identity partial And the decorator case becomes... @partial def func(): ... And we get this equivalency as well.... @partial == partial # I think this could work. The reason these things are interesting to me is that, I've been thinking about the order of function arguments and if there could be some generalised concepts that can be applied to that problem. Being able to use normal functions effectively in these ways, (with partials, and continuations), is related to the order of the arguments and how they are commonly used. Cheers, Ron From steve at pearwood.info Sat Feb 8 23:06:57 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 9 Feb 2014 09:06:57 +1100 Subject: [Python-ideas] @partials = decorators In-Reply-To: References: Message-ID: <20140208220657.GK3799@ando> On Sat, Feb 08, 2014 at 11:55:56AM -0600, Ron Adam wrote: > What if these *calls* were equivalent... > > S(x, y, z) == S(x)(y)(z) If you want Haskell, you know where to find it :-) http://www.haskell.org/haskellwiki/Currying But seriously, I think that is a perfectly fine design decision for a functional language like Haskell, but it's one which would be awful for a multi-paradigm language like Python which is aimed at a more general programming audience. That means you would never again get an explicit TypeError from leaving out a mandatory argument[1], since *no* arguments are mandatory, you'd just get a mysterious partial application object. When a beginner, or Java programmer, writes: result = getattr(obj) # Oops, forgot the attribute name. # much later on... print(result + 1) I wouldn't like to be the one to explain the eventual error they get. Or worse, they do this: method = getattr(obj) # Oops, forgot the method name. result = method(some_string) which is likely to mysteriously succeed for some strings, and fail for others. I think that for Python, having to explicitly perform partial application is a good thing. [1] With the possible exception of not providing any arguments at all. -- Steven From ron3200 at gmail.com Sat Feb 8 23:41:54 2014 From: ron3200 at gmail.com (Ron Adam) Date: Sat, 08 Feb 2014 16:41:54 -0600 Subject: [Python-ideas] @partials = decorators In-Reply-To: References: Message-ID: On 02/08/2014 11:55 AM, Ron Adam wrote: > > On 02/06/2014 05:14 AM, Nick Coghlan wrote:> On 6 February 2014 20:34, Ron > Adam wrote: > Hmmm... Could the @ syntax be generalised in this case? > > @foo(func) Partial waiting for rest... > > > Then the more general case... > > a_op = @foo(op) # Partial waiting for rest. > a_x = @a_op(x) > a = a_x(y) > > That would be cool, unifies decorators and partials with decorator syntax! > > > I think it's even back-words compatible if you allow this equivalency. > > def foo(x, y): > ... > > @foo(x, y) == foo(x, y) # An identity partial > > > And the decorator case becomes... > > @partial > def func(): > ... > > And we get this equivalency as well.... > > @partial == partial # I think this could work. > > > The reason these things are interesting to me is that, I've been thinking > about the order of function arguments and if there could be some > generalised concepts that can be applied to that problem. Being able to > use normal functions effectively in these ways, (with partials, and > continuations), is related to the order of the arguments and how they are > commonly used. Here's a rough python version... It currently gets tripped up on optional arguments, but it does work for non optional cases. I think a builtin, along with the new signatures capabilities could fix those cases. (and to address Stevens conserns.. It would be explicit and won't effect the beginner cases if you don't use the @... In this example I'm using P. And a builtin function could work just as well. Although I would like the @ syntax to just work without the function. It's just a first attempt, with probably a lot of issues to be handled still, but it does show it's doable. Cheers, Ron #------------------------------- # A builtin could do this much better. class P: def __init__(self, f, *args, **kwds): self.f = f self.args = args self.kwds = kwds def __call__(self, *args, **kwds): args = list(self.args) + list(args) kwds.update(self.kwds) try: return self.f(*args, **kwds) except TypeError: return self.__class__(self.f, *args, **kwds) # This the easy way to write decorators!!! :-) @P def deco(f, *args): print("Got: ", args) return f(*args) # And a partial function example with it. :-) @P @deco def foo(x, y, z): return x + y + z print(foo(1, 2, 3)) print(foo(1, 2)(3)) print(foo(1)(2)(3)) #------------------------- The output... ra:~ >python3 partial_deco.py Got: () Got: (1, 2, 3) 6 Got: (1, 2) Got: (1, 2, 3) 6 Got: (1,) Got: (1, 2) Got: (1, 2, 3) 6 From graffatcolmingov at gmail.com Sat Feb 8 23:46:06 2014 From: graffatcolmingov at gmail.com (Ian Cordasco) Date: Sat, 8 Feb 2014 16:46:06 -0600 Subject: [Python-ideas] @partials = decorators In-Reply-To: References: Message-ID: On Sat, Feb 8, 2014 at 4:41 PM, Ron Adam wrote: > > > On 02/08/2014 11:55 AM, Ron Adam wrote: > >> >> On 02/06/2014 05:14 AM, Nick Coghlan wrote:> On 6 February 2014 20:34, Ron >> Adam wrote: >> > > Hmmm... Could the @ syntax be generalised in this case? >> >> @foo(func) Partial waiting for rest... >> >> >> Then the more general case... >> >> a_op = @foo(op) # Partial waiting for rest. >> a_x = @a_op(x) >> a = a_x(y) >> >> That would be cool, unifies decorators and partials with decorator syntax! >> >> >> I think it's even back-words compatible if you allow this equivalency. >> >> def foo(x, y): >> ... >> >> @foo(x, y) == foo(x, y) # An identity partial >> >> >> And the decorator case becomes... >> >> @partial >> def func(): >> ... >> >> And we get this equivalency as well.... >> >> @partial == partial # I think this could work. >> >> >> The reason these things are interesting to me is that, I've been thinking >> about the order of function arguments and if there could be some >> generalised concepts that can be applied to that problem. Being able to >> use normal functions effectively in these ways, (with partials, and >> continuations), is related to the order of the arguments and how they are >> commonly used. >> > > Here's a rough python version... It currently gets tripped up on optional > arguments, but it does work for non optional cases. > > I think a builtin, along with the new signatures capabilities could fix > those cases. > > (and to address Stevens conserns.. It would be explicit and won't effect > the beginner cases if you don't use the @... > > In this example I'm using P. And a builtin function could work just as > well. Although I would like the @ syntax to just work without the function. > > It's just a first attempt, with probably a lot of issues to be handled > still, but it does show it's doable. > > Cheers, > Ron > > > #------------------------------- > > # A builtin could do this much better. > class P: > def __init__(self, f, *args, **kwds): > self.f = f > self.args = args > self.kwds = kwds > def __call__(self, *args, **kwds): > args = list(self.args) + list(args) > kwds.update(self.kwds) > try: > return self.f(*args, **kwds) > except TypeError: > return self.__class__(self.f, *args, **kwds) > > > # This the easy way to write decorators!!! :-) > @P > def deco(f, *args): > print("Got: ", args) > return f(*args) > > > # And a partial function example with it. :-) > @P > @deco > def foo(x, y, z): > return x + y + z > > print(foo(1, 2, 3)) > print(foo(1, 2)(3)) > print(foo(1)(2)(3)) > #------------------------- > > The output... > > ra:~ >python3 partial_deco.py > Got: () > Got: (1, 2, 3) > 6 > Got: (1, 2) > Got: (1, 2, 3) > 6 > Got: (1,) > Got: (1, 2) > Got: (1, 2, 3) > > 6 > I experimented with something similar over at https://github.com/sigmavirus24/curryer but never released it. It's fairly well tested but I'm not sure I'd use it for anything other than a toy. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron3200 at gmail.com Mon Feb 10 00:21:08 2014 From: ron3200 at gmail.com (Ron Adam) Date: Sun, 09 Feb 2014 17:21:08 -0600 Subject: [Python-ideas] @partials = decorators In-Reply-To: References: Message-ID: <52F80D64.5040907@gmail.com> On 02/08/2014 04:46 PM, Ian Cordasco wrote: > I experimented with something similar over at > https://github.com/sigmavirus24/curryer but never released it. It's > fairly well tested but I'm not sure I'd use it for anything other > than a toy. Thanks I take a look, probably later this week, and see what is different. Today I made a nicer version. This fits both uses as decorator helpers and as a way to capture arguments for nested functions.What it's does is factor out nesting used to capture the arguments. And has some useful properties to help it work in more situations than the functools partial function. It also expresses lambda expressions surprisingly well! ;-) @P def I(x): """ I := ?x.x (Identity) """ return x @P def K(x, y): """ K := ?x.?y.x """ return x @P def S(x, y, z): """ S := ?x.?y.?z.x z (y z) """ return x(z)(y(z)) # Use "_" as the lambda symbol. _ = P # Allows _(S, K, K, Z) in place of S(K)(K)(Z) or S(K, K, Z) assert _(I, 'A'), 'A' assert _(K, 'A', 'B'), 'B' assert _(S, K, K, 'C'), 'C' # Yes, This really works! :-) When I get back from a short 3 day trip, I'm going to test how it works in the library by replacing occurrences of "\n at ..." with "\n at P(...)" and see what breaks. Should be interesting. I expect it to get hung up on optional args and keywords in places. But possibly not, as they will be included at the same time the partial is complete in most, if not all cases. One thing that bother me is that, catching TypeError is way to general. It would be very nice if "wrong number of args" raised a subclass of TypeError so we can test for just that type of error. Cheers, Ron (Tests not included to keep it short.) ------------------------------------------------------ """ ### NOTE: NOT FULLY TESTED!!! Especially with keywords. A PARTIAL PROTOCOL: Takes partial number of args and applies them to a callable. Calls the callable when all it's functions are present, else returns an updated partial object. THE RULES: * A partial is equivalent to the sum of it's partials. P(f, a, b, c) == P(f)(a)(b)(c) * A complete partial is equivalent to the function f When all the parts are provided. P(f, a, b, c) == f(a, b, c) * If f takes no arguments. (same rule) P(f) == f() * And empty partial evaluates to a partial. P() == P * If f requires at least one (not provided) argument. P(f)() == P(f) P()(f) == P(f) ERROR CASES: P(1)() # P(1) --> 1, 1 is not a callable. (* probably more) CONTROLS: It's possible to add controls for both the number of args, and number of Partial calls. Author: Ronald Adam (ron3200 at gmail.com) """ class CollectMore: """ Collect more args and keywords. """ def __init__(self, f, *args, **kwds): self.f = f self.args = list(args) self.kwds = kwds def __call__(self, *args, **kwds): args = self.args + list(args) self.kwds.update(kwds) return self.f(*args, **self.kwds) def P(*args, **kwds): """ P - Partial function """ if len(args) == len(kwds) == 0: return P if len(args) > 0: f = args[0] if callable(f): a = args[1:] try: # No required args needed. # or all required args present. return f(*a, **kwds) except TypeError: # Better way to do this? pass elif len(args) == 1 and len(kwds) == 0: return f return CollectMore(P, *args, **kwds) # The implementation above also makes the functions that # use partial decorators become partials too. To avoid # that you can use an N-partial (vs impartial) partial # decorator. @P def NP(n, f, *args, **kwds): """ NP - N-Partial function Example: # Pass only 3 arguments for foo. @NP(3) def foo(*args): return args """ print("NP", n, f, *args, **kwds) if len(args) == 0: raise TypeError # Ready for rest, Partial catches this. elif len(args) < n: raise ValueError("%s is partial to %s values" % (f.__name__, n)) elif len(args) > n: raise ValueError("%s is partial to %s values" % (f.__name__, n)) else: return f(*args, **kwds) # TODO: A partial depth control function. From ron3200 at gmail.com Mon Feb 10 02:11:12 2014 From: ron3200 at gmail.com (Ron Adam) Date: Sun, 09 Feb 2014 19:11:12 -0600 Subject: [Python-ideas] @partials = decorators In-Reply-To: <52F80D64.5040907@gmail.com> References: <52F80D64.5040907@gmail.com> Message-ID: That got de-indented.... so here it is again... :-/ """ ### NOTE: NOT FULLY TESTED!!! Especially with keywords. A PARTIAL PROTOCOL: Takes partial number of args and applies them to a callable. Calls the callable when all it's functions are present, else returns an updated partial object. THE RULES: * A partial is equivalent to the sum of it's partials. P(f, a, b, c) == P(f)(a)(b)(c) * A complete partial is equivalent to the function f When all the parts are provided. P(f, a, b, c) == f(a, b, c) * If f takes no arguments. (same rule) P(f) == f() * And empty partial evaluates to a partial. P() == P * If f requires at least one (not provided) argument. P(f)() == P(f) P()(f) == P(f) ERROR CASES: P(1)() # P(1) --> 1, 1 is not a callable. (* probably more) CONTROLS: It's possible to add controls for both the number of args, and number of Partial calls. Author: Ronald Adam (ron3200 at gmail.com) """ class CollectMore: """ Collect more args and keywords. """ def __init__(self, f, *args, **kwds): self.f = f self.args = list(args) self.kwds = kwds def __call__(self, *args, **kwds): args = self.args + list(args) self.kwds.update(kwds) return self.f(*args, **self.kwds) def P(*args, **kwds): """ P - Partial function """ if len(args) == len(kwds) == 0: return P if len(args) > 0: f = args[0] if callable(f): a = args[1:] try: # No required args needed. # or all required args present. return f(*a, **kwds) except TypeError: # Better way to do this? pass elif len(args) == 1 and len(kwds) == 0: return f return CollectMore(P, *args, **kwds) # The implementation above also makes the functions that # use partial decorators become partials too. To avoid # that you can use an N-partial (vs impartial) partial # decorator. @P def NP(n, f, *args, **kwds): """ NP - N-Partial function Example: # Pass only 3 arguments for foo. @NP(3) def foo(*args): return args """ print("NP", n, f, *args, **kwds) if len(args) == 0: raise TypeError # Ready for rest, Partial catches this. elif len(args) < n: raise ValueError("%s is partial to %s values" % (f.__name__, n)) elif len(args) > n: raise ValueError("%s is partial to %s values" % (f.__name__, n)) else: return f(*args, **kwds) From ram at rachum.com Mon Feb 10 10:39:49 2014 From: ram at rachum.com (Ram Rachum) Date: Mon, 10 Feb 2014 11:39:49 +0200 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <36ABE145-12ED-4A87-B3C0-5FBDEAE36699@yahoo.com> Message-ID: HI everybody, I see that discussion has stalled on this suggestion. What can we do to move this forward? On Sat, Feb 8, 2014 at 10:32 AM, Terry Reedy wrote: > On 2/7/2014 9:28 AM, Serhiy Storchaka wrote: > >> 07.02.14 10:36, Terry Reedy ???????(??): >> >>> I wrote the above with the idea that there would be a third parameter to >>> provide an exception. It would have to have a private default like >>> >>> class _NoException(Exception): pass >>> >> >> Natural default is StopIteration or (). >> > > That does not work because iter has to know if the user called iter with > only one argument, an iterable, or with two or (with the proposal) three, > with the first argument being a callable. > > def iter(iter_or_call, sentinel=, stop_iter= exception>): > if sentinel == and stop_iter == : > # iter_or_call must be an iterable > which is to return iterable.__iter__() > or getitem_iterable(iterable)> > else: > # iter_or_call must be a callable > return callable_iterator(iter_or_call, sentinel, stop_iter) > > where callable_iterator has been modified to take the third parameter and > raise StopIteration if iter_or_call raises stop_iter. > > class Callable_iterator: > def __next__(self): > try: > val = self.func() > except self.stop_iter: > raise StopIteration from None > if self.sentinel == value: > raise StopIteration > else: > return val > > If a user passes sentinel but not stop_iter, the except clause should have > no effect and the method should work as it does currently. A default of > StopIteration would accomplish that but it does not work for the iter > switch. A private exception class also works since no exception raised by > self.func should be an instance of such a class (unless the user violates > convention). > > If a user passes stop_iter but not sentinel, the if clause should have no > effect and the iteration should continue until there is an exception. So > the default sentinel should never compare equal to any value returned by > self.func. A private instance of object on the left of == will invoke > object.__eq__ which compares by identity. Unless a user violates convention > by extracting and using the private instance, the function will never > return it. Since value could compare equal to everything, the order for == > matters. > > -- > Terry Jan Reedy > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > -- > > --- You received this message because you are subscribed to a topic in the > Google Groups "python-ideas" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/python-ideas/UCaNfAHkBlQ/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > python-ideas+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Mon Feb 10 12:14:34 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 10 Feb 2014 06:14:34 -0500 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <36ABE145-12ED-4A87-B3C0-5FBDEAE36699@yahoo.com> Message-ID: On 2/10/2014 4:39 AM, Ram Rachum wrote: > I see that discussion has stalled on this suggestion. I think of it as more or less finished here and time to open an issue on the tracker, which I plan to do soon, with the specific signature and Python equivalent code I have suggested, as well as your example. If you open one, add me as nosy so I can add the above. > What can we do to move this forward? Remind me if neither of us have not opened an issue within a week. -- Terry Jan Reedy From sturla.molden at gmail.com Mon Feb 10 15:31:24 2014 From: sturla.molden at gmail.com (Sturla Molden) Date: Mon, 10 Feb 2014 14:31:24 +0000 (UTC) Subject: [Python-ideas] switch statement as context manager? Message-ID: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> I've noticed that PyExt has a switch statement implemented as a context manager. with switch(foobar): if case(1): pass if case(2): pass Would this be something to consider for the standard lib, e.g. contextlib? Sturla From haoyi.sg at gmail.com Mon Feb 10 15:51:55 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Mon, 10 Feb 2014 06:51:55 -0800 Subject: [Python-ideas] switch statement as context manager? In-Reply-To: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> References: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> Message-ID: MacroPy has it: https://github.com/lihaoyi/macropy#pattern-matching =) On Mon, Feb 10, 2014 at 6:31 AM, Sturla Molden wrote: > I've noticed that PyExt has a switch statement implemented as a context > manager. > > with switch(foobar): > if case(1): pass > if case(2): pass > > Would this be something to consider for the standard lib, e.g. contextlib? > > Sturla > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rymg19 at gmail.com Mon Feb 10 17:43:08 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Mon, 10 Feb 2014 10:43:08 -0600 Subject: [Python-ideas] switch statement as context manager? In-Reply-To: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> References: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> Message-ID: Probably not. When I wrote PyExt, it was with the sole idea that you can get around stuff by abusing Python's reflection mechanisms. Even the code for the switch statement is very hackish. I don't quite think that belongs in stdlib. On Mon, Feb 10, 2014 at 8:31 AM, Sturla Molden wrote: > I've noticed that PyExt has a switch statement implemented as a context > manager. > > with switch(foobar): > if case(1): pass > if case(2): pass > > Would this be something to consider for the standard lib, e.g. contextlib? > > Sturla > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From grosser.meister.morti at gmx.net Mon Feb 10 19:14:30 2014 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Mon, 10 Feb 2014 19:14:30 +0100 Subject: [Python-ideas] switch statement as context manager? In-Reply-To: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> References: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> Message-ID: <52F91706.7080700@gmx.net> You don't need a context manger here: switch = lambda value: lambda other: value == other case = switch(foobar) if case(1): pass if case(2): pass But I don't think this is at all useful. Maybe if case does something more complex than == (e.g. isinstance for classes and matching of regular expressions etc.). But than it's a bit of black magic. What if value and other both are classes or regular expressions? Other version: case = foobar.__eq__ if case(1): pass if case(2): pass Am 2014-02-10 15:31, schrieb Sturla Molden: > I've noticed that PyExt has a switch statement implemented as a context > manager. > > with switch(foobar): > if case(1): pass > if case(2): pass > > Would this be something to consider for the standard lib, e.g. contextlib? > > Sturla > From storchaka at gmail.com Mon Feb 10 19:34:01 2014 From: storchaka at gmail.com (Serhiy Storchaka) Date: Mon, 10 Feb 2014 20:34:01 +0200 Subject: [Python-ideas] sentinel_exception argument to `iter` In-Reply-To: References: <5487bfdf-13fb-49c1-bab2-4f9945097c4c@googlegroups.com> <36ABE145-12ED-4A87-B3C0-5FBDEAE36699@yahoo.com> Message-ID: 08.02.14 10:32, Terry Reedy ???????(??): > On 2/7/2014 9:28 AM, Serhiy Storchaka wrote: >> 07.02.14 10:36, Terry Reedy ???????(??): >>> I wrote the above with the idea that there would be a third parameter to >>> provide an exception. It would have to have a private default like >>> >>> class _NoException(Exception): pass >> >> Natural default is StopIteration or (). > > That does not work because iter has to know if the user called iter with > only one argument, an iterable, or with two or (with the proposal) > three, with the first argument being a callable. Agree. From lemiant at hotmail.com Mon Feb 10 19:49:55 2014 From: lemiant at hotmail.com (Alex Rodrigues) Date: Mon, 10 Feb 2014 11:49:55 -0700 Subject: [Python-ideas] Inline Functions - idea Message-ID: Over the last few days I've been considering a lot of what we talked about regarding Python scoping and inline functions. Today I found a further use case for inline functions (one which is a fair bit harder to replicate using standard techniques). I was working on writing a basic linear algebra calculator for school and one of the annoying things about doing it is that every time you want to do an element-wise operation on the matrix (which is a list of lists) you have to nest what is essentially one line of code inside a pair of for loops.? Below I've done three basic linear algebra functions which are written as if there was a builtin "inline()" which transformed a standard function into an inline function and also assuming that python had more powerful lambdas. I feel that these represent a good use case, because the inline function serves to make the code both shorter and clearer. import copy @inline def element_wise(func): ? ? func = inline(func)? ? ? for i, row in enumerate(matrix): ? ? ? ? result.append([0]*len(row)) ? ? ? ? for j, el in enumerate(row): ? ? ? ? ? ? func() def is_symmetric(matrix): ? ? element_wise(lambda: if matrix[i][j] != matrix[j][i]: return False) ? ? return True ? ? ? ? ? ?? def matrix_add(matrix, matrix2): ? ? result = copy.deepcopy(matrix) ? ? element_wise(lambda: result[i][j] = matrix[i][j] + matrix2[i][j]) ? ? return result def matrix_multiply(matrix, matrix2): ? ? result = copy.deepcopy(matrix) ? ? element_wise(lambda: result[i][j] = sum([a*b for a,b in zip(matrix[i], [r[j] for r in matrix2])])) ? ? return result From ethan at stoneleaf.us Mon Feb 10 20:26:43 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 10 Feb 2014 11:26:43 -0800 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: <52F927F3.6080009@stoneleaf.us> On 02/10/2014 10:49 AM, Alex Rodrigues wrote: > > Below I've done three basic linear algebra functions which are written > as if there was a builtin "inline()" which transformed a standard > function into an inline function and also assuming that python had more > powerful lambdas. Python isn't going to get more powerful lambdas. If you want your `inline` idea to get the best possible interpretation don't combine it with other non-existent features. -- ~Ethan~ From amber.yust at gmail.com Mon Feb 10 20:20:41 2014 From: amber.yust at gmail.com (Amber Yust) Date: Mon, 10 Feb 2014 19:20:41 +0000 Subject: [Python-ideas] Inline Functions - idea References: Message-ID: I find your example code far less readable than just writing out the loops in the appropriate functions, especially for matrix_multiply. On Mon Feb 10 2014 at 10:51:35 AM, Alex Rodrigues wrote: > Over the last few days I've been considering a lot of what we talked about > regarding Python scoping and inline functions. Today I found a further use > case for inline functions (one which is a fair bit harder to replicate > using standard techniques). I was working on writing a basic linear algebra > calculator for school and one of the annoying things about doing it is that > every time you want to do an element-wise operation on the matrix (which is > a list of lists) you have to nest what is essentially one line of code > inside a pair of for loops. > Below I've done three basic linear algebra functions which are written as > if there was a builtin "inline()" which transformed a standard function > into an inline function and also assuming that python had more powerful > lambdas. I feel that these represent a good use case, because the inline > function serves to make the code both shorter and clearer. > > > import copy > > @inline > def element_wise(func): > func = inline(func) > for i, row in enumerate(matrix): > result.append([0]*len(row)) > for j, el in enumerate(row): > func() > > def is_symmetric(matrix): > element_wise(lambda: if matrix[i][j] != matrix[j][i]: return False) > return True > > def matrix_add(matrix, matrix2): > result = copy.deepcopy(matrix) > element_wise(lambda: result[i][j] = matrix[i][j] + matrix2[i][j]) > return result > > def matrix_multiply(matrix, matrix2): > result = copy.deepcopy(matrix) > element_wise(lambda: result[i][j] = sum([a*b for a,b in zip(matrix[i], > [r[j] for r in matrix2])])) > return result > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemiant at hotmail.com Mon Feb 10 20:44:38 2014 From: lemiant at hotmail.com (Alex Rodrigues) Date: Mon, 10 Feb 2014 12:44:38 -0700 Subject: [Python-ideas] @partials = decorators Message-ID: This is a really cool idea. I'm +1. The first time I learned about decorators the way that you pass them arguments was both very non-intuitive and seemed almost like a sneaky hack rather than an intended feature. # In my opinion this is more intuitive than the old way def annotate(msg, foo, *arg, **kwargs): ? ? print msg ? ? foo(*arg, **kwargs) @annotate("Addition") ? ? def func(a, b): ? ? ? ? print a + b #The old way def annotate(msg): ? ? def deco(foo): ? ? ? ? def wrapper(*args, **kwargs): ? ? ? ? ? ? print msg ? ? ? ? ? ? foo(*args, **kwargs) ? ? ? ? return wrapper ? ? return deco @annotate("Addition") def func(a, b): ? ? print a + b (Ironically I had a really hard time getting the "old" one put together correctly). This also very effectively simplifies a lot of closure cases. There is one thing I don't quite understand, is it necessary to explicitly state that some function is a continuation each time you add arguments? As I understand it, that would mean decorators with arguments have to be called by ?"@@deco(a, b)". This would actually help me out in a real world example at this very moment, which is currently really, really hard to do cleanly. I am using iPython notebook and I'm creating two histograms like so: #data is a pandas Series() objects data.hist() data.hist(log=True) In the notebook these are both reasonably skinny graphs but they stack on top of each other instead of side by side. I looked it up and that can be fixed like so: fig = plt.figure() ax1 = plt.subplot(121) ax2 = plt.subplot(122) data.hist(ax=ax1) data.hist(ax=ax2, log=True) This is pretty cumbersome to write out every time so I wanted a function that could just be called on arbitrary histograms like side_by_side(data1.hist(bottom=0.1, log=True), data2.hist(bins=10)). The problem is that I do not have the axes available when I call the function and once I pass the plots to the function they have already been evaluated and there is no way to add arguments retroactively. With the continuation syntax it would be amazingly easier: def side_by_side(plot1, plot2): ? ? fig = plt.figure() ? ??ax1 = plt.subplot(121) ? ??ax2 = plt.subplot(122) ? ??plot1(ax=ax1) ? ??plot2(ax=ax2) side_by_side(@data.hist(), @data.hist(log=True)) #Technically the first one doesn't need to be a continuation, but whatever. tldr; I like it and see a decent number of applications. Also I'm ?wondering if you'd have to chain continuations @@a('arg1')('arg2')('arg3'). - Alex From lemiant at hotmail.com Mon Feb 10 20:55:42 2014 From: lemiant at hotmail.com (Alex Rodrigues) Date: Mon, 10 Feb 2014 12:55:42 -0700 Subject: [Python-ideas] FW: Inline Functions - idea In-Reply-To: References: , , Message-ID: I have to agree about matrix_multiply, it's not that clean. But try writing it out without inline and you'll find that it is quite a bit longer. If we were aiming for more readability I suppose we could do this def matrix_multiply(matrix, matrix2): ? ? def dot_product(i, j): ? ? ? ? sum = 0 ? ? ? ? for x in range(len(matrix2)): ? ??? ??? ??sum += matrix[i][x] * matrix2[x][j] ? ??? ??return sum result = copy.copy(matrix) element_wise(result[i][j] = dot_product(i, j)) return result Which is still superior to the best syntax currently available. So being concerned about the readablility of the function, while legitimate, is not actually a knock on the syntax just on the way I wrote it. > ________________________________ >> From: amber.yust at gmail.com >> Date: Mon, 10 Feb 2014 19:20:41 +0000 >> Subject: Re: [Python-ideas] Inline Functions - idea >> To: lemiant at hotmail.com; python-ideas at python.org >> >> I find your example code far less readable than just writing out the >> loops in the appropriate functions, especially for matrix_multiply. >> >> On Mon Feb 10 2014 at 10:51:35 AM, Alex Rodrigues >> > wrote: >> Over the last few days I've been considering a lot of what we talked >> about regarding Python scoping and inline functions. Today I found a >> further use case for inline functions (one which is a fair bit harder >> to replicate using standard techniques). I was working on writing a >> basic linear algebra calculator for school and one of the annoying >> things about doing it is that every time you want to do an element-wise >> operation on the matrix (which is a list of lists) you have to nest >> what is essentially one line of code inside a pair of for loops. >> Below I've done three basic linear algebra functions which are written >> as if there was a builtin "inline()" which transformed a standard >> function into an inline function and also assuming that python had more >> powerful lambdas. I feel that these represent a good use case, because >> the inline function serves to make the code both shorter and clearer. >> >> >> import copy >> >> @inline >> def element_wise(func): >> func = inline(func) >> for i, row in enumerate(matrix): >> result.append([0]*len(row)) >> for j, el in enumerate(row): >> func() >> >> def is_symmetric(matrix): >> element_wise(lambda: if matrix[i][j] != matrix[j][i]: return False) >> return True >> >> def matrix_add(matrix, matrix2): >> result = copy.deepcopy(matrix) >> element_wise(lambda: result[i][j] = matrix[i][j] + matrix2[i][j]) >> return result >> >> def matrix_multiply(matrix, matrix2): >> result = copy.deepcopy(matrix) >> element_wise(lambda: result[i][j] = sum([a*b for a,b in >> zip(matrix[i], [r[j] for r in matrix2])])) >> return result >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ From amber.yust at gmail.com Mon Feb 10 21:06:02 2014 From: amber.yust at gmail.com (Amber Yust) Date: Mon, 10 Feb 2014 20:06:02 +0000 Subject: [Python-ideas] FW: Inline Functions - idea References: Message-ID: In which case you could simply do e.g.: result = [[dot_product(i, j) for i, el in enumerate(row)] for j, row in enumerate(matrix)] On Mon Feb 10 2014 at 11:57:20 AM, Alex Rodrigues wrote: > I have to agree about matrix_multiply, it's not that clean. But try > writing it out without inline and you'll find that it is quite a bit > longer. If we were aiming for more readability I suppose we could do this > > def matrix_multiply(matrix, matrix2): > def dot_product(i, j): > sum = 0 > for x in range(len(matrix2)): > sum += matrix[i][x] * matrix2[x][j] > return sum > > result = copy.copy(matrix) > element_wise(result[i][j] = dot_product(i, j)) > return result > > Which is still superior to the best syntax currently available. So being > concerned about the readablility of the function, while legitimate, is not > actually a knock on the syntax just on the way I wrote it. > > > ________________________________ > >> From: amber.yust at gmail.com > >> Date: Mon, 10 Feb 2014 19:20:41 +0000 > >> Subject: Re: [Python-ideas] Inline Functions - idea > >> To: lemiant at hotmail.com; python-ideas at python.org > >> > >> I find your example code far less readable than just writing out the > >> loops in the appropriate functions, especially for matrix_multiply. > >> > >> On Mon Feb 10 2014 at 10:51:35 AM, Alex Rodrigues > >> > wrote: > >> Over the last few days I've been considering a lot of what we talked > >> about regarding Python scoping and inline functions. Today I found a > >> further use case for inline functions (one which is a fair bit harder > >> to replicate using standard techniques). I was working on writing a > >> basic linear algebra calculator for school and one of the annoying > >> things about doing it is that every time you want to do an element-wise > >> operation on the matrix (which is a list of lists) you have to nest > >> what is essentially one line of code inside a pair of for loops. > >> Below I've done three basic linear algebra functions which are written > >> as if there was a builtin "inline()" which transformed a standard > >> function into an inline function and also assuming that python had more > >> powerful lambdas. I feel that these represent a good use case, because > >> the inline function serves to make the code both shorter and clearer. > >> > >> > >> import copy > >> > >> @inline > >> def element_wise(func): > >> func = inline(func) > >> for i, row in enumerate(matrix): > >> result.append([0]*len(row)) > >> for j, el in enumerate(row): > >> func() > >> > >> def is_symmetric(matrix): > >> element_wise(lambda: if matrix[i][j] != matrix[j][i]: return False) > >> return True > >> > >> def matrix_add(matrix, matrix2): > >> result = copy.deepcopy(matrix) > >> element_wise(lambda: result[i][j] = matrix[i][j] + matrix2[i][j]) > >> return result > >> > >> def matrix_multiply(matrix, matrix2): > >> result = copy.deepcopy(matrix) > >> element_wise(lambda: result[i][j] = sum([a*b for a,b in > >> zip(matrix[i], [r[j] for r in matrix2])])) > >> return result > >> _______________________________________________ > >> Python-ideas mailing list > >> Python-ideas at python.org > >> https://mail.python.org/mailman/listinfo/python-ideas > >> Code of Conduct: http://python.org/psf/codeofconduct/ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Mon Feb 10 21:39:04 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Mon, 10 Feb 2014 12:39:04 -0800 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: On Feb 10, 2014, at 10:49, Alex Rodrigues wrote: The only reason you think you need more powerful lambdas and inline functions is that you're ignoring features Python already has. For example; > def is_symmetric(matrix): > element_wise(lambda: if matrix[i][j] != matrix[j][i]: return False) > return True This requires full statements as lambdas, and inline functions, and some kind of "multi-level return", where the implicit lambda return is level -1 so an explicit one returns from the outer (dynamic) scope, right? But element_wise is just a genexpr in disguise. And with the builtin all function you don't need early return. And that early return was the only reason your if needed to be a statement. So: return all(matrix[i][j] == matrix[j][i] for i, j in element_wise(matrix)) And this element_wise isn't anything magic, it's just product(range(len(matrix)), repeat=2). From grosser.meister.morti at gmx.net Mon Feb 10 22:25:50 2014 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Mon, 10 Feb 2014 22:25:50 +0100 Subject: [Python-ideas] switch statement as context manager? In-Reply-To: <52F91706.7080700@gmx.net> References: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> <52F91706.7080700@gmx.net> Message-ID: <52F943DE.2080507@gmx.net> See: https://gist.github.com/panzi/8924450 But again, I don't recommend to use this. Just to demonstrate what is already possible with the most simple constructs. Am 2014-02-10 19:14, schrieb Mathias Panzenb?ck: > You don't need a context manger here: > > switch = lambda value: lambda other: value == other > > case = switch(foobar) > if case(1): pass > if case(2): pass > > > But I don't think this is at all useful. Maybe if case does something more complex than == (e.g. isinstance for classes > and matching of regular expressions etc.). But than it's a bit of black magic. What if value and other both are classes > or regular expressions? > > Other version: > > case = foobar.__eq__ > if case(1): pass > if case(2): pass > > Am 2014-02-10 15:31, schrieb Sturla Molden: >> I've noticed that PyExt has a switch statement implemented as a context >> manager. >> >> with switch(foobar): >> if case(1): pass >> if case(2): pass >> >> Would this be something to consider for the standard lib, e.g. contextlib? >> >> Sturla >> > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > From greg.ewing at canterbury.ac.nz Tue Feb 11 00:29:04 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 11 Feb 2014 12:29:04 +1300 Subject: [Python-ideas] switch statement as context manager? In-Reply-To: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> References: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> Message-ID: <52F960C0.4020900@canterbury.ac.nz> Sturla Molden wrote: > I've noticed that PyExt has a switch statement implemented as a context > manager. > > with switch(foobar): > if case(1): pass > if case(2): pass What advantage does this have over an if-else chain? -- Greg From turnbull at sk.tsukuba.ac.jp Tue Feb 11 07:12:47 2014 From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull) Date: Tue, 11 Feb 2014 15:12:47 +0900 Subject: [Python-ideas] Question about `list.insert` In-Reply-To: <20140207233315.519849e6@fsol> References: <7b1c0a37-4873-454f-bd0d-458451e6621b@googlegroups.com> <49931276-ddee-44b2-9de3-992110a1e841@googlegroups.com> <6cd218d1-a29e-4f0e-80f6-434142304c79@googlegroups.com> <87eh3fmbcq.fsf@uwakimon.sk.tsukuba.ac.jp> <20140207155846.1d0561bc@fsol> <878utmmw2i.fsf@uwakimon.sk.tsukuba.ac.jp> <20140207233315.519849e6@fsol> Message-ID: <87ob2exjmo.fsf@uwakimon.sk.tsukuba.ac.jp> Antoine Pitrou writes: > On Sat, 08 Feb 2014 06:50:29 +0900 > "Stephen J. Turnbull" > wrote: > > But the OP refers to *allocation*, and specifically disallows > > "reserving space". > > Ok, so you're arguing about a misunderstanding by the OP about how > memory allocation works. Yes. Why did you think I was doing anything but responding to the OP's message? From techtonik at gmail.com Tue Feb 11 05:35:24 2014 From: techtonik at gmail.com (anatoly techtonik) Date: Tue, 11 Feb 2014 07:35:24 +0300 Subject: [Python-ideas] Make it optional Message-ID: Flexibility of something is the fact that you are not forced to use it in some unnatural way. I'd say that most PEPs related to core Python are flexible, but I am not sure about the rest. PEPs are made to resolve ambiguities. But it doesn't mean limiting flexibility and setting hard limitation. What is hard limitation standard? For example, marking version you can not parse as pre-release. PIP actually does this, which prevents all packages with versions that are not comply with "versioning PEP" from installing. "pre-release" is a feature. Inflexible, tempting to guess, getting in a way, implicit and confusing. I just filled three or four bugs about it. How come that it appeared at all? Well, because it is useful. In some cases. But the problem is that it affects you even if you don't need it. "Make it optional" is a very good principle for those cases when you can not predict how users are going to use your feature and if they need it at all. -- anatoly t. From g.brandl at gmx.net Tue Feb 11 08:20:55 2014 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 11 Feb 2014 08:20:55 +0100 Subject: [Python-ideas] Make it optional In-Reply-To: References: Message-ID: Am 11.02.2014 05:35, schrieb anatoly techtonik: > Flexibility of something is the fact that you are not forced > to use it in some unnatural way. I'd say that most PEPs > related to core Python are flexible, but I am not sure about > the rest. > > PEPs are made to resolve ambiguities. But it doesn't mean > limiting flexibility and setting hard limitation. What is hard > limitation standard? For example, marking version you can > not parse as pre-release. PIP actually does this, which > prevents all packages with versions that are not comply with > "versioning PEP" from installing. > > "pre-release" is a feature. Inflexible, tempting to guess, > getting in a way, implicit and confusing. I just filled three or > four bugs about it. How come that it appeared at all? Well, > because it is useful. In some cases. But the problem is that > it affects you even if you don't need it. > > "Make it optional" is a very good principle for those cases > when you can not predict how users are going to use your > feature and if they need it at all. I have no idea what you are talking about. Care to give some context? Georg From p.f.moore at gmail.com Tue Feb 11 08:48:39 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 11 Feb 2014 07:48:39 +0000 Subject: [Python-ideas] Make it optional In-Reply-To: References: Message-ID: On 11 February 2014 07:20, Georg Brandl wrote: >> "Make it optional" is a very good principle for those cases >> when you can not predict how users are going to use your >> feature and if they need it at all. > > I have no idea what you are talking about. Care to give some context? Anatoly has just raised a series of issues against pip, all essentially reiterating the same point that he doesn't like pip's new handling of prerelease vs release versions (which was thrashed out and agreed in a PEP) I assume he's trying to get support for some vague meta-point here in the hope that by doing so he'll feel justified in arguing further on the pip issues :-( Paul From stephen at xemacs.org Tue Feb 11 09:45:01 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 11 Feb 2014 17:45:01 +0900 Subject: [Python-ideas] Make it optional In-Reply-To: References: Message-ID: <87lhxixcky.fsf@uwakimon.sk.tsukuba.ac.jp> anatoly techtonik writes: > PEPs are made to resolve ambiguities. But it doesn't mean > limiting flexibility and setting hard limitation. What is hard > limitation standard? For example, marking version you can > not parse as pre-release. PIP actually does this, which > prevents all packages with versions that are not comply with > "versioning PEP" from installing. You say you've filed a bug. That's the right thing to do. AFAICT, to the extent that it really *prevents* installation, it doesn't conform to PEP 440 (see "Handling of Pre-Releases" and "Direct References" in the PEP). Since it is arguably a bug, it doesn't need discussion here. From techtonik at gmail.com Tue Feb 11 15:33:33 2014 From: techtonik at gmail.com (anatoly techtonik) Date: Tue, 11 Feb 2014 17:33:33 +0300 Subject: [Python-ideas] Make it optional In-Reply-To: References: Message-ID: On Tue, Feb 11, 2014 at 10:48 AM, Paul Moore wrote: > On 11 February 2014 07:20, Georg Brandl wrote: >>> "Make it optional" is a very good principle for those cases >>> when you can not predict how users are going to use your >>> feature and if they need it at all. >> >> I have no idea what you are talking about. Care to give some context? > > Anatoly has just raised a series of issues against pip, all > essentially reiterating the same point that he doesn't like pip's new > handling of prerelease vs release versions (which was thrashed out and > agreed in a PEP) I assume he's trying to get support for some vague > meta-point here in the hope that by doing so he'll feel justified in > arguing further on the pip issues :-( Right. Smart people invented prerelease feature for PyPI, which works the following way: - if pip thinks that your version is not PEP compliant, it won't install it, because it thinks that everything what is not PEP aware is prerelease I think that final decision whatever package version is prerelease or not should be made by package maintainer, and (s)he should have a final judgement over this fact. But inventors of prerelease feature didn't even think that people may not want and still don't want this features. That's why I proposed "Make it optional" principle as an engineering practice for solutions that affect the whole net. From rymg19 at gmail.com Tue Feb 11 16:22:57 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Tue, 11 Feb 2014 09:22:57 -0600 Subject: [Python-ideas] switch statement as context manager? In-Reply-To: <52F960C0.4020900@canterbury.ac.nz> References: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> <52F960C0.4020900@canterbury.ac.nz> Message-ID: It looks cooler. It also feels slightly less aggravating. On Mon, Feb 10, 2014 at 5:29 PM, Greg Ewing wrote: > Sturla Molden wrote: > >> I've noticed that PyExt has a switch statement implemented as a context >> manager. >> >> with switch(foobar): >> if case(1): pass >> if case(2): pass >> > > What advantage does this have over an if-else chain? > > -- > Greg > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Tue Feb 11 16:25:28 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 12 Feb 2014 02:25:28 +1100 Subject: [Python-ideas] switch statement as context manager? In-Reply-To: References: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> <52F960C0.4020900@canterbury.ac.nz> Message-ID: On Wed, Feb 12, 2014 at 2:22 AM, Ryan Gonzalez wrote: > It looks cooler. It also feels slightly less aggravating. > > > On Mon, Feb 10, 2014 at 5:29 PM, Greg Ewing > wrote: >> >> Sturla Molden wrote: >>> >>> I've noticed that PyExt has a switch statement implemented as a context >>> manager. >>> >>> with switch(foobar): >>> if case(1): pass >>> if case(2): pass >> >> >> What advantage does this have over an if-else chain? >> > Since it fundamentally _is_ an if chain (without the elses), how does it feel less aggravating than one? ChrisA From rymg19 at gmail.com Tue Feb 11 16:37:48 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Tue, 11 Feb 2014 09:37:48 -0600 Subject: [Python-ideas] switch statement as context manager? In-Reply-To: References: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> <52F960C0.4020900@canterbury.ac.nz> Message-ID: Because it doesn't *look *like one. On Tue, Feb 11, 2014 at 9:25 AM, Chris Angelico wrote: > On Wed, Feb 12, 2014 at 2:22 AM, Ryan Gonzalez wrote: > > It looks cooler. It also feels slightly less aggravating. > > > > > > On Mon, Feb 10, 2014 at 5:29 PM, Greg Ewing > > > wrote: > >> > >> Sturla Molden wrote: > >>> > >>> I've noticed that PyExt has a switch statement implemented as a context > >>> manager. > >>> > >>> with switch(foobar): > >>> if case(1): pass > >>> if case(2): pass > >> > >> > >> What advantage does this have over an if-else chain? > >> > > > > Since it fundamentally _is_ an if chain (without the elses), how does > it feel less aggravating than one? > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From hernan.grecco at gmail.com Tue Feb 11 16:53:46 2014 From: hernan.grecco at gmail.com (Hernan Grecco) Date: Tue, 11 Feb 2014 13:53:46 -0200 Subject: [Python-ideas] Make it optional In-Reply-To: References: Message-ID: On Tue, Feb 11, 2014 at 11:33 AM, anatoly techtonik wrote: > > I think that final decision whatever package version is prerelease or not > should be made by package maintainer, and (s)he should have a final > judgement over this fact. But inventors of prerelease feature didn't even > think that people may not want and still don't want this features. > But package maintainers have the final judgement! A package is NOT a pre-release if it has a PEP 440 compliant version number without any pre-release or development segment. A simple rule. The problem is how to deal with old packages with non compliant versions which now are not get installed by default. I would say: - If you are the maintainer of the legacy package, just release a new version with a compliant version. - If you are the maintainer of a project that needs this legacy package, just adjust your requirements file as described in [0] The versioning scheme proposed in PEP440 has a lot of advantages (which are described in the PEP). But it comes with some rules that we need to comply. cheers, Hern?n [0] http://www.pip-installer.org/en/latest/logic.html#pre-release-versions On Tue, Feb 11, 2014 at 11:33 AM, anatoly techtonik wrote: > On Tue, Feb 11, 2014 at 10:48 AM, Paul Moore wrote: > > On 11 February 2014 07:20, Georg Brandl wrote: > >>> "Make it optional" is a very good principle for those cases > >>> when you can not predict how users are going to use your > >>> feature and if they need it at all. > >> > >> I have no idea what you are talking about. Care to give some context? > > > > Anatoly has just raised a series of issues against pip, all > > essentially reiterating the same point that he doesn't like pip's new > > handling of prerelease vs release versions (which was thrashed out and > > agreed in a PEP) I assume he's trying to get support for some vague > > meta-point here in the hope that by doing so he'll feel justified in > > arguing further on the pip issues :-( > > Right. Smart people invented prerelease feature for PyPI, which works > the following way: > > - if pip thinks that your version is not PEP compliant, it won't install > it, > because it thinks that everything what is not PEP aware is prerelease > > I think that final decision whatever package version is prerelease or not > should be made by package maintainer, and (s)he should have a final > judgement over this fact. But inventors of prerelease feature didn't even > think that people may not want and still don't want this features. > > That's why I proposed "Make it optional" principle as an engineering > practice for solutions that affect the whole net. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ned at nedbatchelder.com Tue Feb 11 17:44:45 2014 From: ned at nedbatchelder.com (Ned Batchelder) Date: Tue, 11 Feb 2014 11:44:45 -0500 Subject: [Python-ideas] switch statement as context manager? In-Reply-To: References: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> <52F960C0.4020900@canterbury.ac.nz> Message-ID: <52FA537D.70605@nedbatchelder.com> On 2/11/14 10:37 AM, Ryan Gonzalez wrote: > Because it doesn't /look /like one. If it *is* one, it's a good thing to *look* like one. PS: could you bottom-post? It makes it easier to follow the thread. Thanks :) --Ned. > > > On Tue, Feb 11, 2014 at 9:25 AM, Chris Angelico > wrote: > > On Wed, Feb 12, 2014 at 2:22 AM, Ryan Gonzalez > wrote: > > It looks cooler. It also feels slightly less aggravating. > > > > > > On Mon, Feb 10, 2014 at 5:29 PM, Greg Ewing > > > > wrote: > >> > >> Sturla Molden wrote: > >>> > >>> I've noticed that PyExt has a switch statement implemented as > a context > >>> manager. > >>> > >>> with switch(foobar): > >>> if case(1): pass > >>> if case(2): pass > >> > >> > >> What advantage does this have over an if-else chain? > >> > > > > Since it fundamentally _is_ an if chain (without the elses), how does > it feel less aggravating than one? > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > > > -- > Ryan > If anybody ever asks me why I prefer C++ to C, my answer will be > simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't > think that was nul-terminated." > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rymg19 at gmail.com Tue Feb 11 17:49:59 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Tue, 11 Feb 2014 10:49:59 -0600 Subject: [Python-ideas] switch statement as context manager? In-Reply-To: <52FA537D.70605@nedbatchelder.com> References: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> <52F960C0.4020900@canterbury.ac.nz> <52FA537D.70605@nedbatchelder.com> Message-ID: On Tue, Feb 11, 2014 at 10:44 AM, Ned Batchelder wrote: > > On 2/11/14 10:37 AM, Ryan Gonzalez wrote: > > Because it doesn't *look *like one. > > > If it *is* one, it's a good thing to *look* like one. > > PS: could you bottom-post? It makes it easier to follow the thread. > Thanks :) > > --Ned. > > > > On Tue, Feb 11, 2014 at 9:25 AM, Chris Angelico wrote: > >> On Wed, Feb 12, 2014 at 2:22 AM, Ryan Gonzalez wrote: >> > It looks cooler. It also feels slightly less aggravating. >> > >> > >> > On Mon, Feb 10, 2014 at 5:29 PM, Greg Ewing < >> greg.ewing at canterbury.ac.nz> >> > wrote: >> >> >> >> Sturla Molden wrote: >> >>> >> >>> I've noticed that PyExt has a switch statement implemented as a >> context >> >>> manager. >> >>> >> >>> with switch(foobar): >> >>> if case(1): pass >> >>> if case(2): pass >> >> >> >> >> >> What advantage does this have over an if-else chain? >> >> >> > >> >> Since it fundamentally _is_ an if chain (without the elses), how does >> it feel less aggravating than one? >> >> ChrisA >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > > -- > Ryan > If anybody ever asks me why I prefer C++ to C, my answer will be simple: > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was > nul-terminated." > > > > _______________________________________________ > Python-ideas mailing listPython-ideas at python.orghttps://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > Depends on whether or not you actually like the way it looks. P.S. Sorry, GMail top-posts by default. -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From flying-sheep at web.de Tue Feb 11 17:58:16 2014 From: flying-sheep at web.de (Philipp A.) Date: Tue, 11 Feb 2014 17:58:16 +0100 Subject: [Python-ideas] switch statement as context manager? In-Reply-To: References: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> <52F960C0.4020900@canterbury.ac.nz> <52FA537D.70605@nedbatchelder.com> Message-ID: 2014-02-11 17:44 GMT+01:00 Ned Batchelder : If it *is* one, it?s a good thing to *look* like one. well, you also get to remove the repeated variable name. but a real switch statement should take advantage of python?s hashing. the best we have is def _handle_spam(): vomit() def _handle_eggs(): yum() handlers = { 'spam': _handle_spam, 'eggs': _handle_eggs, } handlers[food]() which makes me _handle_spam(). ------------------------------ the problem of a real switch statement is apparently that there?s no good syntax: switch food case 'spam': ... is strange due to the missing colon. we have that nowhere in python. yet switch food: case 'spam': ... is strange because after the colon, everywhere else is an indentation. but switch food: case 'spam': ... is indentation overkill. nobody has found a good syntax yet. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rymg19 at gmail.com Tue Feb 11 18:26:46 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Tue, 11 Feb 2014 11:26:46 -0600 Subject: [Python-ideas] switch statement as context manager? In-Reply-To: References: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> <52F960C0.4020900@canterbury.ac.nz> <52FA537D.70605@nedbatchelder.com> Message-ID: C and C++ have: switch(value) { case 1: do_stuff; break; case 2: case 3: do_stuff_2(); break; default: break; } On Tue, Feb 11, 2014 at 10:58 AM, Philipp A. wrote: > 2014-02-11 17:44 GMT+01:00 Ned Batchelder : > > If it *is* one, it's a good thing to *look* like one. > > well, you also get to remove the repeated variable name. > > but a real switch statement should take advantage of python's hashing. the > best we have is > > def _handle_spam(): > vomit() > def _handle_eggs(): > yum() > > handlers = { > 'spam': _handle_spam, > 'eggs': _handle_eggs, > } > > handlers[food]() > > which makes me _handle_spam(). > ------------------------------ > > the problem of a real switch statement is apparently that there's no good > syntax: > > switch food > case 'spam': > ... > > is strange due to the missing colon. we have that nowhere in python. yet > > switch food: > case 'spam': > ... > > is strange because after the colon, everywhere else is an indentation. but > > switch food: > case 'spam': > ... > > is indentation overkill. nobody has found a good syntax yet. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From skip at pobox.com Tue Feb 11 19:27:52 2014 From: skip at pobox.com (Skip Montanaro) Date: Tue, 11 Feb 2014 12:27:52 -0600 Subject: [Python-ideas] switch statement as context manager? In-Reply-To: References: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> <52F960C0.4020900@canterbury.ac.nz> <52FA537D.70605@nedbatchelder.com> Message-ID: On Tue, Feb 11, 2014 at 10:58 AM, Philipp A. wrote: > the problem of a real switch statement is apparently that there?s no good > syntax Probably worth mentioning the now 5+yo rejected PEP: http://www.python.org/dev/peps/pep-3103/ Skip From liam.marsh.home at gmail.com Tue Feb 11 19:38:03 2014 From: liam.marsh.home at gmail.com (Liam Marsh) Date: Tue, 11 Feb 2014 19:38:03 +0100 Subject: [Python-ideas] int() Message-ID: Hello, do you think it is possible to operate with bytes like with ints? useing the fact than ord('A')==65 ; will it possible to use b'A' like 65 or 00100001, in binary, for example for encryption of files, etc... thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From geoffspear at gmail.com Tue Feb 11 20:21:30 2014 From: geoffspear at gmail.com (Geoffrey Spear) Date: Tue, 11 Feb 2014 14:21:30 -0500 Subject: [Python-ideas] int() In-Reply-To: References: Message-ID: On Tue, Feb 11, 2014 at 1:38 PM, Liam Marsh wrote: > Hello, > do you think it is possible to operate with bytes like with ints? > useing the fact than > ord('A')==65 ; > will it possible to use b'A' like 65 or 00100001, in binary, for example for > encryption of files, etc... > thank you. What idea are you proposing here? You can already call ord() on the single byte or iterate a bytes object to get integer values. Being able to treat b"A" itself as identical to the int 65, if that's what you're suggesting, doesn't really make sense. From techtonik at gmail.com Tue Feb 11 21:11:02 2014 From: techtonik at gmail.com (anatoly techtonik) Date: Tue, 11 Feb 2014 23:11:02 +0300 Subject: [Python-ideas] Make it optional In-Reply-To: References: Message-ID: On Tue, Feb 11, 2014 at 6:53 PM, Hernan Grecco wrote: >> >> >> I think that final decision whatever package version is prerelease or not >> should be made by package maintainer, and (s)he should have a final >> judgement over this fact. But inventors of prerelease feature didn't even >> think that people may not want and still don't want this features. > > But package maintainers have the final judgement! A package is NOT a > pre-release if it has a PEP 440 compliant version number without any > pre-release or development segment. A simple rule. Except that for some of my projects history, documentation, milestones and tags all use (semantic) versioning, which appears to be incompatible with this PEP. > The problem is how to deal with old packages with non compliant versions > which now are not get installed by default. Although it is out of scope of this idea, but can you tell me why you do you want to deal with non compliant versions? I hope there are better arguments that just for this fuzzy warm feeling of total compliance. > I would say: > - If you are the maintainer of the legacy package, just release a new > version with a compliant version. > - If you are the maintainer of a project that needs this legacy package, > just adjust your requirements file as described in [0] You forgot to mention changing current development instructions, development toolchain to rewrite some internal logic to sort old and new versions correctly. No matter how big the PEP 440 is going to get, it unlikely to cover all use cases that people have for package versions. Look at Ubuntu vs Debian if you want to get impression what kind of versioning scheme is pretty possible when you have to maintain several custom packages for different projects against upstream. You can really provide an ton of useful advice to me, but really - just let people handle it themselves - make it optional. > The versioning scheme proposed in PEP440 has a lot of advantages (which are > described in the PEP). But it comes with some rules that we need to comply. I am not sure what advantages are you talking about. They seem to be pretty subjective and for some specific projects I just want to opt-out. From cs at zip.com.au Tue Feb 11 22:45:55 2014 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 12 Feb 2014 08:45:55 +1100 Subject: [Python-ideas] int() In-Reply-To: References: Message-ID: <20140211214555.GA60012@cskk.homeip.net> On 11Feb2014 19:38, Liam Marsh wrote: > do you think it is possible to operate with bytes like with ints? > useing the fact than > ord('A')==65 ; > will it possible to use b'A' like 65 or 00100001, in binary, for example > for encryption of files, etc... In Python 2, b'A' makes a str. In python 3, b'A' makes a bytes object, and array of small (8-bit) ints. Element 0 of that array _is_ the byte 65, an integer. So: Python 3.3.3 (default, Nov 23 2013, 13:21:35) [GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> x=b'A' >>> x b'A' >>> x[0] 65 >>> Just move to python 3 for your code and you'll be better off for what you seem to be wanting. Cheers, -- Cameron Simpson More computing sins are committed in the name of efficiency (without necessarily achieving it) than for any other single reason - including blind stupidity. - W.A. Wulf From steve at pearwood.info Tue Feb 11 23:10:57 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 12 Feb 2014 09:10:57 +1100 Subject: [Python-ideas] int() In-Reply-To: References: Message-ID: <20140211221057.GM3799@ando> On Tue, Feb 11, 2014 at 02:21:30PM -0500, Geoffrey Spear wrote: > On Tue, Feb 11, 2014 at 1:38 PM, Liam Marsh wrote: > > Hello, > > do you think it is possible to operate with bytes like with ints? > > useing the fact than > > ord('A')==65 ; > > will it possible to use b'A' like 65 or 00100001, in binary, for example for > > encryption of files, etc... > > thank you. > > What idea are you proposing here? You can already call ord() on the > single byte or iterate a bytes object to get integer values. Being > able to treat b"A" itself as identical to the int 65, if that's what > you're suggesting, doesn't really make sense. It makes sense, that it to say the idea isn't entirely incoherent. It just is a bad idea. That is to say, for this idea to work, we'd want things like these to be true: b'A' == 65 which implies that int(b'A') == 65 b'A' < 100 b'CAT' == 4407636 Seems okay so far, but now we get into behaviour that contradicts existing behaviour: b'<'*2 == b'x' # instead of b'<<' b'A' + b'B' == b'\x83' # instead of b'AB' And things which simply look wrong and bizarre: int(b'2') == 50 # instead of 2 b'B' - b'A' == b'\x01' b'A' - b'B' == -1 b'B'//2 == b'!' b'B'/2 == 33.0 It would also imply some pretty strange things about ints: since b'CAT' equals 4407636, and b'CAT'[2] equals 84, does that mean we want to be able to say 4407636[2] and get 84? -- Steven From greg.ewing at canterbury.ac.nz Tue Feb 11 23:22:56 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 12 Feb 2014 11:22:56 +1300 Subject: [Python-ideas] switch statement as context manager? In-Reply-To: References: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> <52F960C0.4020900@canterbury.ac.nz> <52FA537D.70605@nedbatchelder.com> Message-ID: <52FAA2C0.9000300@canterbury.ac.nz> Skip Montanaro wrote: > On Tue, Feb 11, 2014 at 10:58 AM, Philipp A. wrote: > >>the problem of a real switch statement is apparently that there?s no good >>syntax Last time this was discussed at length, the main sticking point seemed to be that people generally expect a switch statement to be implemented by something more efficient than a series of if-tests. But that's extremely difficult to do in Python except when all of the case values are literals, which is a very special situation. So in general, any switch statement in Python would just be pure syntactic sugar for an if-else chain, and that doesn't seem to be considered enough of an advantage to bother with. -- Greg From steve at pearwood.info Tue Feb 11 23:29:49 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 12 Feb 2014 09:29:49 +1100 Subject: [Python-ideas] Inline Functions - idea In-Reply-To: References: Message-ID: <20140211222949.GN3799@ando> On Mon, Feb 10, 2014 at 11:49:55AM -0700, Alex Rodrigues wrote: > Over the last few days I've been considering a lot of what we talked > about regarding Python scoping and inline functions. Alex, can you explain the difference (if any) between your proposal and dynamic scoping? -- Steven From bruce at leapyear.org Wed Feb 12 00:05:53 2014 From: bruce at leapyear.org (Bruce Leban) Date: Tue, 11 Feb 2014 15:05:53 -0800 Subject: [Python-ideas] combining two threads: switch statements and inline functions Message-ID: What if we had the ability to write dictionaries with inline functions in them. def sample(i, op, j): switcher = {{ '-':: if i > j: return i - j else: return j - i;; '+':: return i + j;; }} return switcher[op]() Please don't pay attention to {{ :: ;; }} which would never be the actual syntax or what this code is doing (nothing useful) or whether i or j should be passed as parameters rather than closure. This is equivalent to: def sample(i, op, j): def diff(): if i > j: return i - j else: return j - i def add(): return i + j switcher = { '-': diff, '+': add, } return switcher[op]() I don't know if there's a good idea here, but I do know that this pattern of writing dispatchable functions like this is fairly common, just as the switch pattern is. There are some obvious problems, most notably that in the switch statement, the scope of the statements is the same as the enclosing scope while in this code, each function has it's own scope. Notwithstanding that, I thought it was an interesting enough idea to share. --- Bruce Learn how hackers think: http://j.mp/gruyere-security -------------- next part -------------- An HTML attachment was scrubbed... URL: From rymg19 at gmail.com Wed Feb 12 00:40:26 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Tue, 11 Feb 2014 17:40:26 -0600 Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: References: Message-ID: That's an interesting idea. I'm not sure, though, whether that should be considered an inline function or a multi-line lambda. On Tue, Feb 11, 2014 at 5:05 PM, Bruce Leban wrote: > What if we had the ability to write dictionaries with inline functions in > them. > > def sample(i, op, j): > switcher = {{ > '-':: if i > j: > return i - j > else: > return j - i;; > '+':: return i + j;; > }} > return switcher[op]() > > > Please don't pay attention to {{ :: ;; }} which would never be the actual > syntax or what this code is doing (nothing useful) or whether i or j should > be passed as parameters rather than closure. This is equivalent to: > > def sample(i, op, j): > def diff(): > if i > j: > return i - j > else: > return j - i > def add(): > return i + j > switcher = { > '-': diff, > '+': add, > } > return switcher[op]() > > > I don't know if there's a good idea here, but I do know that this pattern > of writing dispatchable functions like this is fairly common, just as the > switch pattern is. There are some obvious problems, most notably that in > the switch statement, the scope of the statements is the same as the > enclosing scope while in this code, each function has it's own scope. > Notwithstanding that, I thought it was an interesting enough idea to share. > > --- Bruce > Learn how hackers think: http://j.mp/gruyere-security > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Wed Feb 12 00:42:27 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 11 Feb 2014 15:42:27 -0800 Subject: [Python-ideas] switch statement as context manager? In-Reply-To: References: <244268823413735102.885411sturla.molden-gmail.com@news.gmane.org> <52F960C0.4020900@canterbury.ac.nz> <52FA537D.70605@nedbatchelder.com> Message-ID: <8087EC2F-FA14-4819-81E6-A3F7720AECBE@yahoo.com> On Feb 11, 2014, at 8:58, "Philipp A." wrote: > 2014-02-11 17:44 GMT+01:00 Ned Batchelder : > > If it is one, it?s a good thing to look like one. > > well, you also get to remove the repeated variable name. > Only by repeating the word "case" instead. Compare your syntax: switch some_long_expr(): if case(0): pass if case(1): pass if case(2): pass ... to what you can already write: case = some_long_expr() if case == 0: pass if case == 1: pass if case == 2: pass So you haven't saved anything at all. You've added more syntax, more characters, and an extra level of indentation. (Note that in more realistic uses, with real statements being controlled, your syntax indents them all twice instead of once.) If you really like using parens instead of equals, you can even do that: case = some_long_expr().__eq__ if case(0): pass if case(1): pass if case(2): pass Maybe you wanted your switch statement to automatically turn each if case into an elif. If so, then the existing alternative does require an extra "el" at the head of each line... But I think it's a lot clearer that at most one of these things will be executed that way. Also, of course, the elif chain is more flexible. Consider how hard it would be to write this as a switch: case = some_long_expr().__lt__ if case(0): pass elif case(1): pass elif case(2): pass Or maybe you wanted it to turn the chain of if case statements into a dict lookup or something. If so, that's even more limited, and a lot more magical, for very little benefit. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Wed Feb 12 00:45:57 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 12 Feb 2014 10:45:57 +1100 Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: References: Message-ID: On Wed, Feb 12, 2014 at 10:05 AM, Bruce Leban wrote: > What if we had the ability to write dictionaries with inline functions in > them. > > def sample(i, op, j): > switcher = {{ > '-':: if i > j: > return i - j > else: > return j - i;; > '+':: return i + j;; > }} > return switcher[op]() > > > Please don't pay attention to {{ :: ;; }} which would never be the actual > syntax or what this code is doing (nothing useful) or whether i or j should > be passed as parameters rather than closure. This is equivalent to: > > def sample(i, op, j): > def diff(): > if i > j: > return i - j > else: > return j - i > def add(): > return i + j > switcher = { > '-': diff, > '+': add, > } > return switcher[op]() > > Hmm. In terms of scoping rules and so on, would it be cleaner to instead make it more equivalent to a big fat expression? Here's legal, but somewhat ugly, Python code to do the same thing: def throw(ex): """Raise an exception in an expression context.""" raise ex def sample(i, op, j): return ( i + j if op == '+' else (i - j if i > j else j - i) if op == '-' else throw(ValueError("Unrecognized operator "+op)) ) If your new syntax translates into this, it'll mandate that the separate sections all be expressions, but it'll be cleaner in terms of scoping. Also, the use of 'return' inside the switch costs clarity, imo; it happens to be valid in your example because the result of 'switch' is being returned, but that's coincidental. Making it honestly be an expression makes it clearer that it's working with expressions. ChrisA From abarnert at yahoo.com Wed Feb 12 00:47:43 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 11 Feb 2014 15:47:43 -0800 Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: References: Message-ID: <9542A4A5-B325-4D6F-80AE-85395B4128F4@yahoo.com> On Feb 11, 2014, at 15:40, Ryan Gonzalez wrote: > That's an interesting idea. I'm not sure, though, whether that should be considered an inline function or a multi-line lambda. Well, a general purpose multiline lambda would get you Bruce's proposal for free (well, for the cost of writing the word "lambda" or whatever the syntax is), but I think he's specifically proposing something more limited in hopes that it might be more acceptable (maybe more readable, maybe even not requiring as heavy syntax) if it weren't fully general purpose. That being said, I think this has the same problem as all multiline lambda type proposals: it embeds a statement in the middle of an expression, breaks the simple indentation rules, etc. It's not an accident that the provisional syntax is ugly; no one has yet come up with syntax that can solve those fundamental problems, and limiting the scope to a special type of expression doesn't seem to do anything relevant to help. > > > On Tue, Feb 11, 2014 at 5:05 PM, Bruce Leban wrote: >> What if we had the ability to write dictionaries with inline functions in them. >> >> def sample(i, op, j): >> switcher = {{ >> '-':: if i > j: >> return i - j >> else: >> return j - i;; >> '+':: return i + j;; >> }} >> return switcher[op]() >> >> Please don't pay attention to {{ :: ;; }} which would never be the actual syntax or what this code is doing (nothing useful) or whether i or j should be passed as parameters rather than closure. This is equivalent to: >> >> def sample(i, op, j): >> def diff(): >> if i > j: >> return i - j >> else: >> return j - i >> def add(): >> return i + j >> switcher = { >> '-': diff, >> '+': add, >> } >> return switcher[op]() >> >> I don't know if there's a good idea here, but I do know that this pattern of writing dispatchable functions like this is fairly common, just as the switch pattern is. There are some obvious problems, most notably that in the switch statement, the scope of the statements is the same as the enclosing scope while in this code, each function has it's own scope. Notwithstanding that, I thought it was an interesting enough idea to share. >> >> --- Bruce >> Learn how hackers think: http://j.mp/gruyere-security >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ > > > > -- > Ryan > If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rymg19 at gmail.com Wed Feb 12 00:51:36 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Tue, 11 Feb 2014 17:51:36 -0600 Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: References: Message-ID: Uhhh...isn't that the same as a lambda? i.e.: def sample(i, op, j): return {'+': lambda: i+j, '-': lambda: i-j if i>j else j-i }[op] On Tue, Feb 11, 2014 at 5:45 PM, Chris Angelico wrote: > On Wed, Feb 12, 2014 at 10:05 AM, Bruce Leban wrote: > > What if we had the ability to write dictionaries with inline functions in > > them. > > > > def sample(i, op, j): > > switcher = {{ > > '-':: if i > j: > > return i - j > > else: > > return j - i;; > > '+':: return i + j;; > > }} > > return switcher[op]() > > > > > > Please don't pay attention to {{ :: ;; }} which would never be the actual > > syntax or what this code is doing (nothing useful) or whether i or j > should > > be passed as parameters rather than closure. This is equivalent to: > > > > def sample(i, op, j): > > def diff(): > > if i > j: > > return i - j > > else: > > return j - i > > def add(): > > return i + j > > switcher = { > > '-': diff, > > '+': add, > > } > > return switcher[op]() > > > > > > Hmm. In terms of scoping rules and so on, would it be cleaner to > instead make it more equivalent to a big fat expression? Here's legal, > but somewhat ugly, Python code to do the same thing: > > def throw(ex): > """Raise an exception in an expression context.""" > raise ex > > def sample(i, op, j): > return ( > i + j if op == '+' else > (i - j if i > j else j - i) if op == '-' else > throw(ValueError("Unrecognized operator "+op)) > ) > > If your new syntax translates into this, it'll mandate that the > separate sections all be expressions, but it'll be cleaner in terms of > scoping. Also, the use of 'return' inside the switch costs clarity, > imo; it happens to be valid in your example because the result of > 'switch' is being returned, but that's coincidental. Making it > honestly be an expression makes it clearer that it's working with > expressions. > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Wed Feb 12 00:50:02 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 11 Feb 2014 15:50:02 -0800 Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: References: Message-ID: On Feb 11, 2014, at 15:45, Chris Angelico wrote: > On Wed, Feb 12, 2014 at 10:05 AM, Bruce Leban wrote: >> What if we had the ability to write dictionaries with inline functions in >> them. >> >> def sample(i, op, j): >> switcher = {{ >> '-':: if i > j: >> return i - j >> else: >> return j - i;; >> '+':: return i + j;; >> }} >> return switcher[op]() >> >> >> Please don't pay attention to {{ :: ;; }} which would never be the actual >> syntax or what this code is doing (nothing useful) or whether i or j should >> be passed as parameters rather than closure. This is equivalent to: >> >> def sample(i, op, j): >> def diff(): >> if i > j: >> return i - j >> else: >> return j - i >> def add(): >> return i + j >> switcher = { >> '-': diff, >> '+': add, >> } >> return switcher[op]() > > Hmm. In terms of scoping rules and so on, would it be cleaner to > instead make it more equivalent to a big fat expression? > > Hmm. In terms of scoping rules and so on, would it be cleaner to > instead make it more equivalent to a big fat expression? That just means it's a lambda form without the lambda keyword, right? That's certainly cleaner, but I'm not sure what it buys. From bruce at leapyear.org Wed Feb 12 01:00:00 2014 From: bruce at leapyear.org (Bruce Leban) Date: Tue, 11 Feb 2014 16:00:00 -0800 Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: References: Message-ID: On Tue, Feb 11, 2014 at 3:51 PM, Ryan Gonzalez wrote: > Uhhh...isn't that the same as a lambda? i.e.: > I knew I should have thrown something else in there more than 'return' to show I wasn't just doing lambda. See added lines below. def sample(i, op, j): switcher = {{ '-':: if i > j: return i - j else: return j - i;; '+':: return i + j;; '*':: result = 1 for k in range(j): result += i return result;; '@':: print(i, j) raise OopsException();; }} return switcher[op]() I also left out the default case for simplicity. On Tue, Feb 11, 2014 at 3:47 PM, Andrew Barnert wrote: > It's not an accident that the provisional syntax is ugly > You think that's ugly? I can make it uglier. --- Bruce -------------- next part -------------- An HTML attachment was scrubbed... URL: From rymg19 at gmail.com Wed Feb 12 01:02:57 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Tue, 11 Feb 2014 18:02:57 -0600 Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: References: Message-ID: I was talking about Chris' code, not yours. On Tue, Feb 11, 2014 at 6:00 PM, Bruce Leban wrote: > > On Tue, Feb 11, 2014 at 3:51 PM, Ryan Gonzalez wrote: > > Uhhh...isn't that the same as a lambda? i.e.: >> > > > I knew I should have thrown something else in there more than 'return' to > show I wasn't just doing lambda. See added lines below. > > def sample(i, op, j): > switcher = {{ > '-':: if i > j: > return i - j > else: > return j - i;; > '+':: return i + j;; > > '*':: result = 1 > > for k in range(j): > > result += i > > return result;; > > '@':: print(i, j) > raise OopsException();; > > }} > return switcher[op]() > > > I also left out the default case for simplicity. > > On Tue, Feb 11, 2014 at 3:47 PM, Andrew Barnert > wrote: > >> It's not an accident that the provisional syntax is ugly >> > > You think that's ugly? I can make it uglier. > > --- Bruce > -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Wed Feb 12 01:17:03 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 12 Feb 2014 11:17:03 +1100 Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: References: Message-ID: On Wed, Feb 12, 2014 at 10:51 AM, Ryan Gonzalez wrote: > Uhhh...isn't that the same as a lambda? i.e.: > > def sample(i, op, j): > return {'+': lambda: i+j, > '-': lambda: i-j if i>j else j-i > }[op] I was thinking it would be possible to craft a syntax more like the OP's but which, courtesy of MacroPy or something, translated into the version I gave. That way, it's not a nested function, which has scoping and traceback implications. ChrisA From stephen at xemacs.org Wed Feb 12 03:57:25 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 12 Feb 2014 11:57:25 +0900 Subject: [Python-ideas] Make it optional In-Reply-To: References: Message-ID: <87k3d1xckq.fsf@uwakimon.sk.tsukuba.ac.jp> anatoly techtonik writes: > Except that for some of my projects history, documentation, > milestones and tags all use (semantic) versioning, which appears to > be incompatible with this PEP. There's some sort of feature for mapping semantic ("generic") version tags to PEP-compatible version strings. Please find out what it is, *before* you reply, and explain why it doesn't work for you. > No matter how big the PEP 440 is going to get, it unlikely to cover > all use cases that people have for package versions. It's unlikely to get bigger. PEP 440 is not just about coming up with a regular notation for versions; it is also about dependency computations. There's enough junk in there for that. From steve at pearwood.info Wed Feb 12 04:29:19 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 12 Feb 2014 14:29:19 +1100 Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: References: Message-ID: <20140212032919.GP3799@ando> On Tue, Feb 11, 2014 at 03:05:53PM -0800, Bruce Leban wrote: > What if we had the ability to write dictionaries with inline functions in > them. What's an inline function? I've asked in the inline function thread how it differs from a regular function with dynamic scoping, but haven't had a response yet. Since you're now extending the proposal, perhaps you can answer. Is it a function that uses dynamic scoping instead of static scoping? A multi-line anonomous function? Some sort of closure? A thunk? Some of the above? In the previous discussion, it appears to be a regular function using dynamic scoping. Here it appears to be more kind of like a thunk, in that it makes up an expression. https://en.wikipedia.org/wiki/Thunk_%28functional_programming%29 Looking at your example here: > def sample(i, op, j): > switcher = {{ > '-':: if i > j: > return i - j > else: > return j - i;; > '+':: return i + j;; > }} > return switcher[op]() I think it is silly to limit these "inline functions" to only be placed inside dictionaries. What if you want one stand-alone? Or in a list? Inserting them inside a dictionary should be merely a special case of the more general case, a multi-line "inline function" that is syntactically an expression. (Like lambda, only super-powered.) So let's toss out the mutated dict syntax, and use a normal dict, assuming only some special syntax for something to delay computation until called later. I'm going to call it "thunk" rather than inline function, since to me that appears to be the closest match, and use the simplest syntax that might work. Consider this pseudo-code rather than my proposal for actual "thunk" syntax. switcher = { # Receives i, j and op from the scope of the caller. '-': thunk if i > j: return i - j else: return j - i, '+': thunk return i + j, None: thunk raise OperatorError("unknown operator %s" % op), } You then use these thunks something like this: def sample(i, op, j): return switcher.get(op)() It doesn't really buy you much benefit over normal functions with static (lexical) scoping: def sample(i, op, j): return switcher.get(op)(i, op, j) If there is a benefit to dynamic scoping, surely it needs to be more than saving a couple of explicit arguments? I would argue that the benefit of being able to explicitly read the arguments there in the function call far outweighs the cost of having to explicitly write the arguments. Remember that dynamic scoping is rare in programming languages, because it is normally considered a poor idea. It's not hard to implement, but it is hard to use right. Personally, I'm very excited by the idea of dynamic scoping, but in my opinion it's in desperate need of a good use-case. So what have we got here? - The idea of dynamic scoping doesn't seem to gain us much, perhaps a little less typing, but only at the cost of making calls less explicit and hence harder to understand. (In other words, I'm still looking for a good use-case. This isn't it.) - We're no closer to finding good syntax for a multi-line anonymous function which can be used in expressions, regardless of what scoping rules the function uses. My "thunk" syntax above doesn't really work for me, it's just the first and most explicit syntax I thought of. - But if we somehow solve the second problem, regardless of how the functions are scoped, then we can trivially use it inside function calls, dicts, sets, lists, attribute assignments, ... In other words: * scoping and multi-line functions are orthogonal; * dicts are a red herring; * we're no closer to a multi-line lambda than we were 20 years ago. -- Steven From haoyi.sg at gmail.com Wed Feb 12 05:06:45 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Tue, 11 Feb 2014 20:06:45 -0800 Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: <20140212032919.GP3799@ando> References: <20140212032919.GP3799@ando> Message-ID: You can write totally dictionaries with inline functions (i.e. lambdas) in them. Behold, a dictionary with functions, out in the wild: https://github.com/lihaoyi/macropy/blob/master/macropy/core/__init__.py#L125-L237 Maybe not pretty, but definitely prettier than a forest of *elif* statements On Tue, Feb 11, 2014 at 7:29 PM, Steven D'Aprano wrote: > On Tue, Feb 11, 2014 at 03:05:53PM -0800, Bruce Leban wrote: > > What if we had the ability to write dictionaries with inline functions in > > them. > > What's an inline function? I've asked in the inline function thread how > it differs from a regular function with dynamic scoping, but haven't had > a response yet. Since you're now extending the proposal, perhaps you can > answer. > > Is it a function that uses dynamic scoping instead of static scoping? A > multi-line anonomous function? Some sort of closure? A thunk? Some of > the above? In the previous discussion, it appears to be a regular > function using dynamic scoping. Here it appears to be more kind of like > a thunk, in that it makes up an expression. > > https://en.wikipedia.org/wiki/Thunk_%28functional_programming%29 > > > Looking at your example here: > > > def sample(i, op, j): > > switcher = {{ > > '-':: if i > j: > > return i - j > > else: > > return j - i;; > > '+':: return i + j;; > > }} > > return switcher[op]() > > > I think it is silly to limit these "inline functions" to only be placed > inside dictionaries. What if you want one stand-alone? Or in a list? > Inserting them inside a dictionary should be merely a special case of > the more general case, a multi-line "inline function" that is > syntactically an expression. (Like lambda, only super-powered.) > > So let's toss out the mutated dict syntax, and use a normal dict, > assuming only some special syntax for something to delay computation > until called later. I'm going to call it "thunk" rather than inline > function, since to me that appears to be the closest match, and use the > simplest syntax that might work. Consider this pseudo-code rather than > my proposal for actual "thunk" syntax. > > > switcher = { > # Receives i, j and op from the scope of the caller. > '-': thunk if i > j: > return i - j > else: > return j - i, > '+': thunk return i + j, > None: thunk raise OperatorError("unknown operator %s" % op), > } > > > > You then use these thunks something like this: > > def sample(i, op, j): > return switcher.get(op)() > > > It doesn't really buy you much benefit over normal functions with static > (lexical) scoping: > > def sample(i, op, j): > return switcher.get(op)(i, op, j) > > > If there is a benefit to dynamic scoping, surely it needs to be more > than saving a couple of explicit arguments? I would argue that the > benefit of being able to explicitly read the arguments there in the > function call far outweighs the cost of having to explicitly write the > arguments. > > Remember that dynamic scoping is rare in programming languages, because > it is normally considered a poor idea. It's not hard to implement, but > it is hard to use right. Personally, I'm very excited by the idea of > dynamic scoping, but in my opinion it's in desperate need of a good > use-case. > > So what have we got here? > > - The idea of dynamic scoping doesn't seem to gain us much, perhaps a > little less typing, but only at the cost of making calls less explicit > and hence harder to understand. (In other words, I'm still looking for > a good use-case. This isn't it.) > > - We're no closer to finding good syntax for a multi-line anonymous > function which can be used in expressions, regardless of what scoping > rules the function uses. My "thunk" syntax above doesn't really work > for me, it's just the first and most explicit syntax I thought of. > > - But if we somehow solve the second problem, regardless of how the > functions are scoped, then we can trivially use it inside function > calls, dicts, sets, lists, attribute assignments, ... > > In other words: > > * scoping and multi-line functions are orthogonal; > > * dicts are a red herring; > > * we're no closer to a multi-line lambda than we were 20 years ago. > > > > -- > Steven > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From skip at pobox.com Wed Feb 12 06:35:31 2014 From: skip at pobox.com (Skip Montanaro) Date: Tue, 11 Feb 2014 23:35:31 -0600 Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: References: <20140212032919.GP3799@ando> Message-ID: On Tue, Feb 11, 2014 at 10:06 PM, Haoyi Li wrote: > You can write totally dictionaries with inline functions (i.e. lambdas) in them. Lambdas are not what I think of when I think of inline functions. Substituting them for named functions gains you nothing. They are just (very limited) functions you call in the usual fashion, and which have all the usual overhead you associate with calling Python functions. I think of inline functions as they exist in C++ (or before that, in GCC). If you declare a function to be inline, the compiler is free to inline the body of the function's code at the call point and make the necessary fix-ups to the prolog and epilog to preserve semantics (as if it had not been declared inline), but eliminate call overhead, and much of the stack manipulation. This works in statically typed languages like C and C++, because at compile time the compiler knows everything about the types of the functions arguments and return values, as well as the environment in which the function and the call point exist. I don't think that kind of function inlining would be possible in CPython. At minimum, the compiler simply doesn't know the types of the arguments to the function call at compile time. Heck, it probably wouldn't even know (except in the most trivial of circumstances) that any particular function available is the one to inline. You'd have to look to PyPy for the necessary tools to perform this feat, and it would be accomplished at run-time. Skip From abarnert at yahoo.com Wed Feb 12 08:36:15 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 11 Feb 2014 23:36:15 -0800 (PST) Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: References: <20140212032919.GP3799@ando> Message-ID: <1392190575.76354.YahooMailNeo@web181002.mail.ne1.yahoo.com> From: Skip Montanaro Sent: Tuesday, February 11, 2014 9:35 PM > On Tue, Feb 11, 2014 at 10:06 PM, Haoyi Li wrote: >> You can write totally dictionaries with inline functions (i.e. lambdas) in > them. > > Lambdas are not what I think of when I think of inline functions. > Substituting them for named functions gains you nothing. They are just > (very limited) functions you call in the usual fashion, and which have > all the usual overhead you associate with calling Python functions. > > I think of inline functions as they exist in C++ (or before that, in > GCC). Then I think you're thinking of them wrong. > If you declare a function to be inline, the compiler is free to > inline the body of the function's code at the call point and make the > necessary fix-ups to the prolog and epilog to preserve semantics (as > if it had not been declared inline), but eliminate call overhead, and > much of the stack manipulation. The compiler is _already_ free to do this for any function call where the definition is available in the current translation unit, even if you didn't declare it inline. And, conversely, the compiler is free to insert a regular function call even if you did declare it inline. Effectively, inline is just a hint. Or, as the standard puts it (7.1.2): >?The inline specifier indicates to the implementation that inline substitution of the? > function body at the point of call is to be preferred to the usual function call? > mechanism. An implementation is not required to perform this inline substitution? > at the point of call; however, even if this inline substitution is omitted, the? > other rules for inline functions defined by 7.1.2 shall still be respected." There are a few minor differences with respect to the ODR and so forth, but basically, a C++ inline function is just a plain old function with a compiler hint attached to it. Meanwhile, unlike the proposals in the earlier threads, a C++ inline function?even if they really worked the way you think they do?wouldn't be relevant to what people are talking about in this thread. It?doesn't get dynamic scoping or any other special scoping,?can't force a return from the calling function,?can't be declared anywhere a normal top-level function couldn't, still have external linkage, etc. > This works in statically typed languages like C and C++, because at > compile time the compiler knows everything about the types of the > functions arguments and return values, as well as the environment in > which the function and the call point exist. I don't think that kind > of function inlining would be possible in CPython. At minimum, the > compiler simply doesn't know the types of the arguments to the > function call at compile time. How is relevant? If anything, dynamic typing makes inlining _easier_. In C++, you need to know the argument types in order to pass them on the stack or in registers?they have different sizes, some even have to go in different registers (e.g., floats vs. ints on most platforms). In Python, every argument is pushed on the stack in the same way. In CPython, under the covers, they're all?just PyObject pointers. > Heck, it probably wouldn't even know > (except in the most trivial of circumstances) that any particular > function available is the one to inline.? That, on the other hand, is a real issue. Callables are usually attributes on a module or object, looked up by name at runtime, which means there is nothing to inline until runtime. The only case where you could meaningfully inline a function call would be if the function were defined locally within the calling function. Meanwhile, your focus on optimizing out the cost of a function call is not nearly as relevant to Python as it is to C++. Python code is rarely micro-optimized in the way that C++ code is. C++ is, in fact, designed specifically for this kind of micro-optimization. In Python, the cost of looking up the function's name is already orders of magnitude slower than the stack operations that C++ programmers are trying to optimize out. And even in (modern) C++, this isn't nearly as important an optimization as many programmers seem to think?when it really does matter, automated optimization (especially PGO) almost always does a better job than the programmer's hints anyway. This is why most C++ compilers had to add non-standard "force-inline" directives for the rare cases where the programmer really does know better than the optimizer?because generally, he doesn't. Finally, just like C++, Python could do this optimization without any need for a directive. And, besides the fact that Python is not designed for manual micro-optimization, the more limited scope in which it could work makes the directive less useful anyway. In C++, methods defined inside class definitions are automatically considered to be inline function definitions. Originally, this was a clever hack to get around the ODR while fitting Modula-style class definitions into the traditional C .h/.c model, but it was kept around in C++98 because, idiomatically, methods defined inside a class are generally good candidates for inlining. In the same way, in Python, functions defined inside the function they're called in would generally be good candidates for inlining. And, since those are the only functions that could legally be inlined, why make the programmer add anything else? From davidfstr at gmail.com Wed Feb 12 10:24:27 2014 From: davidfstr at gmail.com (David Foster) Date: Wed, 12 Feb 2014 01:24:27 -0800 Subject: [Python-ideas] async: switch_to_loop and loop_affinity Message-ID: <52FB3DCB.7020904@gmail.com> There has been much talk of the use of async libraries (such as Tulip) within the general programming community to avoid the "christmas tree" pattern of code indentation in code that uses a lot of callbacks. For example, consider the following callback-based code: def duplicate_file_async(filepath, done): def exists_async(does_exist): if not does_exist: done(False) return def entire_file_read(contents): if not contents: done(False) return def file_written(success): done(success) write_entire_file_async(filepath + ' copy', contents, file_written) read_entire_file_async(filepath, entire_file_read) exists_async(filepath) This code be rewritten in Tulip as: @async.coroutine def duplicate_file_async(filepath): if not yield from exists_async(filepath): return False contents = yield from read_entire_file_async(filepath) if not contents: return False return yield from write_entire_file_async(filepath + ' copy', contents) Much more clear, no? Tulip, however, does not appear to address a slightly different problem I run into where I need different parts of the same conceptual function to run on specific *different* threads. Consider the following code which needs different parts executed on a UI thread, database thread, and background thread: def tree_node_clicked(tree_node): print('Clicked: ' + repr(tree_node)) def db_task(): db_node = fetch_node_from_db(tree_node) def process_node(db_node): tree_node.set_contents(db_node.contents) # (done) if db_node is not None: def bg_task(): db_node = fetch_node_from_internet(tree_node.url) def db_task(): insert_into_db(db_node) ui_call_soon(process_node, db_node) db_call_soon(db_task) bg_call_soon(bg_task) else: ui_call_soon(process_node, db_node) db_call_soon(db_task) Imagine if you could write this function like the following: ui_loop = ... db_loop = ... bg_loop = ... @async.coroutine def tree_node_clicked(tree_node): print('Clicked: ' + repr(tree_node)) yield from switch_to_loop(db_loop) db_node = fetch_node_from_db(tree_node) if db_node is not None: yield from switch_to_loop(bg_loop) db_node = fetch_node_from_internet(tree_node.url) yield from switch_to_loop(db_loop) insert_into_db(db_node) yield from switch_to_loop(ui_loop) tree_node.set_contents(db_node.contents) Or even better: If you created a decorators like @loop_affinity(*_loop) that automatically called switch_to_loop(...) if the current event loop wasn't correct, you could even write the following even-more simplified version: @async.coroutine def tree_node_clicked(tree_node): print('Clicked: ' + repr(tree_node)) db_node = yield from fetch_node_from_db(tree_node) if db_node is not None: db_node = yield from fetch_node_from_internet(tree_node.url) yield from insert_into_db(db_node) yield from tree_node.set_contents(db_node.contents) @async.loop_affinity(db_loop) def fetch_node_from_db(...): ... @async.loop_affinity(bg_loop) def fetch_node_from_internet(...): ... @async.loop_affinity(db_loop) def insert_into_db(...): ... @async.loop_affinity(ui_loop) def set_contents(...): ... Wouldn't that be just grand? I have prototyped switch_to_loop(...) successfully in Tulip, albeit with a race condition I was unable to isolate. How about some equivalent to switch_to_loop(...) and loop_affinity(...) in a future version of Tulip? -- David Foster http://dafoster.net/ From jonathan at slenders.be Wed Feb 12 10:33:32 2014 From: jonathan at slenders.be (Jonathan Slenders) Date: Wed, 12 Feb 2014 10:33:32 +0100 Subject: [Python-ideas] async: switch_to_loop and loop_affinity In-Reply-To: <52FB3DCB.7020904@gmail.com> References: <52FB3DCB.7020904@gmail.com> Message-ID: Do you know about run_in_executor? http://docs.python.org/3.4/library/asyncio-eventloop.html#asyncio.BaseEventLoop.run_in_executor 2014-02-12 10:24 GMT+01:00 David Foster : > There has been much talk of the use of async libraries (such as Tulip) > within the general programming community to avoid the "christmas tree" > pattern of code indentation in code that uses a lot of callbacks. For > example, consider the following callback-based code: > > def duplicate_file_async(filepath, done): > def exists_async(does_exist): > if not does_exist: > done(False) > return > def entire_file_read(contents): > if not contents: > done(False) > return > def file_written(success): > done(success) > write_entire_file_async(filepath + ' copy', contents, > file_written) > read_entire_file_async(filepath, entire_file_read) > exists_async(filepath) > > This code be rewritten in Tulip as: > > @async.coroutine > def duplicate_file_async(filepath): > if not yield from exists_async(filepath): > return False > contents = yield from read_entire_file_async(filepath) > if not contents: > return False > return yield from write_entire_file_async(filepath + ' copy', > contents) > > Much more clear, no? > > Tulip, however, does not appear to address a slightly different problem I > run into where I need different parts of the same conceptual function to > run on specific *different* threads. Consider the following code which > needs different parts executed on a UI thread, database thread, and > background thread: > > def tree_node_clicked(tree_node): > print('Clicked: ' + repr(tree_node)) > def db_task(): > db_node = fetch_node_from_db(tree_node) > > def process_node(db_node): > tree_node.set_contents(db_node.contents) > # (done) > > if db_node is not None: > def bg_task(): > db_node = fetch_node_from_internet(tree_node.url) > def db_task(): > insert_into_db(db_node) > ui_call_soon(process_node, db_node) > db_call_soon(db_task) > bg_call_soon(bg_task) > else: > ui_call_soon(process_node, db_node) > db_call_soon(db_task) > > Imagine if you could write this function like the following: > > ui_loop = ... > db_loop = ... > bg_loop = ... > > @async.coroutine > def tree_node_clicked(tree_node): > print('Clicked: ' + repr(tree_node)) > > yield from switch_to_loop(db_loop) > db_node = fetch_node_from_db(tree_node) > if db_node is not None: > yield from switch_to_loop(bg_loop) > db_node = fetch_node_from_internet(tree_node.url) > > yield from switch_to_loop(db_loop) > insert_into_db(db_node) > > yield from switch_to_loop(ui_loop) > tree_node.set_contents(db_node.contents) > > Or even better: If you created a decorators like @loop_affinity(*_loop) > that automatically called switch_to_loop(...) if the current event loop > wasn't correct, you could even write the following even-more simplified > version: > > @async.coroutine > def tree_node_clicked(tree_node): > print('Clicked: ' + repr(tree_node)) > db_node = yield from fetch_node_from_db(tree_node) > if db_node is not None: > db_node = yield from fetch_node_from_internet(tree_node.url) > yield from insert_into_db(db_node) > yield from tree_node.set_contents(db_node.contents) > > @async.loop_affinity(db_loop) > def fetch_node_from_db(...): ... > > @async.loop_affinity(bg_loop) > def fetch_node_from_internet(...): ... > > @async.loop_affinity(db_loop) > def insert_into_db(...): ... > > @async.loop_affinity(ui_loop) > def set_contents(...): ... > > Wouldn't that be just grand? I have prototyped switch_to_loop(...) > successfully in Tulip, albeit with a race condition I was unable to isolate. > > How about some equivalent to switch_to_loop(...) and loop_affinity(...) in > a future version of Tulip? > > -- > David Foster > http://dafoster.net/ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Wed Feb 12 10:52:44 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 12 Feb 2014 04:52:44 -0500 Subject: [Python-ideas] async: switch_to_loop and loop_affinity In-Reply-To: <52FB3DCB.7020904@gmail.com> References: <52FB3DCB.7020904@gmail.com> Message-ID: On 2/12/2014 4:24 AM, David Foster wrote: > Tulip, however, does not appear to address a slightly different problem > I run into where I need different parts of the same conceptual function > to run on specific *different* threads. Consider the following code > which needs different parts executed on a UI thread, database thread, > and background thread: ... There is a tulip google group where tulip discussion takes place. I don't have the specific reference though. -- Terry Jan Reedy From ncoghlan at gmail.com Wed Feb 12 11:05:42 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 12 Feb 2014 20:05:42 +1000 Subject: [Python-ideas] async: switch_to_loop and loop_affinity In-Reply-To: References: <52FB3DCB.7020904@gmail.com> Message-ID: On 12 Feb 2014 19:54, "Terry Reedy" wrote: > > On 2/12/2014 4:24 AM, David Foster wrote: > >> Tulip, however, does not appear to address a slightly different problem >> I run into where I need different parts of the same conceptual function >> to run on specific *different* threads. Consider the following code >> which needs different parts executed on a UI thread, database thread, >> and background thread: > > > ... > There is a tulip google group where tulip discussion takes place. I don't have the specific reference though. With 3.4 nearing release, I suspect we're going to have to get used to the idea of gently redirecting pip and asyncio/tulip discussions :) An update to the "communications" section of the dev guide may be in order, too. Cheers, Nick. > > -- > Terry Jan Reedy > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Wed Feb 12 11:07:36 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 12 Feb 2014 02:07:36 -0800 (PST) Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: <20140212032919.GP3799@ando> References: <20140212032919.GP3799@ando> Message-ID: <1392199656.52500.YahooMailNeo@web181004.mail.ne1.yahoo.com> From: Steven D'Aprano Sent: Tuesday, February 11, 2014 7:29 PM > On Tue, Feb 11, 2014 at 03:05:53PM -0800, Bruce Leban wrote: >> What if we had the ability to write dictionaries with inline functions in >> them. > > What's an inline function? I've asked in the inline function thread how > it differs from a regular function with dynamic scoping, but haven't had > a response yet. Since you're now extending the proposal, perhaps you can > answer. > > Is it a function that uses dynamic scoping instead of static scoping? A > multi-line anonomous function? Some sort of closure? A thunk? I think neither Alex Rodrigues's initial idea nor Bruce Leban's side idea area any of these. First, Bruce Leban isn't really asking for multi-line anonymous functions, but for multi-line functions that are expressions. These are entirely independent things. (Look at JavaScript: "function [name] { }" is an expression.) Notice that Bruce wrote a second post to specifically clarify that he wants to do non-expression things like assigning to variables. Being able to use statements in an expression is where all the power comes from, and all of the problems, when people talk about "multi-line lambdas", not being able to leave the name off.?Bruce was just proposing YA syntax for the usual extended lambdas?but getting some mileage out of restricting them to appearing to a new special kind of display expression. Meanwhile, Alex was proposing something very different. Really, what he wants is runtime computational (i.e., bytecode) macros.?Look at his?motivating example: ? ? @inline ? ? def element_wise(func): ? ? ? ? func = inline(func)? ? ? ? ? for ?: ? ? ? ? ? ? func() ? ? def is_symmetric(matrix): ? ? ? ? element_wise(lambda: if matrix[i][j] != matrix[j][i]: return False) ? ? ? ? return True Forget about how the variables work for the moment (and the fact that, on top of his main desire, he _also_ wants a statement inside his lambda?) and look at that "return False". Clearly it has to return from not just the lambda, but element_wise, and is_symmetric. That's the key here; that's what makes it "inline" in Alex's terms. It's not a function on the stack with its own frame, it's just code that gets compiled as a function, but then executed inline directly in the calling function's frame. This doesn't quite work for a few relatively minor reasons (functions get an implicit "return None", variables are referred to by index and may not have the right indices, etc.), but the idea isn't nonsense. Once you see that, dynamic vs. lexical scope is a red herring. Either way, the stack frame in which matrix gets evaluated is the stack frame of is_symmetric. The existing closure variable lookup does the right thing. (Note that the definition of the lambda forces matrix to be looked up via *_DEREF in is_symmetric, so the *_DEREF code in the lambda is correct when inlined.)?And it would _have_ to be right for the return-through to make any sense. Dynamic scope doesn't seem to add any possibilities here that it wouldn't already add in regular functions; I think it's orthogonal, and irrelevant, to the proposal. But I could be wrong. Anyway, I don't think he really needs runtime macros; he only needs "func = inline(func)" because?element_wise itself is defined as a function and converted to a macro with "@inline". If the latter were compile-time, the former could be as well. (Imagine a "defmacro" statement and a "lambdamacro" expression.) And that eliminates a whole lot of complexity (dynamically generating dynamic code-fixup code, etc.).?And if you don't need runtime macros, you probably don't need computational macros, which eliminates even more (e.g., in the AST, "matrix" is a name, not an index). The key thing he wants is macros. But what he asked for is runtime computational macros. From denis.spir at gmail.com Wed Feb 12 11:12:37 2014 From: denis.spir at gmail.com (spir) Date: Wed, 12 Feb 2014 11:12:37 +0100 Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: References: <20140212032919.GP3799@ando> Message-ID: <52FB4915.1040705@gmail.com> On 02/12/2014 06:35 AM, Skip Montanaro wrote: > On Tue, Feb 11, 2014 at 10:06 PM, Haoyi Li wrote: >> You can write totally dictionaries with inline functions (i.e. lambdas) in them. > > Lambdas are not what I think of when I think of inline functions. > Substituting them for named functions gains you nothing. They are just > (very limited) functions you call in the usual fashion, and which have > all the usual overhead you associate with calling Python functions. > > I think of inline functions as they exist in C++ (or before that, in > GCC). If you declare a function to be inline, the compiler is free to > inline the body of the function's code at the call point and make the > necessary fix-ups to the prolog and epilog to preserve semantics (as > if it had not been declared inline), but eliminate call overhead, and > much of the stack manipulation. > > This works in statically typed languages like C and C++, because at > compile time the compiler knows everything about the types of the > functions arguments and return values, as well as the environment in > which the function and the call point exist. I don't think that kind > of function inlining would be possible in CPython. At minimum, the > compiler simply doesn't know the types of the arguments to the > function call at compile time. Heck, it probably wouldn't even know > (except in the most trivial of circumstances) that any particular > function available is the one to inline. You'd have to look to PyPy > for the necessary tools to perform this feat, and it would be > accomplished at run-time. What the compiler need, more than the type, is the "weight" (memory size); so as to be able to (statically) reserve slots in the stack (or register, wherever). [In general, static efficiency comes from knowing the weight of values, also at times their structures (say globally their "formats"), rather than their types.] At first sight, it seems to me that python's optional and "default" args may also be a big obstacle to inlining. (I wonder how they're done for regular func def and cal, by the way.) d From denis.spir at gmail.com Wed Feb 12 11:27:20 2014 From: denis.spir at gmail.com (spir) Date: Wed, 12 Feb 2014 11:27:20 +0100 Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: <1392199656.52500.YahooMailNeo@web181004.mail.ne1.yahoo.com> References: <20140212032919.GP3799@ando> <1392199656.52500.YahooMailNeo@web181004.mail.ne1.yahoo.com> Message-ID: <52FB4C88.6010201@gmail.com> On 02/12/2014 11:07 AM, Andrew Barnert wrote: > From: Steven D'Aprano > > > I think neither Alex Rodrigues's initial idea nor Bruce Leban's side idea area any of these. > > [...] > > Meanwhile, Alex was proposing something very different. Really, what he wants is runtime computational (i.e., bytecode) macros. Look at his motivating example: > > @inline > def element_wise(func): > func = inline(func) > for ?: > func() > > def is_symmetric(matrix): > element_wise(lambda: if matrix[i][j] != matrix[j][i]: return False) > return True > > > Forget about how the variables work for the moment (and the fact that, on top of his main desire, he _also_ wants a statement inside his lambda?) and look at that "return False". Clearly it has to return from not just the lambda, but element_wise, and is_symmetric. That's the key here; that's what makes it "inline" in Alex's terms. It's not a function on the stack with its own frame, it's just code that gets compiled as a function, but then executed inline directly in the calling function's frame. This doesn't quite work for a few relatively minor reasons (functions get an implicit "return None", variables are referred to by index and may not have the right indices, etc.), but the idea isn't nonsense. > > Once you see that, dynamic vs. lexical scope is a red herring. Either way, the stack frame in which matrix gets evaluated is the stack frame of is_symmetric. The existing closure variable lookup does the right thing. (Note that the definition of the lambda forces matrix to be looked up via *_DEREF in is_symmetric, so the *_DEREF code in the lambda is correct when inlined.) And it would _have_ to be right for the return-through to make any sense. Dynamic scope doesn't seem to add any possibilities here that it wouldn't already add in regular functions; I think it's orthogonal, and irrelevant, to the proposal. But I could be wrong. > > Anyway, I don't think he really needs runtime macros; he only needs "func = inline(func)" because element_wise itself is defined as a function and converted to a macro with "@inline". If the latter were compile-time, the former could be as well. (Imagine a "defmacro" statement and a "lambdamacro" expression.) And that eliminates a whole lot of complexity (dynamically generating dynamic code-fixup code, etc.). And if you don't need runtime macros, you probably don't need computational macros, which eliminates even more (e.g., in the AST, "matrix" is a name, not an index). The key thing he wants is macros. But what he asked for is runtime computational macros. How does this relate to Ruby blocks, procs, and lambdas? One of them is to solves the issue of which-is-the-actual-piece-of-code-to-return-from, IIRC (dunno much of Ruby), like your motivating example of "inline" funcs here. d From ram.rachum at gmail.com Wed Feb 12 22:02:57 2014 From: ram.rachum at gmail.com (Ram Rachum) Date: Wed, 12 Feb 2014 13:02:57 -0800 (PST) Subject: [Python-ideas] except expression Message-ID: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> Hi, Here's an idea that would help shortening code. Allow a ternary expression based on except, like so: first_entry = entries[0] except IndexError else None item = my_queue.get() except queue.Empty else None response_text = request('http://whatever.com').text except HttpError else "Can't access data" Aside from the fact that this would be a big grammar addition, a big problem here is the usage of the `else` keyword, that when used with except usually means "what would happen if there wasn't an exception" and here means the opposite. But I couldn't think of a nicer syntax. I realize that this is a big change and that most people would be opposed to this... But I guess I just wanted to share my idea :) Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Wed Feb 12 23:00:20 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 12 Feb 2014 22:00:20 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> Message-ID: On 12 February 2014 21:02, Ram Rachum wrote: > I realize that this is a big change and that most people would be opposed to > this... But I guess I just wanted to share my idea :) This was discussed a while ago on the list. I can't find the thread right now, but you may want to search the archives. Paul PS Looks like posting via google groups not only breaks filtering and threading, it also breaks reply-all (as I can't post to the google group). Could you please post direct to the group rather than through the google groups interface? Thanks. From ram.rachum at gmail.com Wed Feb 12 22:12:17 2014 From: ram.rachum at gmail.com (Ram Rachum) Date: Wed, 12 Feb 2014 13:12:17 -0800 (PST) Subject: [Python-ideas] a in x or in y Message-ID: Hi, What do you think about adding this to Python: 'whatever a long string' in x or in y I've often wished for this because the current way is quite verbose: 'whatever a long string' in x or 'whatever a long string' in y We might add "and in" wwai\ dafsdfasdfasdfa while we're at it. Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Thu Feb 13 00:50:59 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 12 Feb 2014 15:50:59 -0800 (PST) Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: <52FB4915.1040705@gmail.com> References: <20140212032919.GP3799@ando> <52FB4915.1040705@gmail.com> Message-ID: <1392249059.83235.YahooMailNeo@web181005.mail.ne1.yahoo.com> From: spir Sent: Wednesday, February 12, 2014 2:12 AM > What the compiler need, more than the type, is the "weight" (memory > size); so as to be able to (statically) reserve slots in the stack (or register, > wherever). [In general, static efficiency comes from knowing the weight of > values, also at times their structures (say globally their "formats"), > rather than their types.] > > At first sight, it seems to me that python's optional and > "default" args may also be a big obstacle to inlining. (I wonder how > they're done for regular func def and cal, by the way.) You're still thinking in C terms. First, in Python, the stack is a stack of objects, not bytes. They all have the same weight and structure (in CPython, each one is a PyObject pointer).? Meanwhile, a function is an object with attributes, including a tuple of default values, a code object with its own attributes like a tuple of argument names, and so on. The interpreter (in particular, the CALL_FUNCTION bytecode handler) is able to match up arguments and parameters at runtime.? Also, a frame is an object with attributes, including a reference to the previous frame, not just a chunk of bytes on the stack. So, there is no "call prologue" compiled into each function. And, more generally,?the problems you're expecting just don't exist. The caller just pushes the function and the arguments and does a CALL_FUNCTION, then gets the return value on the stack. The callee just starts off with its frame's locals already set up, and runs exactly the code you wrote and noting more.?How it works is all documented, although scattered in a few different places (the bytecode specs in the dis module docs, the attributes of function and code objects in the inspect module docs, the calls and definitions sections of the reference, and the callable C API). I tried to put a summary together at?http://stupidpythonideas.blogspot.com/2014/02/arguments-and-parameters-under-covers.html if it helps. So, default parameter values are not an obstacle to inlining at all. They're handled at runtime by CALL_FUNCTION just like arguments are. The obstacle to inlining is that you're ultimately trying to inline a runtime object at compile time, which doesn't make any sense. Elsewhere in the thread I explained some ways you could do something similar but sensible, but it's not going to be like C or C++ inlining. From carl at oddbird.net Thu Feb 13 00:55:09 2014 From: carl at oddbird.net (Carl Meyer) Date: Wed, 12 Feb 2014 16:55:09 -0700 Subject: [Python-ideas] a in x or in y In-Reply-To: References: Message-ID: <52FC09DD.3080100@oddbird.net> On 02/12/2014 02:12 PM, Ram Rachum wrote: > What do you think about adding this to Python: > > 'whatever a long string' in x or in y > > I've often wished for this because the current way is quite verbose: > > 'whatever a long string' in x or 'whatever a long string' in y Except you'd never actually do that, you'd just put the long string in a variable. Or the other option: any('whatever a long string' in i for i in [x, y]) (And 'all' similarly covers the 'and' case.) Carl From python at mrabarnett.plus.com Thu Feb 13 00:56:02 2014 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 12 Feb 2014 23:56:02 +0000 Subject: [Python-ideas] except expression In-Reply-To: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> Message-ID: <52FC0A12.7000802@mrabarnett.plus.com> On 2014-02-12 21:02, Ram Rachum wrote: > Hi, > > Here's an idea that would help shortening code. Allow a ternary > expression based on except, like so: > > first_entry = entries[0] except IndexError else None > item = my_queue.get() except queue.Empty else None > response_text = request('http://whatever.com').text except HttpError else "Can't access data" > > Aside from the fact that this would be a big grammar addition, a big > problem here is the usage of the `else` keyword, that when used with > except usually means "what would happen if there wasn't an exception" > and here means the opposite. But I couldn't think of a nicer syntax. > If you don't mind having a colon in the middle of an expression: first_entry = entries[0] except IndexError: None item = my_queue.get() except queue.Empty: None response_text = request('http://whatever.com').text except HttpError: "Can't access data" What would its precedence be? Maybe it would apply to the preceding expression or subexpression: total = (entries[0] except IndexError: 0) + (entries[-1] except IndexError: 0) > I realize that this is a big change and that most people would be > opposed to this... But I guess I just wanted to share my idea :) > From raymond.hettinger at gmail.com Thu Feb 13 01:08:01 2014 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Thu, 13 Feb 2014 00:08:01 +0000 Subject: [Python-ideas] except expression In-Reply-To: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> Message-ID: On Feb 12, 2014, at 9:02 PM, Ram Rachum wrote: > Here's an idea that would help shortening code. Allow a ternary expression based on except, like so: > > first_entry = entries[0] except IndexError else None > item = my_queue.get() except queue.Empty else None > response_text = request('http://whatever.com').text except HttpError else "Can't access data" > > Aside from the fact that this would be a big grammar addition, a big problem here is the usage of the `else` keyword, that when used with except usually means "what would happen if there wasn't an exception" and here means the opposite. But I couldn't think of a nicer syntax. > > I realize that this is a big change and that most people would be opposed to this... But I guess I just wanted to share my idea :) I would like to see something like this come to fruition. We need a clean way to express the idea of "take an arbitrary, exception-raising function and give it a default argument". Hopefully, this would end the gradual but never-ending requests to bloat APIs with "default" arguments. For example, if your idea or some variant had been in place, the min() and max() functions likely wouldn't have grown complex signatures in Python 3.4. Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Thu Feb 13 01:08:49 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 13 Feb 2014 11:08:49 +1100 Subject: [Python-ideas] a in x or in y References: Message-ID: <85sirnan72.fsf@benfinney.id.au> Ram Rachum writes: > I've often wished for this because the current way is quite verbose: > > 'whatever a long string' in x or 'whatever a long string' in y No need to repeat the literal: >>> x = 'foo' >>> y = 'bar whatever a long string bar' >>> z = 'baz' >>> any('whatever a long string' in item for item in [x, y, z]) True -- \ ?Politics is not the art of the possible. It consists in | `\ choosing between the disastrous and the unpalatable.? ?John | _o__) Kenneth Galbraith, 1962-03-02 | Ben Finney From ben+python at benfinney.id.au Thu Feb 13 01:14:12 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 13 Feb 2014 11:14:12 +1100 Subject: [Python-ideas] except expression References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> Message-ID: <85ob2bamy3.fsf@benfinney.id.au> Ram Rachum writes: > Here's an idea that would help shortening code. Allow a ternary > expression based on except, like so: > > first_entry = entries[0] except IndexError else None > item = my_queue.get() except queue.Empty else None > response_text = request('http://whatever.com').text except HttpError else "Can't access data" That is more obscure, to my eye, than laying out the control branches: first_entry = entries[0] except IndexError else None item = my_queue.get() except queue.Empty else None try: response_text = request('http://whatever.com').text except HttpError: "Can't access data" Do you have some real-use code that your proposal would significantly improve? I find your example to support the refusal of that syntax. -- \ ?The whole area of [treating source code as intellectual | `\ property] is almost assuring a customer that you are not going | _o__) to do any innovation in the future.? ?Gary Barnett | Ben Finney From ram.rachum at gmail.com Thu Feb 13 01:14:45 2014 From: ram.rachum at gmail.com (Ram Rachum) Date: Thu, 13 Feb 2014 02:14:45 +0200 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> Message-ID: I'm happy you're for it. Maybe one day we'll see a Python 4 with no second argument to dict.get, getattr and so many others... On Thu, Feb 13, 2014 at 2:08 AM, Raymond Hettinger < raymond.hettinger at gmail.com> wrote: > > On Feb 12, 2014, at 9:02 PM, Ram Rachum wrote: > > Here's an idea that would help shortening code. Allow a ternary expression > based on except, like so: > > first_entry = entries[0] except IndexError else None > item = my_queue.get() except queue.Empty else None > response_text = request('http://whatever.com').text except HttpError > else "Can't access data" > > Aside from the fact that this would be a big grammar addition, a big > problem here is the usage of the `else` keyword, that when used with except > usually means "what would happen if there wasn't an exception" and here > means the opposite. But I couldn't think of a nicer syntax. > > I realize that this is a big change and that most people would be opposed > to this... But I guess I just wanted to share my idea :) > > > I would like to see something like this come to fruition. > We need a clean way to express the idea of "take an > arbitrary, exception-raising function and give it a default > argument". > > Hopefully, this would end the gradual but never-ending requests > to bloat APIs with "default" arguments. For example, if your idea > or some variant had been in place, the min() and max() functions > likely wouldn't have grown complex signatures in Python 3.4. > > > Raymond > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Thu Feb 13 01:28:42 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 13 Feb 2014 11:28:42 +1100 Subject: [Python-ideas] except expression References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> Message-ID: <85ha83am9x.fsf@benfinney.id.au> Ben Finney writes: > Ram Rachum writes: > > > Here's an idea that would help shortening code. Allow a ternary > > expression based on except, like so: > > > > first_entry = entries[0] except IndexError else None > > item = my_queue.get() except queue.Empty else None > > response_text = request('http://whatever.com').text except HttpError else "Can't access data" > > That is more obscure, to my eye, than laying out the control branches: Sorry, I failed to address your first two examples. I am +0 on the proposal to have something similar to Perl's fallback syntax, ?$foo = bar() or some_default_value?. Yet I still find the proposed syntax less readable for anything but a trivial *and* brief case. For anything longer than a few dozen characters, I still prefer:: try: response_text = request('http://whatever.com').text except HttpError: "Can't access data" for being explicit and clarifying what to expect. -- \ ?I wish there was a knob on the TV to turn up the intelligence. | `\ There's a knob called ?brightness? but it doesn't work.? | _o__) ?Eugene P. Gallagher | Ben Finney From abarnert at yahoo.com Thu Feb 13 02:27:50 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 12 Feb 2014 17:27:50 -0800 (PST) Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: <52FB4C88.6010201@gmail.com> References: <20140212032919.GP3799@ando> <1392199656.52500.YahooMailNeo@web181004.mail.ne1.yahoo.com> <52FB4C88.6010201@gmail.com> Message-ID: <1392254870.87311.YahooMailNeo@web181004.mail.ne1.yahoo.com> From: spir Sent: Wednesday, February 12, 2014 2:27 AM > On 02/12/2014 11:07 AM, Andrew Barnert wrote: >> From: Steven D'Aprano >> >> >> I think neither Alex Rodrigues's initial idea nor Bruce Leban's > side idea area any of these. >> >> [...] >> >> Meanwhile, Alex was proposing something very different. Really, what he > wants is runtime computational (i.e., bytecode) macros. Look at his motivating > example: >> >> ? ? ? @inline >> ? ? ? def element_wise(func): >> ? ? ? ? ? func = inline(func) >> ? ? ? ? ? for ?: >> ? ? ? ? ? ? ? func() >> >> ? ? ? def is_symmetric(matrix): >> ? ? ? ? ? element_wise(lambda: if matrix[i][j] != matrix[j][i]: return > False) >> ? ? ? ? ? return True >> >> >> Forget about how the variables work for the moment (and the fact that, on > top of his main desire, he _also_ wants a statement inside his lambda?) and look > at that "return False". Clearly it has to return from not just the > lambda, but element_wise, and is_symmetric. That's the key here; that's > what makes it "inline" in Alex's terms. It's not a function on > the stack with its own frame, it's just code that gets compiled as a > function, but then executed inline directly in the calling function's frame. > This doesn't quite work for a few relatively minor reasons (functions get an > implicit "return None", variables are referred to by index and may not > have the right indices, etc.), but the idea isn't nonsense. >> >> Once you see that, dynamic vs. lexical scope is a red herring. Either way, > the stack frame in which matrix gets evaluated is the stack frame of > is_symmetric. The existing closure variable lookup does the right thing. (Note > that the definition of the lambda forces matrix to be looked up via *_DEREF in > is_symmetric, so the *_DEREF code in the lambda is correct when inlined.) And it > would _have_ to be right for the return-through to make any sense. Dynamic scope > doesn't seem to add any possibilities here that it wouldn't already add > in regular functions; I think it's orthogonal, and irrelevant, to the > proposal. But I could be wrong. >> >> Anyway, I don't think he really needs runtime macros; he only needs > "func = inline(func)" because element_wise itself is defined as a > function and converted to a macro with "@inline". If the latter were > compile-time, the former could be as well. (Imagine a "defmacro" > statement and a "lambdamacro" expression.) And that eliminates a whole > lot of complexity (dynamically generating dynamic code-fixup code, etc.). And if > you don't need runtime macros, you probably don't need computational > macros, which eliminates even more (e.g., in the AST, "matrix" is a > name, not an index). The key thing he wants is macros. But what he asked for is > runtime computational macros. > > How does this relate to Ruby blocks, procs, and lambdas? One of them is to > solves the issue of which-is-the-actual-piece-of-code-to-return-from, IIRC > (dunno much of Ruby), like your motivating example of "inline" funcs > here. I think this is what you're getting at: Despite what Ruby programmers think, Ruby functions aren't at all like Python functions. They're not closures, they're not dynamically-created objects, they can't be passed around as first-class values, and so on. Ruby procs, on the other hand, _are_ all of that.?However, it's functions that get stack frames; procs run parasitically within functions. This means that, in some ways, they're similar to runtime macros. Relevant here, a return statement in a proc actually returns from the calling function. (Let's ignore the weird quirks that make it hard to actually put a return statement in a proc?) Blocks are just the syntax for defining proc bodies, and lambda is a special syntax that allows you to create procs with more function-like semantics (including "diminutive return", meaning return statements just exit the proc rather than its caller), which is the opposite of what we're looking for here. (Also, to make things more confusing, Ruby also has methods?which have nothing to do with classes; a method is what you get by calling the method function on the symbol for a function, and it's basically a proc-like thing that wraps the function and remembers its name.) Python could adopt this kind of two-level callable scheme. At the Python level, we could have separate syntax for defining and calling "inline functions", and a separate type for them. We'd need some way to do "diminutive return" when you want it (usually) while allowing full return when you want that, whether Ruby's way (a return statement is a full return; falling off the end of the function diminutive-returns the last expression evaluated) or something different.?Under the covers, I think all you'd need to do is add an inline function stack as a member of the normal function stack frame. This wouldn't be running them "inline" in the sense that others in the thread asked for, but I think it might give Alex everything he's asking for. I don't think this would be a change worth doing, but if someone disagrees, they can probably work it through from here. Of course in Ruby, you can't create a proc from a function. (You can wrap a function in a proc by passing its name just by calling it in a block?or, just use a method instead?but either way, that doesn't do any good. The function's just going to return to the proc.) Just as I suggested that he doesn't actually need that feature with macros, I don't think he needs it with procs either. There's no reason to create element_wise and the lamdba as functions and then turn them into procs later; just define element_wise as a proc that takes a proc, and define the lambda as a proc, and you're done. But in Python, with the scheme I suggested above, it might be possible to do this, at least if the syntax for differentiating diminutive and full returns were available in normal functions (where they'd both do the same thing, but it could be useful to distinguish them for functions that you plan to turn into inline functions later). From abarnert at yahoo.com Thu Feb 13 02:52:36 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 12 Feb 2014 17:52:36 -0800 (PST) Subject: [Python-ideas] except expression In-Reply-To: <85ha83am9x.fsf@benfinney.id.au> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> Message-ID: <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> From: Ben Finney Sent: Wednesday, February 12, 2014 4:28 PM > Subject: Re: [Python-ideas] except expression > > Ben Finney writes: > >> Ram Rachum writes: >> >> > Here's an idea that would help shortening code. Allow a ternary >> > expression based on except, like so: >> > >> >? ? first_entry = entries[0] except IndexError else None >> >? ? item = my_queue.get() except queue.Empty else None >> >? ? response_text = request('http://whatever.com').text except > HttpError else "Can't access data" >> >> That is more obscure, to my eye, than laying out the control branches: > > Sorry, I failed to address your first two examples. > > I am +0 on the proposal to have something similar to Perl's fallback > syntax, ?$foo = bar() or some_default_value?. Although this looks nicer than the original proposal, it loses the ability to specify what exception you want to catch. And I think it would be reasonable to want to, e.g., handle queue.Empty but not swallow an AttributeError caused by a typo??On the other hand, I really dislike the misuse of else in the original version. But?the syntax can be bikeshedded, and probably has been before. I have one reservation:?Given that so many functions in Python take a "default value" parameter, could this lead to making the language less predictable and consistent? For some expressions, you write "d.get(key, defval)", while for others you write "d.get(key) except KeyError else defval", and it may not be obvious which are which. And I don't think the answer is to remove all those default values. The d.get(key, defval) is much more concise and a bit more readable, and removes the opportunity to, e.g., screw up and use the wrong exception type. But other than that, if someone can come up with a readable way to write it, I like the idea. > Yet I still find the proposed syntax less readable for anything but a > trivial *and* brief case. I agree?but the same is true for pretty much all expression syntax, and most of it is rarely misused. Consider if-else ternary expressions. It's very easy to make your code completely unreadable by chaining them together,?or using complex test expressions, or using them in the middle of a comprehension, etc. But you rarely see such code. And meanwhile, you see a lot of code that's more concise and readable because it uses trivial if expressions.?I think that except expressions would go the same way. > For anything longer than a few dozen > characters, I still prefer:: > > ? ? try: > ? ? ? ? response_text = request('http://whatever.com').text > ? ? except HttpError: > ? ? ? ? "Can't access data" >? > for being explicit and clarifying what to expect. Except that you're not doing the same thing as the original; you're just evaluating and throwing away the string literal, and not assigning anything to response_text. Having to write "response_text = " twice gives you twice as many places to screw up?as evidenced by the fact that you did so.?The exact same thing happens with if statements vs. if expressions, and in fact I think that's one of the reasons people like if expressions. From amber.yust at gmail.com Thu Feb 13 04:08:14 2014 From: amber.yust at gmail.com (Amber Yust) Date: Wed, 12 Feb 2014 19:08:14 -0800 Subject: [Python-ideas] except expression In-Reply-To: <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: Why not use yield instead of else? foo = something() except BazException yield "bar" On Feb 12, 2014 5:56 PM, "Andrew Barnert" wrote: > From: Ben Finney > > Sent: Wednesday, February 12, 2014 4:28 PM > > > > Subject: Re: [Python-ideas] except expression > > > > Ben Finney writes: > > > >> Ram Rachum writes: > >> > >> > Here's an idea that would help shortening code. Allow a ternary > >> > expression based on except, like so: > >> > > >> > first_entry = entries[0] except IndexError else None > >> > item = my_queue.get() except queue.Empty else None > >> > response_text = request('http://whatever.com').text except > > HttpError else "Can't access data" > >> > >> That is more obscure, to my eye, than laying out the control branches: > > > > Sorry, I failed to address your first two examples. > > > > I am +0 on the proposal to have something similar to Perl's fallback > > syntax, ?$foo = bar() or some_default_value?. > > Although this looks nicer than the original proposal, it loses the ability > to specify what exception you want to catch. And I think it would be > reasonable to want to, e.g., handle queue.Empty but not swallow an > AttributeError caused by a typo? On the other hand, I really dislike the > misuse of else in the original version. But the syntax can be bikeshedded, > and probably has been before. > > I have one reservation: Given that so many functions in Python take a > "default value" parameter, could this lead to making the language less > predictable and consistent? For some expressions, you write "d.get(key, > defval)", while for others you write "d.get(key) except KeyError else > defval", and it may not be obvious which are which. And I don't think the > answer is to remove all those default values. The d.get(key, defval) is > much more concise and a bit more readable, and removes the opportunity to, > e.g., screw up and use the wrong exception type. > > But other than that, if someone can come up with a readable way to write > it, I like the idea. > > > Yet I still find the proposed syntax less readable for anything but a > > trivial *and* brief case. > > I agree?but the same is true for pretty much all expression syntax, and > most of it is rarely misused. > > Consider if-else ternary expressions. It's very easy to make your code > completely unreadable by chaining them together, or using complex test > expressions, or using them in the middle of a comprehension, etc. But you > rarely see such code. And meanwhile, you see a lot of code that's more > concise and readable because it uses trivial if expressions. I think that > except expressions would go the same way. > > > For anything longer than a few dozen > > > characters, I still prefer:: > > > > try: > > response_text = request('http://whatever.com').text > > except HttpError: > > "Can't access data" > > > > for being explicit and clarifying what to expect. > > > Except that you're not doing the same thing as the original; you're just > evaluating and throwing away the string literal, and not assigning anything > to response_text. Having to write "response_text = " twice gives you twice > as many places to screw up?as evidenced by the fact that you did so. The > exact same thing happens with if statements vs. if expressions, and in fact > I think that's one of the reasons people like if expressions. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Thu Feb 13 04:16:00 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 13 Feb 2014 14:16:00 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: On Thu, Feb 13, 2014 at 2:08 PM, Amber Yust wrote: > Why not use yield instead of else? > > foo = something() except BazException yield "bar" yield is already an expression. It'd be theoretically and syntactically valid (if a little weird) to use yield "bar" in place of the name BazException; you'd yield "bar" to your caller, then whatever exception gets sent in would be the one tested for. I honestly cannot conceive of any situation where this would actually be useful, but it does make it a little tricky to reuse that keyword :) ChrisA From amber.yust at gmail.com Thu Feb 13 04:20:03 2014 From: amber.yust at gmail.com (Amber Yust) Date: Wed, 12 Feb 2014 19:20:03 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: Ah, that's a good point (the two-directionality of yield had slipped my mind). I had considered suggesting return instead of yield, which wouldn't have that problem, but it felt like return would be more confusing to see in a context where it doesn't actually return from the enclosing scope. On Feb 12, 2014 7:16 PM, "Chris Angelico" wrote: > On Thu, Feb 13, 2014 at 2:08 PM, Amber Yust wrote: > > Why not use yield instead of else? > > > > foo = something() except BazException yield "bar" > > yield is already an expression. It'd be theoretically and > syntactically valid (if a little weird) to use yield "bar" in place of > the name BazException; you'd yield "bar" to your caller, then whatever > exception gets sent in would be the one tested for. I honestly cannot > conceive of any situation where this would actually be useful, but it > does make it a little tricky to reuse that keyword :) > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amber.yust at gmail.com Thu Feb 13 04:25:50 2014 From: amber.yust at gmail.com (Amber Yust) Date: Wed, 12 Feb 2014 19:25:50 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: Another possible option: foo = something() except None for BarException With possible support for: foo = something() except e.message for BarException as e On Feb 12, 2014 7:20 PM, "Amber Yust" wrote: > Ah, that's a good point (the two-directionality of yield had slipped my > mind). I had considered suggesting return instead of yield, which wouldn't > have that problem, but it felt like return would be more confusing to see > in a context where it doesn't actually return from the enclosing scope. > On Feb 12, 2014 7:16 PM, "Chris Angelico" wrote: > >> On Thu, Feb 13, 2014 at 2:08 PM, Amber Yust wrote: >> > Why not use yield instead of else? >> > >> > foo = something() except BazException yield "bar" >> >> yield is already an expression. It'd be theoretically and >> syntactically valid (if a little weird) to use yield "bar" in place of >> the name BazException; you'd yield "bar" to your caller, then whatever >> exception gets sent in would be the one tested for. I honestly cannot >> conceive of any situation where this would actually be useful, but it >> does make it a little tricky to reuse that keyword :) >> >> ChrisA >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Thu Feb 13 04:38:24 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 13 Feb 2014 14:38:24 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: On Thu, Feb 13, 2014 at 2:20 PM, Amber Yust wrote: > Ah, that's a good point (the two-directionality of yield had slipped my > mind). I had considered suggesting return instead of yield, which wouldn't > have that problem, but it felt like return would be more confusing to see in > a context where it doesn't actually return from the enclosing scope. Yeah. I like the parallel with "get this unless it's empty in which case use this default", but the 'or' keyword is already a bit awkward there, so I don't want to advocate that. name = input("Name [Anonymous]: ") or "Anonymous" phone = addressbook[name] except KeyError or "Unknown" It's a nice parallel, but doesn't read well (plus, >> KeyError or "Unknown" << is already an expression). +1 on the feature but it definitely needs a syntax that makes sense. Of course, it could be done as a function call: def catch(callme, catchme, returnme): try: return callme() except catchme: return returnme phone = catch(lambda: addressbook[name], KeyError, "Unknown") but that's *really* clunky. ChrisA From abarnert at yahoo.com Thu Feb 13 05:41:25 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 12 Feb 2014 20:41:25 -0800 Subject: [Python-ideas] a in x or in y In-Reply-To: <52FC09DD.3080100@oddbird.net> References: <52FC09DD.3080100@oddbird.net> Message-ID: <0312F54D-63DE-4B51-8BAD-C4205F17FAAB@yahoo.com> On Feb 12, 2014, at 15:55, Carl Meyer wrote: > On 02/12/2014 02:12 PM, Ram Rachum wrote: >> What do you think about adding this to Python: >> >> 'whatever a long string' in x or in y >> >> I've often wished for this because the current way is quite verbose: >> >> 'whatever a long string' in x or 'whatever a long string' in y For future reference, using expression_with_side_effects() instead of 'whatever a long string' would be a more compelling use case. With a long string, it's inconvenient and ugly to repeat it; with an expression with side effects, it's all that plus incorrect to boot. But of course the same solutions still work. > Except you'd never actually do that, you'd just put the long string in a > variable. Or the other option: > > any('whatever a long string' in i for i in [x, y]) Or, if you're doing this so often that even this is too verbose: def in_any(element, *containers): return any(element in container for container in containers) in_any('whatever a long string', x, y, z, w) If you're not doing it often enough for in_any to become familiar, then you didn't have a problem to solve in the first place. :) From antony.lee at berkeley.edu Thu Feb 13 06:46:37 2014 From: antony.lee at berkeley.edu (Antony Lee) Date: Wed, 12 Feb 2014 21:46:37 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: Even more generally (not sure allowing multiple clauses is a good idea but at least "if" sounds better than "for", I think. foo = bar() except e.attr1 if FooException as e else e.attr2 if BarException as e Antony Lee 2014-02-12 19:25 GMT-08:00 Amber Yust : > Another possible option: > > foo = something() except None for BarException > > With possible support for: > > foo = something() except e.message for BarException as e > On Feb 12, 2014 7:20 PM, "Amber Yust" wrote: > >> Ah, that's a good point (the two-directionality of yield had slipped my >> mind). I had considered suggesting return instead of yield, which wouldn't >> have that problem, but it felt like return would be more confusing to see >> in a context where it doesn't actually return from the enclosing scope. >> On Feb 12, 2014 7:16 PM, "Chris Angelico" wrote: >> >>> On Thu, Feb 13, 2014 at 2:08 PM, Amber Yust >>> wrote: >>> > Why not use yield instead of else? >>> > >>> > foo = something() except BazException yield "bar" >>> >>> yield is already an expression. It'd be theoretically and >>> syntactically valid (if a little weird) to use yield "bar" in place of >>> the name BazException; you'd yield "bar" to your caller, then whatever >>> exception gets sent in would be the one tested for. I honestly cannot >>> conceive of any situation where this would actually be useful, but it >>> does make it a little tricky to reuse that keyword :) >>> >>> ChrisA >>> _______________________________________________ >>> Python-ideas mailing list >>> Python-ideas at python.org >>> https://mail.python.org/mailman/listinfo/python-ideas >>> Code of Conduct: http://python.org/psf/codeofconduct/ >>> >> > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amber.yust at gmail.com Thu Feb 13 06:53:39 2014 From: amber.yust at gmail.com (Amber Yust) Date: Thu, 13 Feb 2014 05:53:39 +0000 Subject: [Python-ideas] except expression References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: That has a potential conflict in parsing, however, since the if-else ternary already exists. Yes, the 'as' theoretically disambiguates it for the machine parser, but for a human trying to parse it as they read, it's not nearly as simple. The reason why I suggested 'for' is that it still flows somewhat naturally - it's using "for" in the sense of "associated with" or "in the place of". "I'll give you my None for your FooException." On Wed Feb 12 2014 at 9:46:37 PM, Antony Lee wrote: > Even more generally (not sure allowing multiple clauses is a good idea but > at least "if" sounds better than "for", I think. > foo = bar() except e.attr1 if FooException as e else e.attr2 if > BarException as e > > Antony Lee > > > 2014-02-12 19:25 GMT-08:00 Amber Yust : > > Another possible option: > > foo = something() except None for BarException > > With possible support for: > > foo = something() except e.message for BarException as e > On Feb 12, 2014 7:20 PM, "Amber Yust" wrote: > > Ah, that's a good point (the two-directionality of yield had slipped my > mind). I had considered suggesting return instead of yield, which wouldn't > have that problem, but it felt like return would be more confusing to see > in a context where it doesn't actually return from the enclosing scope. > On Feb 12, 2014 7:16 PM, "Chris Angelico" wrote: > > On Thu, Feb 13, 2014 at 2:08 PM, Amber Yust wrote: > > Why not use yield instead of else? > > > > foo = something() except BazException yield "bar" > > yield is already an expression. It'd be theoretically and > syntactically valid (if a little weird) to use yield "bar" in place of > the name BazException; you'd yield "bar" to your caller, then whatever > exception gets sent in would be the one tested for. I honestly cannot > conceive of any situation where this would actually be useful, but it > does make it a little tricky to reuse that keyword :) > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Thu Feb 13 10:24:59 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 13 Feb 2014 19:24:59 +1000 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: General comment: like Raymond, I'm inclined to favour a nice expression friendly exception handling syntax, precisely because of the proliferation of relatively ad hoc alternative solutions (in particular, the popularity of being able to pass in default values to handle empty iterables). One use case, for example, is handing IndexError when retrieving an item from a sequence (which currently has no nice standard spelling, and isn't amenable to the "pass in a default answer" solution because it isn't a normal function call). Another case not handled well by the status quo is when the default answer is expensive to calculate for some reason, so you really only want to calculate it if you actually need it. Unfortunately, like PEP 308 before it, the hard part is coming up with a reasonable spelling that won't have people breaking out the torches and pitchforks if the PEP is accepted. On 13 Feb 2014 13:34, "Amber Yust" wrote: > > Another possible option: > > foo = something() except None for BarException > > With possible support for: > > foo = something() except e.message for BarException as e This is also one of the possible spellings I came up with, and it is my current least disliked option. The main concern I have with it is the substantially different interpretation it gives to the "for" keyword - as far as is practical, we try to ensure that a given keyword relates to a consistent concept, and the link to iteration is rather tenuous here (it's only present in the fact you can use an iterable of exception types rather than just one). Aside from that concern, I think it scores well on the readability front. "if" would be better, but, as you already noted, poses significant parsing challenges (since it likely wouldn't be easy to require that ternary expressions use parentheses in this new construct, but still allow the parentheses to be omitted in the general case). Cheers, Nick. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Thu Feb 13 10:28:49 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 13 Feb 2014 19:28:49 +1000 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: On 13 Feb 2014 19:24, "Nick Coghlan" wrote: > > General comment: like Raymond, I'm inclined to favour a nice expression friendly exception handling syntax, precisely because of the proliferation of relatively ad hoc alternative solutions (in particular, the popularity of being able to pass in default values to handle empty iterables). > > One use case, for example, is handing IndexError when retrieving an item from a sequence (which currently has no nice standard spelling, and isn't amenable to the "pass in a default answer" solution because it isn't a normal function call). > > Another case not handled well by the status quo is when the default answer is expensive to calculate for some reason, so you really only want to calculate it if you actually need it. > > Unfortunately, like PEP 308 before it, the hard part is coming up with a reasonable spelling that won't have people breaking out the torches and pitchforks if the PEP is accepted. > > On 13 Feb 2014 13:34, "Amber Yust" wrote: > > > > Another possible option: > > > > foo = something() except None for BarException > > > > With possible support for: > > > > foo = something() except e.message for BarException as e > > This is also one of the possible spellings I came up with, and it is my current least disliked option. The main concern I have with it is the substantially different interpretation it gives to the "for" keyword - as far as is practical, we try to ensure that a given keyword relates to a consistent concept, and the link to iteration is rather tenuous here (it's only present in the fact you can use an iterable of exception types rather than just one). Aside from that concern, I think it scores well on the readability front. > > "if" would be better, but, as you already noted, poses significant parsing challenges (since it likely wouldn't be easy to require that ternary expressions use parentheses in this new construct, but still allow the parentheses to be omitted in the general case). "from" is another keyword choice worth considering here - that already has no strong semantics of its own ("from x import y" and "yield from iter" derive their meaning from the other keyword involved) Cheers, Nick. > > Cheers, > Nick. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mal at egenix.com Thu Feb 13 11:10:40 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 13 Feb 2014 11:10:40 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: <52FC9A20.9040303@egenix.com> On 13.02.2014 10:24, Nick Coghlan wrote: > General comment: like Raymond, I'm inclined to favour a nice expression > friendly exception handling syntax, precisely because of the proliferation > of relatively ad hoc alternative solutions (in particular, the popularity > of being able to pass in default values to handle empty iterables). Here's a variant the resembles the code you'd write in a helper function to achieve the same thing, only stripped down somewhat: x = something() except ValueError return default_value def try_something(): try: return something() except ValueError: return default_value x = something() except ValueError as exc return exc.message def try_something(): try: return something() except ValueError as exc return exc.message Obviously, having a keyword "use" would make a better fit :-) x = something() except ValueError use default_value -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 13 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From denis.spir at gmail.com Thu Feb 13 11:22:21 2014 From: denis.spir at gmail.com (spir) Date: Thu, 13 Feb 2014 11:22:21 +0100 Subject: [Python-ideas] on colon ':', and some more Message-ID: <52FC9CDD.8010003@gmail.com> Hello, This is not a proposal for change, rather a few reflexions I guess were worth being shared here, because this is a list where design questions about Python are debated. If you don't care, just press 'del'. I have always been puzzled by Python's usage of colons terminating block headlines. First, this does not match the overall noise-less design of the syntax; second, it compares weirdly with the absence of (required) semi-colons terminating instructions [1]. I stepped several times, in online articles, on vague evocations of "studies" supposed to "show" that this format is clearer to beginners, albeit each time without any reference to said studies [2]. Finally, I recently found what seems to be the actual story of ':' [3], telling that, according to _one_ non-programmer, colons should be added there to play the role of _introducing_ the following block, indented. This notion of _introduction_ endly clicked for me, making some sense. From a distinct perspective: another weird point is using '=' for assignment and '==' for equality [4]. In my view, the right assignment sign (rather: the sign to define a symbol, in general) is ':' precisely [5]. While thinking at syntax for a low-level language, I stepped on the special case of symbols which are labelled blocks (all targets of jumps/goto's, including procedures). In assembly and some other langs, they are usually written 'label:' or ':label:'. And indeed it is clear here that the label introduces the following block, with ':' obviously playing this introductory role. Why is it instead not obvious in Python [6]? Maybe the reason lies in the actual usage of colons, in fact in the actual headlines in python code. In assembly, one could have code looking like: jnz case-true ... (case false) ... case-true: ld rax result ... Here the label is a kind of title line for the block. There is a similar usage in python: def average (numbers): sum = numbers.sum() return sum / len(numbers) Again, here the headline is a kind of title. However, when the block is a control statement if/else, for, while, then the headline is rather an _articulation_ in the discourse: for n in numbers: print(n) while more: go_on() if cond: this() else that() From yet a third perspective: for a high-level language this time, which otherwise uses indented blocks like Python, I was searching for a nice syntax for single-line control blocks [7]. As in python's: for n in numbers: print(n) while cond: go_on() if cond: this() else: that() I first thought at using 'do': for n in numbers do print(n) while cond do go_on() if cond do this() else do that() if cond do this() else do that() which is very nice, I guess. [8] [9] Then, for whatever reason, came to my mind the following: for n in numbers, print(n) while cond, go_on() if cond, this() else, that() if cond, this() else, that() Just using ',', a bare comma. [10] It seems to me that this usage of the comma matches its meaning in ordinary (written) discourse. And there is indeed a question of discourse articulation. Here, of logical articulation, precisely. Maybe this is why, finally, Python's usage of the colon does not meaningfully click as it should: because most of the time it's not a question of a title introducing a block, but of discourse articulation; where ',' would do a better job, maybe. Note: I don't mean this post to introduce (sic!) endless violent discussions (as often, on such topics); and I won't reply if this happens. I wished to share a different view on the topic, for people who care about meaningful syntax. d [1] I think the parser should not an accept optional ';' at end of line. This alternative is never used, and, if it were, would only trouble novice programmers (What the heck does ';' mean? nothing...) [2] Which always made me laugh, knowing how supposed studies in the fields of psychology or sociology usually are highly contestable; so what about programmers or CS professionals driving such studies? [3] Someone will surely have a pointer ;-) [4] Forgetting ':' and using '=' for equality are my favorite syntax errors in Python. I seem to still love them, still after long years of frequentation. [5] Python uses it in dict notation, which is nice. But dict key->value links could use '->', while for symbol defs ':' or '?' or '?' would be perfect. (Actually, the double head arrow is ideal, but unfortunately there is no such key on common keyboards.) [6] Not obvious at all, for me, instead I had to be told about it, I mean to read about it. Maybe it is non-obvious for most people, since this argument does not seem to come up in discussions about ':'. [7] The actual keywords are different, also the syntactic schema for procedure call, but I will use Python's syntax here in examples for readability. [8] However, syntax highlighting for keywords would help, here. [9] Not invent here: Lua uses 'do'; however also for multiline blocks, and only for loops, not for if/else. [10] This is even better in the case of this lang project, where like in Lisp collection items are just separated by spaces: no commas. Thus, the comma has a clear and single role in syntax. From denis.spir at gmail.com Thu Feb 13 11:39:00 2014 From: denis.spir at gmail.com (spir) Date: Thu, 13 Feb 2014 11:39:00 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: <52FCA0C4.4080303@gmail.com> On 02/13/2014 06:46 AM, Antony Lee wrote: > Even more generally (not sure allowing multiple clauses is a good idea but > at least "if" sounds better than "for", I think. Agreed. foo = something() except None if BarException d From denis.spir at gmail.com Thu Feb 13 11:45:00 2014 From: denis.spir at gmail.com (spir) Date: Thu, 13 Feb 2014 11:45:00 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: <52FCA22C.6060406@gmail.com> On 02/13/2014 06:53 AM, Amber Yust wrote: > The reason why I suggested 'for' is that it still flows somewhat naturally > - it's using "for" in the sense of "associated with" or "in the place of". > "I'll give you my None for your FooException." True, but it is probably not wise to use a natural language preposition in two different syntactic schemas with two different meanings: here 'for' meaning either traversal loop or "expression-exception-condition" (would be ok if the meaning was the same). d From steve at pearwood.info Thu Feb 13 11:57:17 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 13 Feb 2014 21:57:17 +1100 Subject: [Python-ideas] a in x or in y In-Reply-To: References: Message-ID: <20140213105716.GA3799@ando> On Wed, Feb 12, 2014 at 01:12:17PM -0800, Ram Rachum wrote: > Hi, > > What do you think about adding this to Python: > > 'whatever a long string' in x or in y I like it. In natural language, people often say things like: my keys are in the car or in my pocket which fools them into writing: keys in car or pocket which does the wrong thing. Chained "in" comparisons is a natural extension to Python's already natural language-like syntax. Python already has other chained comparisons. Being able to write: keys in car or in pocket feels natural and right to me. (We can't *quite* match the human idiom where the second "in" is left out, but one can't have everything.) This is particularly useful when there are side-effects involved: something_with_side_effects() in this and in that or in other I'm not usually one for introducing syntax just to avoid a temporary variable or extra line: temp = something_with_side_effects() temp in this and temp in that or temp in other but I think that chained comparisons are one of Python's best syntactic features, and this just extends it to "in". The only two concerns I have are: - given the restrictions on the parser, is this even possible? and - the difference between "x in y and z" and "x in y and in z" is quite subtle, and hence may be an unfortunately common source of errors. So a tentative +1 on the idea. -- Steven From denis.spir at gmail.com Thu Feb 13 12:02:03 2014 From: denis.spir at gmail.com (spir) Date: Thu, 13 Feb 2014 12:02:03 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: <52FCA62B.90005@gmail.com> On 02/13/2014 10:24 AM, Nick Coghlan wrote: > General comment: like Raymond, I'm inclined to favour a nice expression > friendly exception handling syntax, precisely because of the proliferation > of relatively ad hoc alternative solutions (in particular, the popularity > of being able to pass in default values to handle empty iterables). I think the right way is not to call the function at all, but to check it. Conceptually: if col.is_empty(): handle_special_case() else: handle_standard_case() > One use case, for example, is handing IndexError when retrieving an item > from a sequence (which currently has no nice standard spelling, and isn't > amenable to the "pass in a default answer" solution because it isn't a > normal function call). > > Another case not handled well by the status quo is when the default answer > is expensive to calculate for some reason, so you really only want to > calculate it if you actually need it. The above schema applies to all cases where the failure (for the called service to perform its task) is _predictable_ by the client. Exceptions, in my view, are only for cases where the failure is impredictable (see below) *and* nevertheless belongs to to the application semantics; meaning, it does not result from a programming error (eg the collection should _not_ be empty), else one should not use exception catching, but instead let an error message helpfully inform us about the logical error. Typical cases of impredictability on the client side are search/find functions, and dealing with the outer world, in particular the file system. In addition, in such cases using for instance a 'has' function (or in python 'in') to first check would do the job (of searching) twice. This is why, probably, there are alternative funcs like python's 'get' for dicts. Maybe this scheme could be generalised: not only eg list.get(i, default), but all cases of potentially failing funcs that return a result. For functions (actions, in fact) that instead perform an effect, cases are more diverse and not always in our hands. For instance, one may think at a func which would create file if not existant, instead of replacing its contents (or appending to it): but this would I guess require the filesystem to provide such a facility. As far as I know, we are left with having routines search twice (for the abstract file location in the dir tree). d From ben+python at benfinney.id.au Thu Feb 13 12:10:16 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 13 Feb 2014 22:10:16 +1100 Subject: [Python-ideas] except expression References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCA62B.90005@gmail.com> Message-ID: <854n439skn.fsf@benfinney.id.au> spir writes: > I think the right way is not to call the function at all, but to check > it. Conceptually: > > if col.is_empty(): > handle_special_case() > else: > handle_standard_case() Or, better from two perspectives (?empty? should normally entail ?evaluates to boolean false?; and, the normal case should be the first branch from the ?if?):: if col: handle_standard_case() else: handle_empty_case() -- \ ?Intellectual property is to the 21st century what the slave | `\ trade was to the 16th.? ?David Mertz | _o__) | Ben Finney From denis.spir at gmail.com Thu Feb 13 12:25:07 2014 From: denis.spir at gmail.com (spir) Date: Thu, 13 Feb 2014 12:25:07 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: <52FCAB93.40504@gmail.com> On 02/13/2014 04:38 AM, Chris Angelico wrote: > On Thu, Feb 13, 2014 at 2:20 PM, Amber Yust wrote: >> Ah, that's a good point (the two-directionality of yield had slipped my >> mind). I had considered suggesting return instead of yield, which wouldn't >> have that problem, but it felt like return would be more confusing to see in >> a context where it doesn't actually return from the enclosing scope. > > Yeah. I like the parallel with "get this unless it's empty in which > case use this default", but the 'or' keyword is already a bit awkward > there, so I don't want to advocate that. > > name = input("Name [Anonymous]: ") or "Anonymous" > phone = addressbook[name] except KeyError or "Unknown" > > It's a nice parallel, but doesn't read well (plus, >> KeyError or > "Unknown" << is already an expression). > > +1 on the feature but it definitely needs a syntax that makes sense. I don't see any issue with: phone = addressbook[name] except "Unknown" if KeyError phone = addressbook[name] except "Unknown" if KeyError as e It is similar in syntax and meaning with if-expressions, with the first keyword 'except' obviously making all the difference we need. [By the way, this shows that: x = b if cond else a should really be: x = a else b if cond The difference being that the standard case is expressed first, the exceptional one being then introduced as an special variant.] In some languages (eg Lua) there is no difference between absent values (vars, attributes, items...) because of an arror (they should be there, in python we get an exception) and optional values (in python there is None for this meaning). In the latter case, presence of absence both are valid alternatives in the app's logic. Such a lack of distinction (in Lua, both yield nil) is very bad, indeed, but combined with "permissive" logical expressions (in which operands may not be logical, and result value as well) allows: phone = addressbook[name] or "Unknown" However, this idiom is mainly common in lua because there are no standard param values (defaults): Shape.fill = function (shape, color) color = color or black ... end This also shows that the cases in python were we do need such an idiom are pretty rare, all in all: should we bother? > Of course, it could be done as a function call: > > def catch(callme, catchme, returnme): > try: > return callme() > except catchme: > return returnme > > phone = catch(lambda: addressbook[name], KeyError, "Unknown") > > but that's *really* clunky. I'd like such a solution if builtin (for it to be standard, thus widely shared in python code) and did not force writing a lambda. d From denis.spir at gmail.com Thu Feb 13 12:27:13 2014 From: denis.spir at gmail.com (spir) Date: Thu, 13 Feb 2014 12:27:13 +0100 Subject: [Python-ideas] except expression In-Reply-To: <854n439skn.fsf@benfinney.id.au> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCA62B.90005@gmail.com> <854n439skn.fsf@benfinney.id.au> Message-ID: <52FCAC11.6060009@gmail.com> On 02/13/2014 12:10 PM, Ben Finney wrote: > spir writes: > >> I think the right way is not to call the function at all, but to check >> it. Conceptually: >> >> if col.is_empty(): >> handle_special_case() >> else: >> handle_standard_case() > > Or, better from two perspectives (?empty? should normally entail > ?evaluates to boolean false?; and, the normal case should be the first > branch from the ?if?):: > > if col: > handle_standard_case() > else: > handle_empty_case() You are right (I always forget that empty means false in python, in a logical context, which i don't find obvious at all --I wonder if any other lang follows this choice). d From rosuav at gmail.com Thu Feb 13 12:27:19 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 13 Feb 2014 22:27:19 +1100 Subject: [Python-ideas] a in x or in y In-Reply-To: <20140213105716.GA3799@ando> References: <20140213105716.GA3799@ando> Message-ID: On Thu, Feb 13, 2014 at 9:57 PM, Steven D'Aprano wrote: > - given the restrictions on the parser, is this even possible? and Generalizing the syntax, I'd see this as: operand1 binary-op1 operand2 {and|or} binary-op2 operand3 which implicitly places the value (not the code) of operand1 between and/or and binary-op2. Since all binary operators have higher precedence than either and or or (all that's lower is lambda and if/else), this notation is currently guaranteed to fail... except in two cases, namely + and -, which exist in unary form as well. So if + and - are excluded, it should be unambiguous. Whether or not the parser can actually handle it is a question for someone who knows what he's talking about, though :) The way I see it, there should ideally be no syntactic rule against using different operators on the two sides: input("> ") in legalcommands and not in forbiddencommands value > 20 or in {3,5,7,11} even though it would allow insanity: if 5 < int(input("Enter a number: ")) or < int(input("Greater than five please: ")) or < int(input("Come on now! ")) or == print("Bah, I give up."): print("Thank you.") In this way, it's like chained comparisons: 1 < x <= 10 which don't mind mixing and matching. ChrisA From ncoghlan at gmail.com Thu Feb 13 12:31:40 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 13 Feb 2014 21:31:40 +1000 Subject: [Python-ideas] on colon ':', and some more In-Reply-To: <52FC9CDD.8010003@gmail.com> References: <52FC9CDD.8010003@gmail.com> Message-ID: On 13 February 2014 20:22, spir wrote: > Hello, > > This is not a proposal for change, rather a few reflexions I guess were > worth being shared here, because this is a list where design questions about > Python are debated. If you don't care, just press 'del'. > > I have always been puzzled by Python's usage of colons terminating block > headlines. First, this does not match the overall noise-less design of the > syntax; second, it compares weirdly with the absence of (required) > semi-colons terminating instructions [1]. I stepped several times, in online > articles, on vague evocations of "studies" supposed to "show" that this > format is clearer to beginners, albeit each time without any reference to > said studies [2]. Finally, I recently found what seems to be the actual > story of ':' [3], telling that, according to _one_ non-programmer, colons > should be added there to play the role of _introducing_ the following block, > indented. This notion of _introduction_ endly clicked for me, making some > sense. For your footnote 3: http://python-history.blogspot.com.au/2011/07/karin-dewar-indentation-and-colon.html :) There may also be more in the papers about ABC: http://homepages.cwi.nl/~steven/abc/publications.html (I skimmed the "Issues in the Design of a Beginners' Programming Language" one, which has some interesting parts about the indentation based syntax, but doesn't really discuss the value of the colons that introduce a new suite - I do personally like them though, as they provide a clear distinction between statement headers and multi-line expressions, especially when the header *includes* a multi-line expression) > From a distinct perspective: another weird point is using '=' for assignment > and '==' for equality [4]. In my view, the right assignment sign (rather: > the sign to define a symbol, in general) is ':' precisely [5]. While > thinking at syntax for a low-level language, I stepped on the special case > of symbols which are labelled blocks (all targets of jumps/goto's, including > procedures). In assembly and some other langs, they are usually written > 'label:' or ':label:'. And indeed it is clear here that the label introduces > the following block, with ':' obviously playing this introductory role. As far as I am aware, the =/== annotation is just an artefact of Python's C heritage, rather than being based on anything more profound. > Why is it instead not obvious in Python [6]? I don't know if Guido actually designed it this way, but if you evaluate Python's original syntax as "simple statements are based on C, compound statements are based on ABC", a lot of things make more sense (and then the rest of the language evolved from there). So if someone comes to Python having learned a language like C, C++ or Java first, then it is only the ABC inspired bits (especially the indentation based suites) that will puzzle them. If a new user learns Python as their first programming language, then it all seems arbitrary anyway, so they have no reason to expect anything different. It requires expectations calibrated in a different environment (perhaps including formal mathematics?) to make the C-style =/== model seem particularly strange. Mathematicians may also be puzzled by the use of "j" for complex numbers (although electrical engineers will find that notation entirely familiar). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From p.f.moore at gmail.com Thu Feb 13 12:36:55 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 13 Feb 2014 11:36:55 +0000 Subject: [Python-ideas] except expression In-Reply-To: <52FCAB93.40504@gmail.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: On 13 February 2014 11:25, spir wrote: > I don't see any issue with: > phone = addressbook[name] except "Unknown" if KeyError > phone = addressbook[name] except "Unknown" if KeyError as e > It is similar in syntax and meaning with if-expressions, with the first > keyword 'except' obviously making all the difference we need. What I dislike about this variant is that in a normal try statement, the exception goes after the "except" keyword. Here, it's the alternative value that goes there. I find that very easy to misread. Personally, I quite like "EXPR except EXCEPTIONTYPE return VALUE" if we have to stick with existing keywords. If I had free rein I might go for a new keyword "then" instead of "return". Paul From rosuav at gmail.com Thu Feb 13 12:48:08 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 13 Feb 2014 22:48:08 +1100 Subject: [Python-ideas] except expression In-Reply-To: <52FCAB93.40504@gmail.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: On Thu, Feb 13, 2014 at 10:25 PM, spir wrote: > I don't see any issue with: > phone = addressbook[name] except "Unknown" if KeyError > phone = addressbook[name] except "Unknown" if KeyError as e > It is similar in syntax and meaning with if-expressions, with the first > keyword 'except' obviously making all the difference we need. So the keyword 'if' would come up in two different expression contexts: value if cond else othervalue value except othervalue if cond In each case, the condition (which in the latter is an exception type, while in the former it's a boolean) follows the word 'if', just as it does in the statement form. That's reasonably elegant. Problem is, that loses the elegance of matching the statement form of 'except', which has the exception type following the word 'except'. I'd kinda like to see it worded as "if except", but then it needs something else to separate the exception(s) from the value. It could actually be done just like the if/else form, except (pun intended) that that overemphasizes the exceptional case: phone = "Unknown" if except KeyError else addressbook[name] In terms of argument order, I'm looking for: phone = addressbook[name] unless except KeyError as e then "Unknown" but neither 'unless' nor 'then' is currently a keyword. > [By the way, this shows that: > x = b if cond else a > should really be: > x = a else b if cond > The difference being that the standard case is expressed first, the > exceptional one being then introduced as an special variant.] My example tends to agree with you... but my example is using "if" to introduce the abnormal case, whereas it's common to spell a block if the other way: if normal-case: code code code else: abnormal code C's ternary operator puts the expressions in the order "condition, if_true, if_false". Python's puts them "if_true, condition, if_false". You're proposing "if_false, if_true, condition". We're half way to covering all the permutations! ChrisA From rosuav at gmail.com Thu Feb 13 12:50:36 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 13 Feb 2014 22:50:36 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: On Thu, Feb 13, 2014 at 10:36 PM, Paul Moore wrote: > Personally, I quite like "EXPR except EXCEPTIONTYPE return VALUE" if > we have to stick with existing keywords. If I had free rein I might go > for a new keyword "then" instead of "return". Hmm. Stupid idea: EXPR expr EXCEPTIONTYPE pass VALUE Passing something is kinda like returning it, right? *ducks the rotten tomatoes* ChrisA From denis.spir at gmail.com Thu Feb 13 12:52:35 2014 From: denis.spir at gmail.com (spir) Date: Thu, 13 Feb 2014 12:52:35 +0100 Subject: [Python-ideas] on colon ':', and some more In-Reply-To: References: <52FC9CDD.8010003@gmail.com> Message-ID: <52FCB203.1070507@gmail.com> On 02/13/2014 12:31 PM, Nick Coghlan wrote: > For your footnote 3: > http://python-history.blogspot.com.au/2011/07/karin-dewar-indentation-and-colon.html > :) Thank you! > There may also be more in the papers about ABC: > http://homepages.cwi.nl/~steven/abc/publications.html (I skimmed the > "Issues in the Design of a Beginners' Programming Language" Read it as well. > [...] >> Why is it instead not obvious in Python [6]? > > I don't know if Guido actually designed it this way, but if you > evaluate Python's original syntax as "simple statements are based on > C, compound statements are based on ABC", a lot of things make more > sense (and then the rest of the language evolved from there). So if > someone comes to Python having learned a language like C, C++ or Java > first, then it is only the ABC inspired bits (especially the > indentation based suites) that will puzzle them. If a new user learns > Python as their first programming language, then it all seems > arbitrary anyway, so they have no reason to expect anything different. This perspective makes sense. Although for beginners, precisely, I think using ',' is meaningful. > It requires expectations calibrated in a different environment > (perhaps including formal mathematics?) to make the C-style =/== model > seem particularly strange. Mathematicians may also be puzzled by the > use of "j" for complex numbers (although electrical engineers will > find that notation entirely familiar). Yes, 'j' is not weird at all for me ;-) (background in automation, which also includes much of electrotechnics) d From denis.spir at gmail.com Thu Feb 13 12:53:28 2014 From: denis.spir at gmail.com (spir) Date: Thu, 13 Feb 2014 12:53:28 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: <52FCB238.4040602@gmail.com> On 02/13/2014 12:36 PM, Paul Moore wrote: > On 13 February 2014 11:25, spir wrote: >> >I don't see any issue with: >> > phone = addressbook[name] except "Unknown" if KeyError >> > phone = addressbook[name] except "Unknown" if KeyError as e >> >It is similar in syntax and meaning with if-expressions, with the first >> >keyword 'except' obviously making all the difference we need. > What I dislike about this variant is that in a normal try statement, > the exception goes after the "except" keyword. Here, it's the > alternative value that goes there. I find that very easy to misread. You are right! d From ram at rachum.com Thu Feb 13 12:54:04 2014 From: ram at rachum.com (Ram Rachum) Date: Thu, 13 Feb 2014 13:54:04 +0200 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: On Thu, Feb 13, 2014 at 1:50 PM, Chris Angelico wrote: > On Thu, Feb 13, 2014 at 10:36 PM, Paul Moore wrote: > > Personally, I quite like "EXPR except EXCEPTIONTYPE return VALUE" if > > we have to stick with existing keywords. If I had free rein I might go > > for a new keyword "then" instead of "return". > > Hmm. Stupid idea: > > EXPR expr EXCEPTIONTYPE pass VALUE > > Passing something is kinda like returning it, right? *ducks the rotten > tomatoes* > http://i.imgur.com/xk003.gif > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > -- > > --- > You received this message because you are subscribed to a topic in the > Google Groups "python-ideas" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/python-ideas/ZoBGdwuH3uk/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > python-ideas+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Thu Feb 13 12:55:28 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 13 Feb 2014 21:55:28 +1000 Subject: [Python-ideas] a in x or in y In-Reply-To: <20140213105716.GA3799@ando> References: <20140213105716.GA3799@ando> Message-ID: On 13 February 2014 20:57, Steven D'Aprano wrote: > - given the restrictions on the parser, is this even possible? and I haven't worked through an actual candidate grammar (so I could potentially be wrong) but I'm pretty sure it requires more lookahead than Guido is willing to allow for the language definition. At the point we hit: X in Y and we're going to want to process "X in Y" as an expression, and then start looking at compiling the RHS of the "and". This proposal would mean that when the "in" shows up after the "and", we need to backtrack, lift "X" out as its own expression, and then reinsert it as the LHS of *multiple* containment tests. It's worth noting that this deliberate "no more than one token lookahead" design constraint in the language syntax isn't just for the benefit of compiler writers: it's there for the benefit of human readers as well. In linguistics, there's a concept called "garden path" sentences - these are sentences where the first part is a grammatically coherent sentence, but by *adding more words to the end*, you change the meaning of words that appeared earlier. This is a jarring experience for readers. This is one of the classic examples: The horse raced past the barn fell. That's a grammatical English sentence. If you're yelling at your computer telling me that there's no way something that awkward can be grammatically correct, you're not alone in feeling that way, but the awkwardness isn't due to bad grammar. The reason it feels bad, is that the first six words form a sentence in their own right: The horse raced past the barn. This is the sentence our brains typically start constructing as we read the seven word version, but then we get to the extra word "fell", and the original grammar structure falls apart - the previous sentence was already complete, and has no room for the extra word. So our brain has to back track and come up with this alternate parsing: The horse [that was] raced past the barn fell. The extra word at the end reaches back to change the entire structure of the sentence, and effectively changing the meaning of "raced" as well. The "only one token lookahead" rule in Python's syntax design helps to make it more difficult to write "garden path expressions" where information presented late in the expression forces you to go back and reevaluate information presented earlier in the expression. (You can still write them - there'll just be some marker earlier on in the expression to suggest that trickery may be afoot). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From rosuav at gmail.com Thu Feb 13 13:00:02 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 13 Feb 2014 23:00:02 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: On Thu, Feb 13, 2014 at 10:54 PM, Ram Rachum wrote: > On Thu, Feb 13, 2014 at 1:50 PM, Chris Angelico wrote: >> >> On Thu, Feb 13, 2014 at 10:36 PM, Paul Moore wrote: >> > Personally, I quite like "EXPR except EXCEPTIONTYPE return VALUE" if >> > we have to stick with existing keywords. If I had free rein I might go >> > for a new keyword "then" instead of "return". >> >> Hmm. Stupid idea: >> >> EXPR expr EXCEPTIONTYPE pass VALUE >> >> Passing something is kinda like returning it, right? *ducks the rotten >> tomatoes* > > http://i.imgur.com/xk003.gif Do you mean my idea is win, or the notion of throwing rotten tomatoes at me is win? I'm inclined to the latter theory :) Oh, and that should be "EXPR except EXCEPTIONTYPE", of course. I wasn't sure if I'd clicked Send or not, edited, and reclicked Send, so you may have a corrected copy as well as that one. But you knew already what it ought to have been. ChrisA From denis.spir at gmail.com Thu Feb 13 13:07:51 2014 From: denis.spir at gmail.com (spir) Date: Thu, 13 Feb 2014 13:07:51 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: <52FCB597.8070507@gmail.com> On 02/13/2014 12:48 PM, Chris Angelico wrote: On Thu, Feb 13, 2014 at 10:25 PM, spir wrote: > > [By the way, this shows that: > > x = b if cond else a > > should really be: > > x = a else b if cond > > The difference being that the standard case is expressed first, the > > exceptional one being then introduced as an special variant.] > > My example tends to agree with you... but my example is using "if" to > introduce the abnormal case, whereas it's common to spell a block if > the other way: > > if normal-case: > code code code > else: > abnormal code I rather write code the the other way round, with the condition determinig the special case. This also matches the common idom: if special-case: deal-with-it return [result] deal-with-normal-case # no else needed Generally, I think logical vars should be 'false' by default, expressing the standard/rest/off state, with 'true' meaning something new or activated ('on'), or otherwise special. > C's ternary operator puts the expressions in the order "condition, > if_true, if_false". Python's puts them "if_true, condition, if_false". > You're proposing "if_false, if_true, condition". That's because I think (1) the standard value should come first (2) the condition should express the special case instead. > We're half way to > covering all the permutations! ;-) Actually, the ideal order for me would be: if_false, condition, if_true in the sense of normal_value, special_condition, special_value but this does not match the meaning of natural language preposition (here, 'if'). d From denis.spir at gmail.com Thu Feb 13 13:11:23 2014 From: denis.spir at gmail.com (spir) Date: Thu, 13 Feb 2014 13:11:23 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: <52FCB66B.9010801@gmail.com> On 02/13/2014 01:00 PM, Chris Angelico wrote: > On Thu, Feb 13, 2014 at 10:54 PM, Ram Rachum wrote: >> On Thu, Feb 13, 2014 at 1:50 PM, Chris Angelico wrote: >>> >>> On Thu, Feb 13, 2014 at 10:36 PM, Paul Moore wrote: >>>> Personally, I quite like "EXPR except EXCEPTIONTYPE return VALUE" if >>>> we have to stick with existing keywords. If I had free rein I might go >>>> for a new keyword "then" instead of "return". >>> >>> Hmm. Stupid idea: >>> >>> EXPR expr EXCEPTIONTYPE pass VALUE >>> >>> Passing something is kinda like returning it, right? *ducks the rotten >>> tomatoes* >> >> http://i.imgur.com/xk003.gif > > Do you mean my idea is win, or the notion of throwing rotten tomatoes > at me is win? I'm inclined to the latter theory :) > > Oh, and that should be "EXPR except EXCEPTIONTYPE", of course. I > wasn't sure if I'd clicked Send or not, edited, and reclicked Send, so > you may have a corrected copy as well as that one. But you knew > already what it ought to have been. Your proposal has the big advantage of stating things in (the right) order: normal_value, special_condition, special_value and it still lets the door open to naming the exception, using 'as', if later needed: EXPR except EXCEPTIONTYPE as EXCEPTIONNAME pass VALUE d From ncoghlan at gmail.com Thu Feb 13 13:50:55 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 13 Feb 2014 22:50:55 +1000 Subject: [Python-ideas] except expression In-Reply-To: <52FC9A20.9040303@egenix.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FC9A20.9040303@egenix.com> Message-ID: On 13 February 2014 20:10, M.-A. Lemburg wrote: > On 13.02.2014 10:24, Nick Coghlan wrote: >> General comment: like Raymond, I'm inclined to favour a nice expression >> friendly exception handling syntax, precisely because of the proliferation >> of relatively ad hoc alternative solutions (in particular, the popularity >> of being able to pass in default values to handle empty iterables). > > Here's a variant the resembles the code you'd write in a helper > function to achieve the same thing, only stripped down somewhat: > > x = something() except ValueError return default_value > > def try_something(): > try: > return something() > except ValueError: > return default_value > > x = something() except ValueError as exc return exc.message > > def try_something(): > try: > return something() > except ValueError as exc > return exc.message > > Obviously, having a keyword "use" would make a better fit :-) > > x = something() except ValueError use default_value Even if we don't agree on a resolution for 3.5, I think there's more than enough interest for it to be worth someone's while to collate some of the proposals in a PEP - if nothing else, it will save rehashing the whole discussion next time it comes up :) The benefits of this: - the status quo is that various APIs are growing "default" parameters to handle the case where they would otherwise throw an exception - this is creating inconsistencies, as some such functions can be used easily as expressions without risking any exception (those where such a parameter has been added), as well as a temptation to use "Look Before You Leap" pre-checks, even in cases where exception handling would be a better choice - sequence indexing is a case where there is no current standard mechanism for providing a default value, so you're either use a pre-check for the system length, or else using a full try statement or context manager to handle the IndexError - by providing a clean, expression level syntax for handling a single except clause and providing an alternate value for the expression, this problem could be solved once and for all in a systematic way, rather than needing to incrementally change the API of a variety of functions (as well as addressing the container subscripting case in a way that doesn't require switching away from using the subscript syntax to a normal function call, or switching from use an expression to a statement) Some of the specific syntactic proposals: x = op() except default if Exception x = op() except default for Exception x = op() except default from Exception x = op() except Exception return default x = op() except exc.attr if Exception as exc x = op() except exc.attr for Exception as exc x = op() except exc.attr from Exception as exc x = op() except Exception as exc return exc.attr The except/if construct has parser ambiguity issues. While they're potentially solvable by requiring parens around conditional expressions in that context, that would come at the cost of a significant redesign of the language grammar. The except/for option reads quite nicely, but introduces a substantially different meaning for "for". The except/from option reads OK when combined with "as" and actually using the caught exception, but otherwise reads strangely. The except/return option looks like it should either introduce a new scope or else return from the current function. The presence of the "as exc" clause in all variants actually suggests a new scope could be a good idea, given past experience with iteration variables in list comprehensions. So, if we take the point of view that the new syntax is almost *literally* a shorthand for: def _helper(op, exc, make_default): try: return op() except exc: return make_default() x = _helper(op, Exception, make_default) Then that would suggest the following syntax and interpretation: op() except Exception pass def _work_or_return_None(): try: return op() except Exception: pass _work_or_return_None() x = op() except Exception return value def _work_or_return_default(): try: return op() except Exception: return value x = _work_or_return_default() x = op() except Exception as exc return exc.attr def _work_or_process_exception(): try: return op() except Exception as exc: return exc.attr x = _work_or_process_exception() OK, with the "introduces a new scope" caveat, consider me a fan of MAL's syntax unless/until someone points out a potential flaw that I have missed. A possible addition: allow "raise" in addition to "pass" and "return" (for exception transformation as an expression) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From tjreedy at udel.edu Thu Feb 13 14:16:12 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 13 Feb 2014 08:16:12 -0500 Subject: [Python-ideas] on colon ':', and some more In-Reply-To: <52FC9CDD.8010003@gmail.com> References: <52FC9CDD.8010003@gmail.com> Message-ID: Terminating a header says "I believe I have entered a syntactically correct header" The receiving system can use this to immediately do a header syntax check that would be invalid if performed too soon. The console and Idle Shell interactive interpreters console do this. Idle version: (note the English use of ':' to end a header ;-) >>> a = 3 >>> if a == ( 3): pass >>> if a == (: SyntaxError: invalid syntax Deleting or making the : optional is not compatibly with implicit line continuation. Explicit \ would have to be the only way to continue a logical header line on another physical line. ----- = versus ==: Info theory and common sense say that the more frequent indication should generally be given the shorter indication. Assignment is much more common that equality comparison. Math gets away with overloading = partly because 'binding' is a declaration of equality, partly because logic and arithmetic are usually kept separate, and partly because logical equivalence is indicated with a 3-line character when needed. Since that is not an ascii character, we use a 2 character digraph as replacement. -- Terry Jan Reedy From flying-sheep at web.de Thu Feb 13 14:19:10 2014 From: flying-sheep at web.de (Philipp A.) Date: Thu, 13 Feb 2014 14:19:10 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FC9A20.9040303@egenix.com> Message-ID: actually i like the first proposal, because it mirrors ?the trinary?: ```python 'spam' if mustard else 'eggs' eat() except SickException else puke() ``` but it?s a bit weird. I don?t like `return` at all, because it?s exclusively about returning from functions. `for` is for loops. And qhile from is used more versatile, it?s even weirder than `else` here. I also don?t like putting anythiing but the exception type after the `except`. It should definitely be `EXPR except EXC_TYPE [as EXC_NAME] KEYWORD ALT_EXPR`, with `EXC_NAME` being defined in `ALT_EXPR` and afterwards. and i?m leaning towards `pass` or `else` as `KEYWORD`. `pass` is only used to mean ?do nothing? right now, and indeed that?s fitting: it?s just a border between `EXC_TYPE [as EXC_NAME]` and `ALT_EXPR`, and its other meaning makes sense in english grammar. Maybe even `try`? Like ?use this, except if that doesn?t work, then try using the other thing?. 2014-02-13 13:50 GMT+01:00 Nick Coghlan : > On 13 February 2014 20:10, M.-A. Lemburg wrote: > > On 13.02.2014 10:24, Nick Coghlan wrote: > >> General comment: like Raymond, I'm inclined to favour a nice expression > >> friendly exception handling syntax, precisely because of the > proliferation > >> of relatively ad hoc alternative solutions (in particular, the > popularity > >> of being able to pass in default values to handle empty iterables). > > > > Here's a variant the resembles the code you'd write in a helper > > function to achieve the same thing, only stripped down somewhat: > > > > x = something() except ValueError return default_value > > > > def try_something(): > > try: > > return something() > > except ValueError: > > return default_value > > > > x = something() except ValueError as exc return exc.message > > > > def try_something(): > > try: > > return something() > > except ValueError as exc > > return exc.message > > > > Obviously, having a keyword "use" would make a better fit :-) > > > > x = something() except ValueError use default_value > > Even if we don't agree on a resolution for 3.5, I think there's more > than enough interest for it to be worth someone's while to collate > some of the proposals in a PEP - if nothing else, it will save > rehashing the whole discussion next time it comes up :) > > The benefits of this: > > - the status quo is that various APIs are growing "default" parameters > to handle the case where they would otherwise throw an exception > - this is creating inconsistencies, as some such functions can be used > easily as expressions without risking any exception (those where such > a parameter has been added), as well as a temptation to use "Look > Before You Leap" pre-checks, even in cases where exception handling > would be a better choice > - sequence indexing is a case where there is no current standard > mechanism for providing a default value, so you're either use a > pre-check for the system length, or else using a full try statement or > context manager to handle the IndexError > - by providing a clean, expression level syntax for handling a single > except clause and providing an alternate value for the expression, > this problem could be solved once and for all in a systematic way, > rather than needing to incrementally change the API of a variety of > functions (as well as addressing the container subscripting case in a > way that doesn't require switching away from using the subscript > syntax to a normal function call, or switching from use an expression > to a statement) > > > Some of the specific syntactic proposals: > > x = op() except default if Exception > x = op() except default for Exception > x = op() except default from Exception > x = op() except Exception return default > > x = op() except exc.attr if Exception as exc > x = op() except exc.attr for Exception as exc > x = op() except exc.attr from Exception as exc > x = op() except Exception as exc return exc.attr > > The except/if construct has parser ambiguity issues. While they're > potentially solvable by requiring parens around conditional > expressions in that context, that would come at the cost of a > significant redesign of the language grammar. > > The except/for option reads quite nicely, but introduces a > substantially different meaning for "for". > > The except/from option reads OK when combined with "as" and actually > using the caught exception, but otherwise reads strangely. > > The except/return option looks like it should either introduce a new > scope or else return from the current function. The presence of the > "as exc" clause in all variants actually suggests a new scope could be > a good idea, given past experience with iteration variables in list > comprehensions. > > So, if we take the point of view that the new syntax is almost > *literally* a shorthand for: > > def _helper(op, exc, make_default): > try: > return op() > except exc: > return make_default() > > x = _helper(op, Exception, make_default) > > Then that would suggest the following syntax and interpretation: > > op() except Exception pass > > def _work_or_return_None(): > try: > return op() > except Exception: > pass > _work_or_return_None() > > x = op() except Exception return value > > def _work_or_return_default(): > try: > return op() > except Exception: > return value > x = _work_or_return_default() > > x = op() except Exception as exc return exc.attr > > def _work_or_process_exception(): > try: > return op() > except Exception as exc: > return exc.attr > x = _work_or_process_exception() > > OK, with the "introduces a new scope" caveat, consider me a fan of > MAL's syntax unless/until someone points out a potential flaw that I > have missed. > > A possible addition: allow "raise" in addition to "pass" and "return" > (for exception transformation as an expression) > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ram at rachum.com Thu Feb 13 14:26:01 2014 From: ram at rachum.com (Ram Rachum) Date: Thu, 13 Feb 2014 15:26:01 +0200 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FC9A20.9040303@egenix.com> Message-ID: My favorite syntax so far is: x except Exception pass y x except Exception as e pass e.y As weird as it is to use pass this way, it's better than using return. It's better than not having the exception immediately follow `except`, it's better than using a colon, it's better than all the other possible compromises we could make. On Thu, Feb 13, 2014 at 3:19 PM, Philipp A. wrote: > actually i like the first proposal, because it mirrors "the trinary": > > ```python > 'spam' if mustard else 'eggs' > eat() except SickException else puke() > ``` > > but it's a bit weird. > > I don't like `return` at all, because it's exclusively about returning > from functions. `for` is for loops. And qhile from is used more versatile, > it's even weirder than `else` here. > > I also don't like putting anythiing but the exception type after the > `except`. > > It should definitely be `EXPR except EXC_TYPE [as EXC_NAME] KEYWORD > ALT_EXPR`, with `EXC_NAME` being defined in `ALT_EXPR` and afterwards. > > and i'm leaning towards `pass` or `else` as `KEYWORD`. `pass` is only used > to mean "do nothing" right now, and indeed that's fitting: it's just a > border between `EXC_TYPE [as EXC_NAME]` and `ALT_EXPR`, and its other > meaning makes sense in english grammar. Maybe even `try`? Like "use this, > except if that doesn't work, then try using the other thing". > > > 2014-02-13 13:50 GMT+01:00 Nick Coghlan : > > On 13 February 2014 20:10, M.-A. Lemburg wrote: >> > On 13.02.2014 10:24, Nick Coghlan wrote: >> >> General comment: like Raymond, I'm inclined to favour a nice expression >> >> friendly exception handling syntax, precisely because of the >> proliferation >> >> of relatively ad hoc alternative solutions (in particular, the >> popularity >> >> of being able to pass in default values to handle empty iterables). >> > >> > Here's a variant the resembles the code you'd write in a helper >> > function to achieve the same thing, only stripped down somewhat: >> > >> > x = something() except ValueError return default_value >> > >> > def try_something(): >> > try: >> > return something() >> > except ValueError: >> > return default_value >> > >> > x = something() except ValueError as exc return exc.message >> > >> > def try_something(): >> > try: >> > return something() >> > except ValueError as exc >> > return exc.message >> > >> > Obviously, having a keyword "use" would make a better fit :-) >> > >> > x = something() except ValueError use default_value >> >> Even if we don't agree on a resolution for 3.5, I think there's more >> than enough interest for it to be worth someone's while to collate >> some of the proposals in a PEP - if nothing else, it will save >> rehashing the whole discussion next time it comes up :) >> >> The benefits of this: >> >> - the status quo is that various APIs are growing "default" parameters >> to handle the case where they would otherwise throw an exception >> - this is creating inconsistencies, as some such functions can be used >> easily as expressions without risking any exception (those where such >> a parameter has been added), as well as a temptation to use "Look >> Before You Leap" pre-checks, even in cases where exception handling >> would be a better choice >> - sequence indexing is a case where there is no current standard >> mechanism for providing a default value, so you're either use a >> pre-check for the system length, or else using a full try statement or >> context manager to handle the IndexError >> - by providing a clean, expression level syntax for handling a single >> except clause and providing an alternate value for the expression, >> this problem could be solved once and for all in a systematic way, >> rather than needing to incrementally change the API of a variety of >> functions (as well as addressing the container subscripting case in a >> way that doesn't require switching away from using the subscript >> syntax to a normal function call, or switching from use an expression >> to a statement) >> >> >> Some of the specific syntactic proposals: >> >> x = op() except default if Exception >> x = op() except default for Exception >> x = op() except default from Exception >> x = op() except Exception return default >> >> x = op() except exc.attr if Exception as exc >> x = op() except exc.attr for Exception as exc >> x = op() except exc.attr from Exception as exc >> x = op() except Exception as exc return exc.attr >> >> The except/if construct has parser ambiguity issues. While they're >> potentially solvable by requiring parens around conditional >> expressions in that context, that would come at the cost of a >> significant redesign of the language grammar. >> >> The except/for option reads quite nicely, but introduces a >> substantially different meaning for "for". >> >> The except/from option reads OK when combined with "as" and actually >> using the caught exception, but otherwise reads strangely. >> >> The except/return option looks like it should either introduce a new >> scope or else return from the current function. The presence of the >> "as exc" clause in all variants actually suggests a new scope could be >> a good idea, given past experience with iteration variables in list >> comprehensions. >> >> So, if we take the point of view that the new syntax is almost >> *literally* a shorthand for: >> >> def _helper(op, exc, make_default): >> try: >> return op() >> except exc: >> return make_default() >> >> x = _helper(op, Exception, make_default) >> >> Then that would suggest the following syntax and interpretation: >> >> op() except Exception pass >> >> def _work_or_return_None(): >> try: >> return op() >> except Exception: >> pass >> _work_or_return_None() >> >> x = op() except Exception return value >> >> def _work_or_return_default(): >> try: >> return op() >> except Exception: >> return value >> x = _work_or_return_default() >> >> x = op() except Exception as exc return exc.attr >> >> def _work_or_process_exception(): >> try: >> return op() >> except Exception as exc: >> return exc.attr >> x = _work_or_process_exception() >> >> OK, with the "introduces a new scope" caveat, consider me a fan of >> MAL's syntax unless/until someone points out a potential flaw that I >> have missed. >> >> A possible addition: allow "raise" in addition to "pass" and "return" >> (for exception transformation as an expression) >> >> Cheers, >> Nick. >> >> -- >> Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- > > --- > You received this message because you are subscribed to a topic in the > Google Groups "python-ideas" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/python-ideas/ZoBGdwuH3uk/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > python-ideas+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rob.cliffe at btinternet.com Thu Feb 13 14:26:57 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Thu, 13 Feb 2014 13:26:57 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FC9A20.9040303@egenix.com> Message-ID: <52FCC821.4020008@btinternet.com> On 13/02/2014 12:50, Nick Coghlan wrote: > On 13 February 2014 20:10, M.-A. Lemburg wrote: >> On 13.02.2014 10:24, Nick Coghlan wrote: >>> General comment: like Raymond, I'm inclined to favour a nice expression >>> friendly exception handling syntax, precisely because of the proliferation >>> of relatively ad hoc alternative solutions (in particular, the popularity >>> of being able to pass in default values to handle empty iterables). >> Here's a variant the resembles the code you'd write in a helper >> function to achieve the same thing, only stripped down somewhat: >> >> x = something() except ValueError return default_value >> >> def try_something(): >> try: >> return something() >> except ValueError: >> return default_value >> >> x = something() except ValueError as exc return exc.message >> >> def try_something(): >> try: >> return something() >> except ValueError as exc >> return exc.message >> >> Obviously, having a keyword "use" would make a better fit :-) >> >> x = something() except ValueError use default_value > Even if we don't agree on a resolution for 3.5, I think there's more > than enough interest for it to be worth someone's while to collate > some of the proposals in a PEP - if nothing else, it will save > rehashing the whole discussion next time it comes up :) > > The benefits of this: > > - the status quo is that various APIs are growing "default" parameters > to handle the case where they would otherwise throw an exception > - this is creating inconsistencies, as some such functions can be used > easily as expressions without risking any exception (those where such > a parameter has been added), as well as a temptation to use "Look > Before You Leap" pre-checks, even in cases where exception handling > would be a better choice > - sequence indexing is a case where there is no current standard > mechanism for providing a default value, so you're either use a > pre-check for the system length, or else using a full try statement or > context manager to handle the IndexError > - by providing a clean, expression level syntax for handling a single > except clause and providing an alternate value for the expression, > this problem could be solved once and for all in a systematic way, > rather than needing to incrementally change the API of a variety of > functions (as well as addressing the container subscripting case in a > way that doesn't require switching away from using the subscript > syntax to a normal function call, or switching from use an expression > to a statement) > > > Some of the specific syntactic proposals: > > x = op() except default if Exception > x = op() except default for Exception > x = op() except default from Exception > x = op() except Exception return default > > x = op() except exc.attr if Exception as exc > x = op() except exc.attr for Exception as exc > x = op() except exc.attr from Exception as exc > x = op() except Exception as exc return exc.attr > > The except/if construct has parser ambiguity issues. While they're > potentially solvable by requiring parens around conditional > expressions in that context, that would come at the cost of a > significant redesign of the language grammar. > > The except/for option reads quite nicely, but introduces a > substantially different meaning for "for". > > The except/from option reads OK when combined with "as" and actually > using the caught exception, but otherwise reads strangely. > > The except/return option looks like it should either introduce a new > scope or else return from the current function. The presence of the > "as exc" clause in all variants actually suggests a new scope could be > a good idea, given past experience with iteration variables in list > comprehensions. > > So, if we take the point of view that the new syntax is almost > *literally* a shorthand for: > > def _helper(op, exc, make_default): > try: > return op() > except exc: > return make_default() > > x = _helper(op, Exception, make_default) > > Then that would suggest the following syntax and interpretation: > > op() except Exception pass > > def _work_or_return_None(): > try: > return op() > except Exception: > pass > _work_or_return_None() > > x = op() except Exception return value > > def _work_or_return_default(): > try: > return op() > except Exception: > return value > x = _work_or_return_default() > > x = op() except Exception as exc return exc.attr > > def _work_or_process_exception(): > try: > return op() > except Exception as exc: > return exc.attr > x = _work_or_process_exception() > > OK, with the "introduces a new scope" caveat, consider me a fan of > MAL's syntax unless/until someone points out a potential flaw that I > have missed. > > A possible addition: allow "raise" in addition to "pass" and "return" > (for exception transformation as an expression) > > Cheers, > Nick. > It certainly feels right for the order to be normal value, exception, default value. So the syntax I would like is x = entries[0] except IndexError XXX None where XXX is some keyword. Ideally 'then' or perhaps 'when' which read better than 'else', but I understand adding a new keyword is a big deal. (FWIW I also wish trinary expressions were written as if condition then value-if-true else value-if-false which to me reads better than the status quo, but that ship has sailed.) Rob Cliffe From bborcic at gmail.com Thu Feb 13 14:28:23 2014 From: bborcic at gmail.com (Boris Borcic) Date: Thu, 13 Feb 2014 14:28:23 +0100 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> Message-ID: Nick Coghlan wrote: > On 13 February 2014 20:57, Steven D'Aprano wrote: >> - given the restrictions on the parser, is this even possible? and > > I haven't worked through an actual candidate grammar (so I could > potentially be wrong) but I'm pretty sure it requires more lookahead > than Guido is willing to allow for the language definition. > > At the point we hit: > > X in Y and > > we're going to want to process "X in Y" as an expression, and then > start looking at compiling the RHS of the "and". > > This proposal would mean that when the "in" shows up after the "and", > we need to backtrack, lift "X" out as its own expression, and then > reinsert it as the LHS of *multiple* containment tests. > > It's worth noting that this deliberate "no more than one token > lookahead" design constraint in the language syntax isn't just for the > benefit of compiler writers: it's there for the benefit of human > readers as well. > > In linguistics, there's a concept called "garden path" sentences - > these are sentences where the first part is a grammatically coherent > sentence, but by *adding more words to the end*, you change the > meaning of words that appeared earlier. Are you trying to say "X in Y and" forms a complete expression?? And couldn't the lexer recognize "and in" and cognates as single tokens anyway, like it apparently does presently for "is not" ? --- Ce courrier ?lectronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com From tjreedy at udel.edu Thu Feb 13 14:38:32 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 13 Feb 2014 08:38:32 -0500 Subject: [Python-ideas] except expression In-Reply-To: <52FC9A20.9040303@egenix.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FC9A20.9040303@egenix.com> Message-ID: On 2/13/2014 5:10 AM, M.-A. Lemburg wrote: > On 13.02.2014 10:24, Nick Coghlan wrote: >> General comment: like Raymond, I'm inclined to favour a nice expression >> friendly exception handling syntax, precisely because of the proliferation >> of relatively ad hoc alternative solutions (in particular, the popularity >> of being able to pass in default values to handle empty iterables). > > Here's a variant the resembles the code you'd write in a helper > function to achieve the same thing, only stripped down somewhat: > > x = something() except ValueError return default_value > > def try_something(): > try: > return something() > except ValueError: > return default_value > > x = something() except ValueError as exc return exc.message In Python 3 that would be x = something() except ValueError as exc return exc.args[0] > def try_something(): > try: > return something() > except ValueError as exc > return exc.message > > Obviously, having a keyword "use" would make a better fit :-) > > x = something() except ValueError use default_value x = something() except ValueError as exc try exc.message Try otherthing instead of something. That could even be chained x = something() except ValueError as exc continue with exc.message Instead of breaking due to the exception; 'with' could be left out. x = something() except ValueError as exc pass exc.message Pass expression back to x or consumer of expression. -- Terry Jan Reedy From ncoghlan at gmail.com Thu Feb 13 14:51:41 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 13 Feb 2014 23:51:41 +1000 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> Message-ID: On 13 Feb 2014 23:30, "Boris Borcic" wrote: > > Nick Coghlan wrote: > >> On 13 February 2014 20:57, Steven D'Aprano wrote: >>> >>> - given the restrictions on the parser, is this even possible? and >> >> >> I haven't worked through an actual candidate grammar (so I could >> potentially be wrong) but I'm pretty sure it requires more lookahead >> than Guido is willing to allow for the language definition. >> >> At the point we hit: >> >> X in Y and >> >> we're going to want to process "X in Y" as an expression, and then >> start looking at compiling the RHS of the "and". >> >> This proposal would mean that when the "in" shows up after the "and", >> we need to backtrack, lift "X" out as its own expression, and then >> reinsert it as the LHS of *multiple* containment tests. >> >> It's worth noting that this deliberate "no more than one token >> lookahead" design constraint in the language syntax isn't just for the >> benefit of compiler writers: it's there for the benefit of human >> readers as well. >> >> In linguistics, there's a concept called "garden path" sentences - >> these are sentences where the first part is a grammatically coherent >> sentence, but by *adding more words to the end*, you change the >> meaning of words that appeared earlier. > > > Are you trying to say "X in Y and" forms a complete expression?? And couldn't the lexer recognize "and in" and cognates as single tokens anyway, like it apparently does presently for "is not" ? Yes, there are ways around the technical limitation - the point of the rest of the post was to make it clear that was largely irrelevant, because it would still be too hard to read. Cheers, Nick. > > > --- > Ce courrier ?lectronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. > http://www.avast.com > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rurpy at yahoo.com Thu Feb 13 03:49:18 2014 From: rurpy at yahoo.com (rurpy at yahoo.com) Date: Wed, 12 Feb 2014 18:49:18 -0800 (PST) Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> Message-ID: <8c451f7d-8e12-4aad-b5d3-8eb06dacaf64@googlegroups.com> On Wednesday, February 12, 2014 3:00:20 PM UTC-7, Paul Moore wrote: > > On 12 February 2014 21:02, Ram Rachum > > wrote: > [...] > PS Looks like posting via google groups not only breaks filtering and > threading, it also breaks reply-all (as I can't post to the google > group). Could you please post direct to the group rather than through > the google groups interface? Thanks. > There is no need for you to explicitly reply to Google Groups -- it is gatewayed to the ideas list. Here is a link to your post on GG: https://groups.google.com/d/msg/python-ideas/ZoBGdwuH3uk/49PE5ufVHYAJ Many people find GG convenient and prefer to use it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Thu Feb 13 15:34:29 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 13 Feb 2014 14:34:29 +0000 Subject: [Python-ideas] except expression In-Reply-To: <8c451f7d-8e12-4aad-b5d3-8eb06dacaf64@googlegroups.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <8c451f7d-8e12-4aad-b5d3-8eb06dacaf64@googlegroups.com> Message-ID: But because the GG gateway isn't transparent I have to manually edit the headers when I reply-all and if I don't I get a rejection message. Never mind, I appreciate that you find it convenient, I was just pointing out that it inconveniences others (as has been pointed out a few times in the past on this list) Paul On 13 February 2014 02:49, rurpy at yahoo.com wrote: > On Wednesday, February 12, 2014 3:00:20 PM UTC-7, Paul Moore wrote: >> >> On 12 February 2014 21:02, Ram Rachum wrote: >> [...] >> >> PS Looks like posting via google groups not only breaks filtering and >> threading, it also breaks reply-all (as I can't post to the google >> group). Could you please post direct to the group rather than through >> the google groups interface? Thanks. > > > There is no need for you to explicitly reply to Google Groups -- it is > gatewayed to the ideas list. Here is a link to your post on GG: > https://groups.google.com/d/msg/python-ideas/ZoBGdwuH3uk/49PE5ufVHYAJ > Many people find GG convenient and prefer to use it. > > > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From bborcic at gmail.com Thu Feb 13 15:38:31 2014 From: bborcic at gmail.com (Boris Borcic) Date: Thu, 13 Feb 2014 15:38:31 +0100 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> Message-ID: Nick Coghlan wrote: > > >> In linguistics, there's a concept called "garden path" sentences - > >> these are sentences where the first part is a grammatically coherent > >> sentence, but by *adding more words to the end*, you change the > >> meaning of words that appeared earlier. > > > > > > Are you trying to say "X in Y and" forms a complete expression?? And couldn't the lexer recognize "and in" > and cognates as single tokens anyway, like it apparently does presently for "is not" ? > > Yes, there are ways around the technical limitation - the point of the rest of the post was to make it clear > that was largely irrelevant, because it would still be too hard to read. > And my point was that your argument was rather unconvincing, for a variety of reasons going from the difference between indefinite lookahead and (rectifiable) two token lookahead, to the rarity of actual garden path sentences that's evidenced by the fact that the single example you cite is many decades old. Cheers, Boris Borcic --- Ce courrier ?lectronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com From g.brandl at gmx.net Thu Feb 13 15:53:48 2014 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 13 Feb 2014 15:53:48 +0100 Subject: [Python-ideas] except expression In-Reply-To: <52FCAB93.40504@gmail.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: Am 13.02.2014 12:25, schrieb spir: > [By the way, this shows that: > x = b if cond else a > should really be: > x = a else b if cond > The difference being that the standard case is expressed first, the exceptional > one being then introduced as an special variant.] No it shouldn't -- your syntax associates "b" with both the else ("else b") and the "if" clause ("b if cond"), which makes me think "which of them will it be". Also, there's nothing intrinsically more "exceptional" about the "if" branch than the "else" branch; this entirely dependent on the expression. Georg From g.brandl at gmx.net Thu Feb 13 15:58:06 2014 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 13 Feb 2014 15:58:06 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FC9A20.9040303@egenix.com> Message-ID: Am 13.02.2014 13:50, schrieb Nick Coghlan: > On 13 February 2014 20:10, M.-A. Lemburg wrote: >> On 13.02.2014 10:24, Nick Coghlan wrote: >>> General comment: like Raymond, I'm inclined to favour a nice expression >>> friendly exception handling syntax, precisely because of the proliferation >>> of relatively ad hoc alternative solutions (in particular, the popularity >>> of being able to pass in default values to handle empty iterables). >> >> Here's a variant the resembles the code you'd write in a helper >> function to achieve the same thing, only stripped down somewhat: >> >> x = something() except ValueError return default_value >> >> def try_something(): >> try: >> return something() >> except ValueError: >> return default_value >> >> x = something() except ValueError as exc return exc.message >> >> def try_something(): >> try: >> return something() >> except ValueError as exc >> return exc.message >> >> Obviously, having a keyword "use" would make a better fit :-) >> >> x = something() except ValueError use default_value > > Even if we don't agree on a resolution for 3.5, I think there's more > than enough interest for it to be worth someone's while to collate > some of the proposals in a PEP - if nothing else, it will save > rehashing the whole discussion next time it comes up :) > > The benefits of this: > > - the status quo is that various APIs are growing "default" parameters > to handle the case where they would otherwise throw an exception > - this is creating inconsistencies, as some such functions can be used > easily as expressions without risking any exception (those where such > a parameter has been added), as well as a temptation to use "Look > Before You Leap" pre-checks, even in cases where exception handling > would be a better choice > - sequence indexing is a case where there is no current standard > mechanism for providing a default value, so you're either use a > pre-check for the system length, or else using a full try statement or > context manager to handle the IndexError > - by providing a clean, expression level syntax for handling a single > except clause and providing an alternate value for the expression, > this problem could be solved once and for all in a systematic way, > rather than needing to incrementally change the API of a variety of > functions (as well as addressing the container subscripting case in a > way that doesn't require switching away from using the subscript > syntax to a normal function call, or switching from use an expression > to a statement) > > > Some of the specific syntactic proposals: > > x = op() except default if Exception > x = op() except default for Exception > x = op() except default from Exception > x = op() except Exception return default > > x = op() except exc.attr if Exception as exc > x = op() except exc.attr for Exception as exc > x = op() except exc.attr from Exception as exc > x = op() except Exception as exc return exc.attr For me any proposal that doesn't pair "except" with the exception class(es) like the statement version does would be right out. It will be hard enough to remember the order of the expression, but "in the try-except block except gets the exception class, in the except expression it gets the default value and the exception class is specified with FOO" is silly. Georg From storchaka at gmail.com Thu Feb 13 16:04:27 2014 From: storchaka at gmail.com (Serhiy Storchaka) Date: Thu, 13 Feb 2014 17:04:27 +0200 Subject: [Python-ideas] a in x or in y In-Reply-To: <20140213105716.GA3799@ando> References: <20140213105716.GA3799@ando> Message-ID: 13.02.14 12:57, Steven D'Aprano ???????(??): > I like it. In natural language, people often say things like: > > my keys are in the car or in my pocket > > which fools them into writing: > > keys in car or pocket > > which does the wrong thing. Chained "in" comparisons is a natural > extension to Python's already natural language-like syntax. Python > already has other chained comparisons. Being able to write: > > keys in car or in pocket > > feels natural and right to me. (We can't *quite* match the human idiom > where the second "in" is left out, but one can't have everything.) You forgot keyword "are". keys are in car or in pocket From tjreedy at udel.edu Thu Feb 13 16:28:05 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 13 Feb 2014 10:28:05 -0500 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: On 2/13/2014 4:24 AM, Nick Coghlan wrote: > General comment: like Raymond, I'm inclined to favour a nice expression > friendly exception handling syntax, precisely because of the > proliferation of relatively ad hoc alternative solutions (in particular, > the popularity of being able to pass in default values to handle empty > iterables). Leaving aside syntax, the idea makes sense to me as follows: In Python, 'a or b' is not a just a logic operator but is generalized to mean, for instance, the following: 'a, unless a is falsey, in which case, b instead. The proposal is to introduce a syntax that means the same with 'unless a is falsey' replaced by 'unless a raises an exception of class C'. The class 'C' make 'a new_op b' inadequate. The two alternations are related when exception C results from an input that is at least conceptually falsey. Some such cases can be handled by 'or' (see below). Iterators, however, are seen as truthy even when empty (conceptually falsey). And that cannot be fixed since some iterators cannot know if they are empty until __next__ is called and bool is not supposed to raise. > One use case, for example, is handing IndexError when retrieving an item > from a sequence (which currently has no nice standard spelling, and > isn't amenable to the "pass in a default answer" solution because it > isn't a normal function call). This is easy, if not obvious >>> (seq or ['default'])[0] 'default' (Python's precedence rules make the parentheses optional). Doing something similar instead of dict.get is messier but conceptually the same. >>> {}.get('key', 'default') 'default' >>> {} or {'key':'default'}['key'] 'default' The general scheme is f(a or b, *args), where b is the input to f that gives the default as the output. Sort of inverse(partial(f, args))(default). This does not work if the exception-raising inputs are not all seen as falsey (as with empty iterables) or if one cannot invert the partial function to get b. In either case, we have to input a and replace exceptions with b. -- Terry Jan Reedy From g.brandl at gmx.net Thu Feb 13 16:36:43 2014 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 13 Feb 2014 16:36:43 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: Am 13.02.2014 16:28, schrieb Terry Reedy: > On 2/13/2014 4:24 AM, Nick Coghlan wrote: >> General comment: like Raymond, I'm inclined to favour a nice expression >> friendly exception handling syntax, precisely because of the >> proliferation of relatively ad hoc alternative solutions (in particular, >> the popularity of being able to pass in default values to handle empty >> iterables). > > Leaving aside syntax, the idea makes sense to me as follows: > > In Python, 'a or b' is not a just a logic operator but is generalized to > mean, for instance, the following: 'a, unless a is falsey, in which > case, b instead. The proposal is to introduce a syntax that means the > same with 'unless a is falsey' replaced by 'unless a raises an exception > of class C'. The class 'C' make 'a new_op b' inadequate. > > The two alternations are related when exception C results from an input > that is at least conceptually falsey. Some such cases can be handled by > 'or' (see below). Iterators, however, are seen as truthy even when empty > (conceptually falsey). And that cannot be fixed since some iterators > cannot know if they are empty until __next__ is called and bool is not > supposed to raise. > >> One use case, for example, is handing IndexError when retrieving an item >> from a sequence (which currently has no nice standard spelling, and >> isn't amenable to the "pass in a default answer" solution because it >> isn't a normal function call). > > This is easy, if not obvious > > >>> (seq or ['default'])[0] > 'default' > > (Python's precedence rules make the parentheses optional). Not sure if I understand correctly, but in the example above they are not. > Doing something similar instead of dict.get is messier but conceptually > the same. > > >>> {}.get('key', 'default') > 'default' > >>> {} or {'key':'default'}['key'] > 'default' And this also quite certainly does not do what you intended. Georg From elazarg at gmail.com Thu Feb 13 17:11:17 2014 From: elazarg at gmail.com (=?UTF-8?B?15DXnNei15bXqA==?=) Date: Thu, 13 Feb 2014 18:11:17 +0200 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> Message-ID: 2014-02-13 17:04 GMT+02:00 Serhiy Storchaka : > > 13.02.14 12:57, Steven D'Aprano ???????(??): > >> I like it. In natural language, people often say things like: >> >> my keys are in the car or in my pocket >> >> which fools them into writing: >> >> keys in car or pocket >> >> which does the wrong thing. Chained "in" comparisons is a natural >> extension to Python's already natural language-like syntax. Python >> already has other chained comparisons. Being able to write: >> >> keys in car or in pocket >> >> feels natural and right to me. (We can't *quite* match the human idiom >> where the second "in" is left out, but one can't have everything.) > > > You forgot keyword "are". > > keys are in car or in pocket > > And what about keys are in car or in pocket and not in purse it will be too complicated. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Thu Feb 13 17:18:37 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 13 Feb 2014 11:18:37 -0500 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: On 2/13/2014 10:28 AM, Terry Reedy wrote: > On 2/13/2014 4:24 AM, Nick Coghlan wrote: >> General comment: like Raymond, I'm inclined to favour a nice expression >> friendly exception handling syntax, precisely because of the >> proliferation of relatively ad hoc alternative solutions (in particular, >> the popularity of being able to pass in default values to handle empty >> iterables). > > Leaving aside syntax, the idea makes sense to me as follows: > > In Python, 'a or b' is not a just a logic operator but is generalized to > mean, for instance, the following: 'a, unless a is falsey, in which > case, b instead. The proposal is to introduce a syntax that means the > same with 'unless a is falsey' replaced by 'unless a raises an exception > of class C'. The class 'C' make 'a new_op b' inadequate. > > The two alternations are related when exception C results from an input > that is at least conceptually falsey. Some such cases can be handled by > 'or' (see below). Iterators, however, are seen as truthy even when empty > (conceptually falsey). And that cannot be fixed since some iterators > cannot know if they are empty until __next__ is called and bool is not > supposed to raise. > >> One use case, for example, is handing IndexError when retrieving an item >> from a sequence (which currently has no nice standard spelling, and >> isn't amenable to the "pass in a default answer" solution because it >> isn't a normal function call). > > This is easy, if not obvious To be fair, this is only easy for the index 0 case, which is however, common. > >>> (seq or ['default'])[0] > 'default' > > (Python's precedence rules make the parentheses optional). > > Doing something similar instead of dict.get is messier but conceptually > the same. > >>> {}.get('key', 'default') > 'default' > >>> {} or {'key':'default'}['key'] > 'default' However, this is a rare case. > The general scheme is f(a or b, *args), where b is the input to f that > gives the default as the output. Sort of inverse(partial(f, > args))(default). > > This does not work if the exception-raising inputs are not all seen as > falsey (as with empty iterables) or with non-empty lists whose length is less than the index+1 (though this latter example can be 'fixed' by using a conditional expression). > or if one cannot invert the partial function to get b. Or if it is unboundedly expensive. >>> (lis if len(lis) > n else (n+1)*['default'])[n] # () needed 'default' Doing something similar for a non-empty dict that might be missing a key is at least as messy. >In either case, we should or must input a and replace exceptions with b. -- Terry Jan Reedy From python at mrabarnett.plus.com Thu Feb 13 17:38:36 2014 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 13 Feb 2014 16:38:36 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FC9A20.9040303@egenix.com> Message-ID: <52FCF50C.9030408@mrabarnett.plus.com> On 2014-02-13 12:50, Nick Coghlan wrote: > On 13 February 2014 20:10, M.-A. Lemburg wrote: >> On 13.02.2014 10:24, Nick Coghlan wrote: >>> General comment: like Raymond, I'm inclined to favour a nice expression >>> friendly exception handling syntax, precisely because of the proliferation >>> of relatively ad hoc alternative solutions (in particular, the popularity >>> of being able to pass in default values to handle empty iterables). >> >> Here's a variant the resembles the code you'd write in a helper >> function to achieve the same thing, only stripped down somewhat: >> >> x = something() except ValueError return default_value >> >> def try_something(): >> try: >> return something() >> except ValueError: >> return default_value >> >> x = something() except ValueError as exc return exc.message >> >> def try_something(): >> try: >> return something() >> except ValueError as exc >> return exc.message >> >> Obviously, having a keyword "use" would make a better fit :-) >> >> x = something() except ValueError use default_value > > Even if we don't agree on a resolution for 3.5, I think there's more > than enough interest for it to be worth someone's while to collate > some of the proposals in a PEP - if nothing else, it will save > rehashing the whole discussion next time it comes up :) > > The benefits of this: > > - the status quo is that various APIs are growing "default" parameters > to handle the case where they would otherwise throw an exception > - this is creating inconsistencies, as some such functions can be used > easily as expressions without risking any exception (those where such > a parameter has been added), as well as a temptation to use "Look > Before You Leap" pre-checks, even in cases where exception handling > would be a better choice > - sequence indexing is a case where there is no current standard > mechanism for providing a default value, so you're either use a > pre-check for the system length, or else using a full try statement or > context manager to handle the IndexError > - by providing a clean, expression level syntax for handling a single > except clause and providing an alternate value for the expression, > this problem could be solved once and for all in a systematic way, > rather than needing to incrementally change the API of a variety of > functions (as well as addressing the container subscripting case in a > way that doesn't require switching away from using the subscript > syntax to a normal function call, or switching from use an expression > to a statement) > > > Some of the specific syntactic proposals: > > x = op() except default if Exception > x = op() except default for Exception > x = op() except default from Exception > x = op() except Exception return default > > x = op() except exc.attr if Exception as exc > x = op() except exc.attr for Exception as exc > x = op() except exc.attr from Exception as exc > x = op() except Exception as exc return exc.attr > You left one out: x = op() except Exception: default From python at mrabarnett.plus.com Thu Feb 13 17:52:31 2014 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 13 Feb 2014 16:52:31 +0000 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> Message-ID: <52FCF84F.9010801@mrabarnett.plus.com> On 2014-02-13 11:27, Chris Angelico wrote: > On Thu, Feb 13, 2014 at 9:57 PM, Steven D'Aprano wrote: >> - given the restrictions on the parser, is this even possible? and > > Generalizing the syntax, I'd see this as: > > operand1 binary-op1 operand2 {and|or} binary-op2 operand3 > > which implicitly places the value (not the code) of operand1 between > and/or and binary-op2. Since all binary operators have higher > precedence than either and or or (all that's lower is lambda and > if/else), this notation is currently guaranteed to fail... except in > two cases, namely + and -, which exist in unary form as well. So if + > and - are excluded, it should be unambiguous. Whether or not the > parser can actually handle it is a question for someone who knows what > he's talking about, though :) > > The way I see it, there should ideally be no syntactic rule against > using different operators on the two sides: > > input("> ") in legalcommands and not in forbiddencommands > value > 20 or in {3,5,7,11} > > even though it would allow insanity: > > if 5 < int(input("Enter a number: ")) or < int(input("Greater than > five please: ")) or < int(input("Come on now! ")) or == print("Bah, I > give up."): > print("Thank you.") > > In this way, it's like chained comparisons: > > 1 < x <= 10 > > which don't mind mixing and matching. > I think it would require 3-token lookahead (there would be new binary operators such "and in" and "and not in"). It's certainly achievable. From elazarg at gmail.com Thu Feb 13 18:04:06 2014 From: elazarg at gmail.com (=?UTF-8?B?15DXnNei15bXqA==?=) Date: Thu, 13 Feb 2014 19:04:06 +0200 Subject: [Python-ideas] a in x or in y In-Reply-To: <52FCF84F.9010801@mrabarnett.plus.com> References: <20140213105716.GA3799@ando> <52FCF84F.9010801@mrabarnett.plus.com> Message-ID: Another issue: people will expect for i in a and not in b: print(i) to work. Elazar -------------- next part -------------- An HTML attachment was scrubbed... URL: From denis.spir at gmail.com Thu Feb 13 19:30:08 2014 From: denis.spir at gmail.com (spir) Date: Thu, 13 Feb 2014 19:30:08 +0100 Subject: [Python-ideas] except expression In-Reply-To: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> Message-ID: <52FD0F30.4080101@gmail.com> On 02/12/2014 10:02 PM, Ram Rachum wrote: > Hi, > > Here's an idea that would help shortening code. Allow a ternary expression > based on except, like so: > > first_entry = entries[0] except IndexError else None > item = my_queue.get() except queue.Empty else None > response_text = request('http://whatever.com').text except HttpError > else "Can't access data" > > Aside from the fact that this would be a big grammar addition, a big > problem here is the usage of the `else` keyword, that when used with except > usually means "what would happen if there wasn't an exception" and here > means the opposite. But I couldn't think of a nicer syntax. > > I realize that this is a big change and that most people would be opposed > to this... But I guess I just wanted to share my idea :) After some considerations, it seems we reached the point of generalising the idea to: * provide a builtin way to indicate a special value for the special cases where the standard value's expression would raise an exception As Nick Coghlan shows in another post, this is in fact close to a builtin way to deal with potentially failing functions, in general (below 'op'): def _helper(op, exc, make_default): try: return op() except exc: return make_default() x = _helper(op, Exception, make_default) However, this only applies to the case of functions properly speaking (which purpose is computing a product). What about 'actions', meaning procedures that perform an effect? Take the case of count-words for instance, where (in python) when encoutering a word one would add 1 to its count in a dict which keys are the words. On first encounter, there no entry yet for that word. We could check "word in word_counts", but this is doing the dict lookup twice; and catching an exception is worse. The problem I guess, here and also similarly for functions, is that there is no way for the _client_ (the caller) to control exception throwing; while only the caller knows whether the failing case actually is an error or not, meaning belongs or not the application's logics. Maybe what we need is in fact something like: maybe statement [then dependant-block] else alternative-block This is similar in form to exception catching except (sic!) that the actual meaning is to _avoid_ an exception, not to catch it (to avoid it beeing thrown at all). More or less the opposite, in fact. The consequence is that in case of failure no exception is raised, instead the alternative 'else' branch is taken. Such a construct would work fine for both called actions and functions (which calls are expressions in statement). [1] [Similar considerations are evoked in other languages, especially D where I first met them. Generally speaking, people start to realise the seamntic weakness of typical exception catching mecanisms which let no choice or control in the hands of the one who actually knows, namely the client.] d [1] Practically, this just means adding a test before throwing, and setting a flag instead (the carry would do the job nicely, since it can have no meaning on func return, and is the only copy flag one can set arbitrarily). Thus, the above code translates to: statement if flag: reset flag alternative-block [else: dependant-block] From mertz at gnosis.cx Thu Feb 13 19:31:28 2014 From: mertz at gnosis.cx (David Mertz) Date: Thu, 13 Feb 2014 10:31:28 -0800 Subject: [Python-ideas] except expression In-Reply-To: <52FCAB93.40504@gmail.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: On Thu, Feb 13, 2014 at 3:25 AM, spir wrote: > Of course, it could be done as a function call: >> >> def catch(callme, catchme, returnme): >> try: >> return callme() >> except catchme: >> return returnme >> >> phone = catch(lambda: addressbook[name], KeyError, "Unknown") >> but that's *really* clunky. >> > I don't really find this function all the clunky. However, it also does not solve the case of an expensive 'except' clause that Nick pointed out as desirable to avoid evaluating (or, for that matter, an except clause with side effects). E.g. phone = catch(lambda: addressbook[name], KeyError, lookup_from_internet(name)) -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Thu Feb 13 19:34:04 2014 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 13 Feb 2014 18:34:04 +0000 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> <52FCF84F.9010801@mrabarnett.plus.com> Message-ID: <52FD101C.4060407@mrabarnett.plus.com> On 2014-02-13 17:04, ????? wrote: > Another issue: people will expect > for i in a and not in b: > print(i) > to work. Do they currently expect: for i in a and i not in b: print(i) to work? From amber.yust at gmail.com Thu Feb 13 19:43:59 2014 From: amber.yust at gmail.com (Amber Yust) Date: Thu, 13 Feb 2014 18:43:59 +0000 Subject: [Python-ideas] except expression References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: Actually. What if we just reused 'try'? foo = bar() except BazException try 'qux' This also leads naturally to chaining multiple possible fallbacks: foo = bar() except BarException try baz() except BazException try None On Thu Feb 13 2014 at 10:32:16 AM, David Mertz wrote: > On Thu, Feb 13, 2014 at 3:25 AM, spir wrote: > > Of course, it could be done as a function call: > > def catch(callme, catchme, returnme): > try: > return callme() > except catchme: > return returnme > > phone = catch(lambda: addressbook[name], KeyError, "Unknown") > but that's *really* clunky. > > > I don't really find this function all the clunky. However, it also does > not solve the case of an expensive 'except' clause that Nick pointed out as > desirable to avoid evaluating (or, for that matter, an except clause with > side effects). > > E.g. > > phone = catch(lambda: addressbook[name], KeyError, > lookup_from_internet(name)) > > > > > -- > Keeping medicines from the bloodstreams of the sick; food > from the bellies of the hungry; books from the hands of the > uneducated; technology from the underdeveloped; and putting > advocates of freedom in prisons. Intellectual property is > > to the 21st century what the slave trade was to the 16th. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From denis.spir at gmail.com Thu Feb 13 19:45:18 2014 From: denis.spir at gmail.com (spir) Date: Thu, 13 Feb 2014 19:45:18 +0100 Subject: [Python-ideas] except expression In-Reply-To: <52FCC821.4020008@btinternet.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FC9A20.9040303@egenix.com> <52FCC821.4020008@btinternet.com> Message-ID: <52FD12BE.1020008@gmail.com> On 02/13/2014 02:26 PM, Rob Cliffe wrote: > It certainly feels right for the order to be normal value, exception, default > value. So the syntax I would like is > x = entries[0] except IndexError XXX None > where XXX is some keyword. Ideally 'then' or perhaps 'when' which read better > than 'else', but I understand adding a new keyword is a big deal. > (FWIW I also wish trinary expressions were written as > if condition then value-if-true else value-if-false > which to me reads better than the status quo, but that ship has sailed.) > Rob Cliffe What about: x = entries[0] except IndexError then None The weird point with: x = entries[0] except IndexError else None is that 'else' seems to introduce a kind of double negation, where the first negation is due to 'except'. It this seems to indicate what _not_ to do in case of exception, which indeed makes no sense. 'else instead is ok in reverse order: x = entries[0] else None if IndexError However, 'then' in python normally introduces an action, meaning a statement or block (and I'm firmly opposed to giving unrelated meanings to keywords or signs). But in such a case, an expression context, the action is just to choose and pick a value (here for the assignment), thus finally I don't find it that bad. d From denis.spir at gmail.com Thu Feb 13 19:47:05 2014 From: denis.spir at gmail.com (spir) Date: Thu, 13 Feb 2014 19:47:05 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: <52FD1329.9010902@gmail.com> On 02/13/2014 07:43 PM, Amber Yust wrote: > Actually. What if we just reused 'try'? > > foo = bar() except BazException try 'qux' > > This also leads naturally to chaining multiple possible fallbacks: > > foo = bar() except BarException try baz() except BazException try None I like it. Especially because 'try' already works with 'except'. (But note that 'try', like my proposal of 'then', normally introduces a block). d From haoyi.sg at gmail.com Thu Feb 13 19:51:22 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Thu, 13 Feb 2014 10:51:22 -0800 Subject: [Python-ideas] a in x or in y In-Reply-To: <52FD101C.4060407@mrabarnett.plus.com> References: <20140213105716.GA3799@ando> <52FCF84F.9010801@mrabarnett.plus.com> <52FD101C.4060407@mrabarnett.plus.com> Message-ID: I think we're talking about the wrong thing by focusing on whether we want "or in" or "or" or other finnicky syntax things. In terms of whether we can express the logic the OP wanted, we already can: *>>> a = {1, 2, 3}* *>>> b = {4, 5, 6}* *>>> c = 5* *>>> c in a | b* *True* Perfectly concisely, with the exact semantics we want, e.g. when we want to chain more things to the *|* clause, or we want to include *&* clauses, or other things. Maybe we want to extend *|* to other sequences which *in* works on. Maybe we could make *|* chaining on non-sets lazy so we don't build unnecessary data structures. Maybe we want to add a fast-path in the interpreter to speed up this special case. Maybe we want to overload *or* on sets and seqs to behave similarly to how *|* does. On the other hand, I think introducing new syntax for something *that already works with almost the same syntax and exactly the desired semantics* would be a terrible idea. On Thu, Feb 13, 2014 at 10:34 AM, MRAB wrote: > On 2014-02-13 17:04, ????? wrote: > > Another issue: people will expect > > for i in a and not in b: > > print(i) > > to work. > > Do they currently expect: > > for i in a and i not in b: > print(i) > > to work? > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From flying-sheep at web.de Thu Feb 13 19:54:37 2014 From: flying-sheep at web.de (Philipp A.) Date: Thu, 13 Feb 2014 19:54:37 +0100 Subject: [Python-ideas] except expression In-Reply-To: <52FD1329.9010902@gmail.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> <52FD1329.9010902@gmail.com> Message-ID: mainly, a colon introduces a block, which is why i don?t like the colon variant of this expression. 2014-02-13 19:47 GMT+01:00 spir : > On 02/13/2014 07:43 PM, Amber Yust wrote: > >> Actually. What if we just reused 'try'? >> >> foo = bar() except BazException try 'qux' >> >> This also leads naturally to chaining multiple possible fallbacks: >> >> foo = bar() except BarException try baz() except BazException try >> None >> > > I like it. Especially because 'try' already works with 'except'. (But note > that 'try', like my proposal of 'then', normally introduces a block). > > > d > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nathan at cmu.edu Thu Feb 13 20:04:59 2014 From: nathan at cmu.edu (Nathan Schneider) Date: Thu, 13 Feb 2014 14:04:59 -0500 Subject: [Python-ideas] except expression In-Reply-To: <52FD1329.9010902@gmail.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> <52FD1329.9010902@gmail.com> Message-ID: On Thu, Feb 13, 2014 at 1:47 PM, spir wrote: > On 02/13/2014 07:43 PM, Amber Yust wrote: > >> Actually. What if we just reused 'try'? >> >> foo = bar() except BazException try 'qux' >> >> This also leads naturally to chaining multiple possible fallbacks: >> >> foo = bar() except BarException try baz() except BazException try >> None >> > > I like it. Especially because 'try' already works with 'except'. (But note > that 'try', like my proposal of 'then', normally introduces a block). > > This strikes me as counterintuitive because it is inconsistent: 'bar()' is being tried, but does not follow 'try', while the others do. And then the 'try None' has no corresponding 'except'. Suggestion: an expression like foo = (try bar() except BarException) that defaults to None if the exception is caught. This could then be chained with 'or': foo = (try bar() except BarException) or (try baz() except BazException) as distinguished from foo = (try bar() except BarException) or baz() which does not do any exception handling for baz(). (Apologies if something like this has been proposed above; I could not find it from skimming the thread.) Nathan > > d > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Thu Feb 13 20:17:45 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 13 Feb 2014 11:17:45 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> <52FD1329.9010902@gmail.com> Message-ID: On Feb 13, 2014, at 11:04, Nathan Schneider wrote: > On Thu, Feb 13, 2014 at 1:47 PM, spir wrote: >> On 02/13/2014 07:43 PM, Amber Yust wrote: >>> Actually. What if we just reused 'try'? >>> >>> foo = bar() except BazException try 'qux' >>> >>> This also leads naturally to chaining multiple possible fallbacks: >>> >>> foo = bar() except BarException try baz() except BazException try None >> >> I like it. Especially because 'try' already works with 'except'. (But note that 'try', like my proposal of 'then', normally introduces a block). > > This strikes me as counterintuitive because it is inconsistent: 'bar()' is being tried, but does not follow 'try', while the others do. And then the 'try None' has no corresponding 'except'. > > Suggestion: an expression like > > foo = (try bar() except BarException) > > that defaults to None if the exception is caught. This could then be chained with 'or': > > foo = (try bar() except BarException) or (try baz() except BazException) But what if bar() can successfully return None, or just a falsey value in general? Note that this is exactly the reason we needed the if expression: because or is tempting but incorrect in such cases. > > as distinguished from > > foo = (try bar() except BarException) or baz() > > which does not do any exception handling for baz(). > > (Apologies if something like this has been proposed above; I could not find it from skimming the thread.) > > Nathan > >> >> d >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Thu Feb 13 20:19:27 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 13 Feb 2014 11:19:27 -0800 Subject: [Python-ideas] except expression In-Reply-To: <52FD12BE.1020008@gmail.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FC9A20.9040303@egenix.com> <52FCC821.4020008@btinternet.com> <52FD12BE.1020008@gmail.com> Message-ID: <7B85A2EF-F587-488A-80B2-0AEC73DF161A@yahoo.com> On Feb 13, 2014, at 10:45, spir wrote: > On 02/13/2014 02:26 PM, Rob Cliffe wrote: >> It certainly feels right for the order to be normal value, exception, default >> value. So the syntax I would like is >> x = entries[0] except IndexError XXX None >> where XXX is some keyword. Ideally 'then' or perhaps 'when' which read better >> than 'else', but I understand adding a new keyword is a big deal. >> (FWIW I also wish trinary expressions were written as >> if condition then value-if-true else value-if-false >> which to me reads better than the status quo, but that ship has sailed.) >> Rob Cliffe > > What about: > x = entries[0] except IndexError then None > > The weird point with: > x = entries[0] except IndexError else None > > is that 'else' seems to introduce a kind of double negation, where the first negation is due to 'except'. It this seems to indicate what _not_ to do in case of exception, which indeed makes no sense. 'else instead is ok in reverse order: > x = entries[0] else None if IndexError > > However, 'then' in python normally introduces an action, 'then' in Python normally means nothing. If Python _had_ a then keyword in it's if statement, there would have been a much more obvious syntax for the if expression, but it doesn't. > meaning a statement or block (and I'm firmly opposed to giving unrelated meanings to keywords or signs). But in such a case, an expression context, the action is just to choose and pick a value (here for the assignment), thus finally I don't find it that bad. > > d > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From rosuav at gmail.com Thu Feb 13 20:35:47 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 14 Feb 2014 06:35:47 +1100 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> <52FCF84F.9010801@mrabarnett.plus.com> <52FD101C.4060407@mrabarnett.plus.com> Message-ID: On Fri, Feb 14, 2014 at 5:51 AM, Haoyi Li wrote: > I think we're talking about the wrong thing by focusing on whether we want > "or in" or "or" or other finnicky syntax things. In terms of whether we can > express the logic the OP wanted, we already can: > >>>> a = {1, 2, 3} >>>> b = {4, 5, 6} >>>> c = 5 >>>> c in a | b > True > > ... > Maybe we want to extend | to other sequences which in works on. Most definitely. The 'in' operator is far broader than the set() type. Notably: "somestring" in "wholesome" or in "string of words" is not the same as concatenating the two strings and checking if the target string is in the result. ChrisA From abarnert at yahoo.com Thu Feb 13 20:39:05 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 13 Feb 2014 11:39:05 -0800 Subject: [Python-ideas] except expression In-Reply-To: <52FD0F30.4080101@gmail.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <52FD0F30.4080101@gmail.com> Message-ID: On Feb 13, 2014, at 10:30, spir wrote: > On 02/12/2014 10:02 PM, Ram Rachum wrote: >> Hi, >> >> Here's an idea that would help shortening code. Allow a ternary expression >> based on except, like so: >> >> first_entry = entries[0] except IndexError else None >> item = my_queue.get() except queue.Empty else None >> response_text = request('http://whatever.com').text except HttpError >> else "Can't access data" >> >> Aside from the fact that this would be a big grammar addition, a big >> problem here is the usage of the `else` keyword, that when used with except >> usually means "what would happen if there wasn't an exception" and here >> means the opposite. But I couldn't think of a nicer syntax. >> >> I realize that this is a big change and that most people would be opposed >> to this... But I guess I just wanted to share my idea :) > > After some considerations, it seems we reached the point of generalising the idea to: > * provide a builtin way to indicate a special value for the special cases where > the standard value's expression would raise an exception > As Nick Coghlan shows in another post, this is in fact close to a builtin way to deal with potentially failing functions, in general (below 'op'): > > def _helper(op, exc, make_default): > try: > return op() > except exc: > return make_default() > > x = _helper(op, Exception, make_default) > > However, this only applies to the case of functions properly speaking (which purpose is computing a product). What about 'actions', meaning procedures that perform an effect? Take the case of count-words for instance, where (in python) when encoutering a word one would add 1 to its count in a dict which keys are the words. On first encounter, there no entry yet for that word. We could check "word in word_counts", but this is doing the dict lookup twice; and catching an exception is worse. Or you could use setdefault, or use a defaultdict, or, better, a Counter. And I don't see how you expect to handle this situation with new syntax. You want to do word_counts[word] = 0 in the case where word_counts[word] raised--but you also want to "unfail" that word_counts[word] and provide a value for it to return. That can't be the value of word_counts[word] = 0, because statements don't have values. And, even if you could figure out how to provide the number 0, that wouldn't help anyway, because what you're looking for is an augmented assignment target, and those aren't even values in the language that can be passed around. > The problem I guess, here and also similarly for functions, is that there is no way for the _client_ (the caller) to control exception throwing; while only the caller knows whether the failing case actually is an error or not, meaning belongs or not the application's logics. Maybe what we need is in fact something like: > > maybe statement > [then > dependant-block] > else > alternative-block > > This is similar in form to exception catching except (sic!) that the actual meaning is to _avoid_ an exception, not to catch it (to avoid it beeing thrown at all). How do you avoid an exception being thrown? What happens if some function called inside statement tries to raise? If the answer is "we immediately abort that function, and the rest of the statement, and run the alternative block, then what you've described is just exception handling, as it already exists: try: statement except: alternative block else: dependent block All you've done is make it less powerful--you can't limit it to specific types, use the value of the exception, try a complex statement, etc. And I can't think of any other sensible think you could mean here. > More or less the opposite, in fact. The consequence is that in case of failure no exception is raised, instead the alternative 'else' branch is taken. Are you just looking to use a flag to shortcut the creation and propagation of an exception object here? If not, in what other way is this semantically different from "an exception is raised, and handled by the alternative branch"? > Such a construct would work fine for both called actions and functions (which calls are expressions in statement). [1] > > [Similar considerations are evoked in other languages, especially D where I first met them. Generally speaking, people start to realise the seamntic weakness of typical exception catching mecanisms which let no choice or control in the hands of the one who actually knows, namely the client.] > > d > > [1] Practically, this just means adding a test before throwing, and setting a flag instead (the carry would do the job nicely, since it can have no meaning on func return, and is the only copy flag one can set arbitrarily). And then continuing to execute the rest of the function? Or constructing and returning some value nobody will ever look at? And what happens if the exception is raised in a function three levels down the stack? How are you going to abort all three of them? Really, the only thing you can do when the function tries to raise is to abort the whole function, and all the way up to some piece of code that's looking to deal with the problem. You need a flag that's checked in multiple places in the interpreter to make sure it's propagated correctly. Which is exactly what raise already does. So you'd just be providing a different kind of exception--one that can't be caught by try/except, but can be caught by maybe. But if you want two kinds of exception, why not many kinds, which any hierarchy you want--which you can already do today, because exception types are classes. > > Thus, the above code translates to: > statement > if flag: > reset flag > alternative-block > [else: > dependant-block] > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From haoyi.sg at gmail.com Thu Feb 13 20:57:25 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Thu, 13 Feb 2014 11:57:25 -0800 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> <52FCF84F.9010801@mrabarnett.plus.com> <52FD101C.4060407@mrabarnett.plus.com> Message-ID: > "somestring" in "wholesome" or in "string of words" > > is not the same as concatenating the two strings and checking if the > target string is in the result. That's true, but I strongly suspect we can get what we want via plain old *"somestring" in "wholesome" | "string of words"* syntax by overloading the *|* operator to return some data structure that does what we want with *in*. In general, I feel like implementing cool stuff as libraries is superior to implementing cool stuff as hardcoded semantics in the interpreter. This seems like a case where implementing the semantics we want via *|* seems not just feasible, but trivial. On Thu, Feb 13, 2014 at 11:35 AM, Chris Angelico wrote: > On Fri, Feb 14, 2014 at 5:51 AM, Haoyi Li wrote: > > I think we're talking about the wrong thing by focusing on whether we > want > > "or in" or "or" or other finnicky syntax things. In terms of whether we > can > > express the logic the OP wanted, we already can: > > > >>>> a = {1, 2, 3} > >>>> b = {4, 5, 6} > >>>> c = 5 > >>>> c in a | b > > True > > > > ... > > Maybe we want to extend | to other sequences which in works on. > > Most definitely. The 'in' operator is far broader than the set() type. > Notably: > > "somestring" in "wholesome" or in "string of words" > > is not the same as concatenating the two strings and checking if the > target string is in the result. > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Thu Feb 13 21:17:27 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 14 Feb 2014 07:17:27 +1100 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> <52FCF84F.9010801@mrabarnett.plus.com> <52FD101C.4060407@mrabarnett.plus.com> Message-ID: On Fri, Feb 14, 2014 at 6:57 AM, Haoyi Li wrote: >> "somestring" in "wholesome" or in "string of words" >> >> is not the same as concatenating the two strings and checking if the >> target string is in the result. > > That's true, but I strongly suspect we can get what we want via plain old > > "somestring" in "wholesome" | "string of words" > > syntax by overloading the | operator to return some data structure that does > what we want with in. That requires that everything implement some kind of pipe union type. It's putting things backwards. Why should the str type have to be able to make a union of itself and everything else? It's like the reasoning behind having ''.join(seq) rather than seq.join(''). Compare: >>> "foo" in "asdfoobar" >>> "foo" in {"asd","foo","bar"} >>> "foo" in "asdfoobar" or "foo" in {"asd","foo","bar"} Okay, now how are you going to deduplicate the last line? >>> "foo" in "asdfoobar" | {"asd","foo","bar"} The only way this would work is by having some universal "this-or-that" structure that doesn't care about its data types. And once you have that, you may as well make it a language construct and have lazy evaluation semantics: >>> cmd not in forbiddens and in fetch_command_list() If the forbiddens set is small and kept locally (as a blacklist should be), tripping that check should be able to short-circuit the calling of fetch_command_list, just as the fully written out version would: >>> cmd not in forbiddens and cmd in fetch_command_list() There's fundamentally no way to implement that with the | operator. I like the proposal, largely for its parallel with the chained comparison operators (1 References: <20140213105716.GA3799@ando> Message-ID: On 14 Feb 2014 00:39, "Boris Borcic" wrote: > > Nick Coghlan wrote: >> >> >> >> In linguistics, there's a concept called "garden path" sentences - >> >> these are sentences where the first part is a grammatically coherent >> >> sentence, but by *adding more words to the end*, you change the >> >> meaning of words that appeared earlier. >> > >> > >> > Are you trying to say "X in Y and" forms a complete expression?? And couldn't the lexer recognize "and in" >> and cognates as single tokens anyway, like it apparently does presently for "is not" ? >> >> Yes, there are ways around the technical limitation - the point of the rest of the post was to make it clear >> that was largely irrelevant, because it would still be too hard to read. >> > > And my point was that your argument was rather unconvincing, for a variety of reasons going from the difference between indefinite lookahead and (rectifiable) two token lookahead, to the rarity of actual garden path sentences that's evidenced by the fact that the single example you cite is many decades old. Huh? Garden path sentences are rare because they're hard to understand - people instinctively realise that and rephrase them as something less awkward. The example I used is the one that was used to explain the concept to me because it's short and to the point. The point of the elaboration in the post was to make it clear that we *know* it is technical possible to work around the "only one token lookahead" limitation, but that as a general principle of the language design *we won't*. It's not an accident of the implementation, it's a deliberate design choice intended to benefit both human readers and the creators of automated tools. This is a constraint that people *need* to take into account if they wish to make successful syntax change proposals for Python. A suggestion like this, which would require defining two or three word tokens to meet the letter of the guideline while still breaking its spirit, simply isn't going to happen (especially when it doesn't provide a significant increase in expressiveness). Now, if you want to argue with me about whether or not it's a good rule, that's not an argument I'm interested in having - you can certainly design usable languages that don't follow it, but I'm pointing out an existing design principle for Python, and providing the general rationale for it (i.e. simplicity), not trying to make the case that *all* languages should be designed that way. Regards, Nick. > > Cheers, Boris Borcic > > > > --- > Ce courrier ?lectronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. > http://www.avast.com > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Thu Feb 13 22:11:35 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 14 Feb 2014 08:11:35 +1100 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> <52FCF84F.9010801@mrabarnett.plus.com> <52FD101C.4060407@mrabarnett.plus.com> Message-ID: <20140213211135.GB3799@ando> On Thu, Feb 13, 2014 at 10:51:22AM -0800, Haoyi Li wrote: > I think we're talking about the wrong thing by focusing on whether we want > "or in" or "or" or other finnicky syntax things. In terms of whether we can > express the logic the OP wanted, we already can: > > *>>> a = {1, 2, 3}* > *>>> b = {4, 5, 6}* > *>>> c = 5* > *>>> c in a | b* > *True* > > Perfectly concisely, with the exact semantics we want, But they are not the same semantics! They are *quite different* semantics. You may have lead yourself astray because the result of: c in a or c in b happens to give the same result as: c in a|b for the specific values of a, b, c given. But they actually perform semantically different things: the first one lazily performs two separate containment tests, while the second eagerly calculates the union of two sets a|b and then performs a single non-lazy containment test. Because the first form is lazy, this succeeds: a = {1, 2, 3} b = None c = 2 c in a or c in b (admittedly it succeeds by accident) while your eager version needs to be re-written as: c in (a|b if b is not None else a) in order to avoid failure. Another problem: since the __or__ operator can be over-ridden, you cannot afford to assume that a|b will always be a union. Or it might have side-effects. __contains__ also can be over-ridden, and might also have side-effects, but in general they will be *different* side-effects. -- Steven From greg.ewing at canterbury.ac.nz Thu Feb 13 22:18:01 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 14 Feb 2014 10:18:01 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: <52FD3689.3050307@canterbury.ac.nz> Chris Angelico wrote: > phone = addressbook[name] except KeyError or "Unknown" How about phone = addressbook[name] except "Unknown" if KeyError Yes, it puts things in a different order from the except statement, but I don't think that's any worse than the if-expression being ordered differently from the if-statement, and it has the same benefit, i.e. reading more smoothly. -- Greg From haoyi.sg at gmail.com Thu Feb 13 22:20:22 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Thu, 13 Feb 2014 13:20:22 -0800 Subject: [Python-ideas] a in x or in y In-Reply-To: <20140213211135.GB3799@ando> References: <20140213105716.GA3799@ando> <52FCF84F.9010801@mrabarnett.plus.com> <52FD101C.4060407@mrabarnett.plus.com> <20140213211135.GB3799@ando> Message-ID: Ah, I did not think about the laziness. That indeed is a pain since we can't create our own custom lazy operators/methods. I would say the correct answer is that we should let people define their own lazy operators, and then define *or_in* or whatever as a lazy operator/method, but I'm sure others would disagree. On Thu, Feb 13, 2014 at 1:11 PM, Steven D'Aprano wrote: > On Thu, Feb 13, 2014 at 10:51:22AM -0800, Haoyi Li wrote: > > I think we're talking about the wrong thing by focusing on whether we > want > > "or in" or "or" or other finnicky syntax things. In terms of whether we > can > > express the logic the OP wanted, we already can: > > > > *>>> a = {1, 2, 3}* > > *>>> b = {4, 5, 6}* > > *>>> c = 5* > > *>>> c in a | b* > > *True* > > > > Perfectly concisely, with the exact semantics we want, > > But they are not the same semantics! They are *quite different* > semantics. You may have lead yourself astray because the result of: > > c in a or c in b > > happens to give the same result as: > > c in a|b > > for the specific values of a, b, c given. But they actually perform > semantically different things: the first one lazily performs two > separate containment tests, while the second eagerly calculates the > union of two sets a|b and then performs a single non-lazy containment > test. > > Because the first form is lazy, this succeeds: > > a = {1, 2, 3} > b = None > c = 2 > c in a or c in b > > (admittedly it succeeds by accident) while your eager version needs to > be re-written as: > > c in (a|b if b is not None else a) > > in order to avoid failure. > > Another problem: since the __or__ operator can be over-ridden, you > cannot afford to assume that a|b will always be a union. Or it might > have side-effects. __contains__ also can be over-ridden, and might also > have side-effects, but in general they will be *different* side-effects. > > > -- > Steven > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amber.yust at gmail.com Thu Feb 13 22:26:42 2014 From: amber.yust at gmail.com (Amber Yust) Date: Thu, 13 Feb 2014 21:26:42 +0000 Subject: [Python-ideas] except expression References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FD3689.3050307@canterbury.ac.nz> Message-ID: As has been noted earlier in the thread when someone proposed except...if - there are parsing issues due to if..else already being a ternary expression. On Thu Feb 13 2014 at 1:18:40 PM, Greg Ewing wrote: > Chris Angelico wrote: > > > phone = addressbook[name] except KeyError or "Unknown" > > How about > > phone = addressbook[name] except "Unknown" if KeyError > > Yes, it puts things in a different order from the except > statement, but I don't think that's any worse than the > if-expression being ordered differently from the if-statement, > and it has the same benefit, i.e. reading more smoothly. > > -- > Greg > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Thu Feb 13 22:31:48 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 14 Feb 2014 10:31:48 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: <52FD39C4.4010508@canterbury.ac.nz> Nick Coghlan wrote: > "if" would be better, but, as you already noted, poses significant > parsing challenges (since it likely wouldn't be easy to require that > ternary expressions use parentheses in this new construct, but still > allow the parentheses to be omitted in the general case). I don't think that would be an insurmountable difficulty. The if-expression appears at the 'test' level in the grammar, so all it should take is for the expression following 'except' to be something lower, such as an 'or_test'. There are precedents for this kind of thing, e.g. yield expressions can appear unparenthesised in some contexts but not others. -- Greg From steve at pearwood.info Thu Feb 13 22:38:37 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 14 Feb 2014 08:38:37 +1100 Subject: [Python-ideas] except expression In-Reply-To: <854n439skn.fsf@benfinney.id.au> References: <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCA62B.90005@gmail.com> <854n439skn.fsf@benfinney.id.au> Message-ID: <20140213213837.GC3799@ando> On Thu, Feb 13, 2014 at 10:10:16PM +1100, Ben Finney wrote: > spir writes: > > > I think the right way is not to call the function at all, but to check > > it. Conceptually: > > > > if col.is_empty(): > > handle_special_case() > > else: > > handle_standard_case() > > Or, better from two perspectives (?empty? should normally entail > ?evaluates to boolean false?; and, the normal case should be the first > branch from the ?if?):: > > if col: > handle_standard_case() > else: > handle_empty_case() I'm afraid that entails a race-condition. col may be non-empty at the moment you check it, but by the time you handle the non-empty case a microsecond later it has become empty. This is why we have both Look Before You Leap and Easier To Ask Forgiveness Than Permission. We can perform LBYL in an expression using ternary if operator: (handle_standard_case if col else handle_empty_case)() but there's no equivalent to a try...except in a single expression, which is what this thread is about. -- Steven From zachary.ware+pyideas at gmail.com Thu Feb 13 22:55:56 2014 From: zachary.ware+pyideas at gmail.com (Zachary Ware) Date: Thu, 13 Feb 2014 15:55:56 -0600 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: On Thu, Feb 13, 2014 at 12:43 PM, Amber Yust wrote: > Actually. What if we just reused 'try'? > > foo = bar() except BazException try 'qux' > > This also leads naturally to chaining multiple possible fallbacks: > > foo = bar() except BarException try baz() except BazException try None This is my favorite so far, followed by s/try/return/. I agree with Nathan Schneider though, it does seem odd to have "try" following "except" rather than the other way around. What about just allowing simple try...except constructs to be condensed to a single line? foo = try bar() except BarException as e e.args[0] -- Zach From amber.yust at gmail.com Thu Feb 13 23:02:23 2014 From: amber.yust at gmail.com (Amber Yust) Date: Thu, 13 Feb 2014 22:02:23 +0000 Subject: [Python-ideas] except expression References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: The last part of that ("as e e.args[0]") is a pain to read due to no clear separation, just whitespace. On Thu Feb 13 2014 at 2:00:00 PM, Zachary Ware < zachary.ware+pyideas at gmail.com> wrote: > On Thu, Feb 13, 2014 at 12:43 PM, Amber Yust wrote: > > Actually. What if we just reused 'try'? > > > > foo = bar() except BazException try 'qux' > > > > This also leads naturally to chaining multiple possible fallbacks: > > > > foo = bar() except BarException try baz() except BazException try > None > > This is my favorite so far, followed by s/try/return/. I agree with > Nathan Schneider though, it does seem odd to have "try" following > "except" rather than the other way around. > > What about just allowing simple try...except constructs to be > condensed to a single line? > > foo = try bar() except BarException as e e.args[0] > > -- > Zach > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zachary.ware+pyideas at gmail.com Thu Feb 13 23:11:15 2014 From: zachary.ware+pyideas at gmail.com (Zachary Ware) Date: Thu, 13 Feb 2014 16:11:15 -0600 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: On Thu, Feb 13, 2014 at 4:02 PM, Amber Yust wrote: > On Thu Feb 13 2014 at 2:00:00 PM, Zachary Ware > wrote: >> What about just allowing simple try...except constructs to be >> condensed to a single line? >> >> foo = try bar() except BarException as e e.args[0] > The last part of that ("as e e.args[0]") is a pain to read due to no clear > separation, just whitespace. I don't disagree, but it's less bad than some of the other alternatives :). It's also not quite as bad when not using the exception (which I suspect would be somewhat more common): foo = try bar() except BarException "default" -- Zach PS: I know Gmail makes it harder than it needs to be, but could you please try to avoid top-posting? (This topic doesn't need to be derailed into another debate about top-posting though, so please either follow this suggestion or ignore it entirely :)) From greg.ewing at canterbury.ac.nz Thu Feb 13 23:22:05 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 14 Feb 2014 11:22:05 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: <52FD458D.7050709@canterbury.ac.nz> Chris Angelico wrote: > phone = "Unknown" if except KeyError else addressbook[name] -42. My brain gets totally derailed when it hits "if except"; it parses that bit as a syntax error. -- Greg From rosuav at gmail.com Thu Feb 13 23:23:06 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 14 Feb 2014 09:23:06 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: On Fri, Feb 14, 2014 at 9:11 AM, Zachary Ware wrote: > On Thu, Feb 13, 2014 at 4:02 PM, Amber Yust wrote: > >> On Thu Feb 13 2014 at 2:00:00 PM, Zachary Ware >> wrote: >>> What about just allowing simple try...except constructs to be >>> condensed to a single line? >>> >>> foo = try bar() except BarException as e e.args[0] > >> The last part of that ("as e e.args[0]") is a pain to read due to no clear >> separation, just whitespace. > > I don't disagree, but it's less bad than some of the other > alternatives :). It's also not quite as bad when not using the > exception (which I suspect would be somewhat more common): > > foo = try bar() except BarException "default" Looks like a bug magnet, if nothing else. Are you allowed newlines in that construct? ChrisA From rosuav at gmail.com Thu Feb 13 23:24:54 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 14 Feb 2014 09:24:54 +1100 Subject: [Python-ideas] except expression In-Reply-To: <52FD458D.7050709@canterbury.ac.nz> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> <52FD458D.7050709@canterbury.ac.nz> Message-ID: On Fri, Feb 14, 2014 at 9:22 AM, Greg Ewing wrote: > Chris Angelico wrote: >> >> phone = "Unknown" if except KeyError else addressbook[name] > > > -42. My brain gets totally derailed when it hits "if except"; > it parses that bit as a syntax error. Yeah, and it reads backwards anyway. I don't particularly like that syntax. ChrisA From greg.ewing at canterbury.ac.nz Thu Feb 13 23:25:12 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 14 Feb 2014 11:25:12 +1300 Subject: [Python-ideas] except expression In-Reply-To: <52FCAB93.40504@gmail.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: <52FD4648.4060603@canterbury.ac.nz> spir wrote: > [By the way, this shows that: > x = b if cond else a > should really be: > x = a else b if cond > The difference being that the standard case is expressed first, the > exceptional one being then introduced as an special variant.] I don't think it shows that at all. Which is the normal case and which is the exceptional one depends entirely on how the condition is expressed. -- Greg From ncoghlan at gmail.com Thu Feb 13 23:42:52 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 14 Feb 2014 08:42:52 +1000 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> Message-ID: On 14 February 2014 06:55, Nick Coghlan wrote: > A suggestion like this, which would require defining two or three word > tokens to meet the letter of the guideline while still breaking its spirit, > simply isn't going to happen (especially when it doesn't provide a > significant increase in expressiveness). OK, I realised this proposal is actually closer to chained comparison operators than I initially thought, and multiword tokens do indeed make it technically feasible. However, that's still just a hack that meet the letter of the guideline while still failing to abide by the spirit of it. (Disclaimer: all parsing descriptions below are intended to be illustrative, and do not necessarily reflect how the CPython compiler actually works. In particular, "compiler" is used as a shorthand to refer to the arbitrary parts of toolchain.) First, lets look at the existing multi-word tokens. "is" vs "is not" is relatively simple: they're both binary operators. In the following expressions: LHS is RHS LHS is not RHS the extra "not" after "is", changes the comparison *operator*, but it doesn't need to reach back and alter the interpretation of the LHS expression itself. "not" vs "not in" is a little different: that relies on the fact that failing to follow a "not" in this position with "in" is a SyntaxError: LHS not RHS # Illegal LHS not in RHS So again, the addition of the "in" doesn't affect the interpretation of the LHS in any way. Contrast that with the proposal in this thread: LHS or in RHS LHS and in RHS LHS or not in RHS LHS and not in RHS In all of these cases, the "or op"/"and op" alters the way the *LHS* is processed. The closest parallel we have to that is chained comparison operators, where the presence of "op2" alters the processing of "X op1 Y": X op1 Y op2 Z All comparison operators are designed such that when the compiler is about to resolve "X op1 Y", it looks ahead at the next token, and if it sees another comparison operator, starts building an "and" construct instead: X op1 Y and Y op2 Z A *generalisation* of the current proposal, to work with arbitrary comparison operators, clearly requires two token look ahead in order to see "op2" after the logical operator: X op1 Y or op2 Z X op1 Y and op2 Z To be reparsed as: X op1 Y or X op2 Z X op1 Y and X op2 Z And allowing constructs like: if x == 1 or == 2 or in range(10, 20): # Do stuff This is why making "or in" (etc) multiword tokens would still break the spirit of the "only one token lookahead" guideline - the proposal put forward is actually perfectly coherent for arbitrary comparison operators, it just requires two token lookahead in order to see both the logical operator *and* the comparison operator after it before deciding how to resolve the processing of the LHS expression. I'm actually more amenable to the generalised proposal than I was to the original more limited idea, but anyone that wants to pursue it further really needs to appreciate how deeply ingrained that "only one token look ahead" guideline is for most of the core development team. I don't actually believe even the more general proposal adds enough expressiveness to convince Guido to change the lookahead restriction, but I'm only -0 on that, whereas I was -1 on the containment-tests-only special cased version. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From greg.ewing at canterbury.ac.nz Thu Feb 13 23:54:55 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 14 Feb 2014 11:54:55 +1300 Subject: [Python-ideas] on colon ':', and some more In-Reply-To: References: <52FC9CDD.8010003@gmail.com> Message-ID: <52FD4D3F.3050408@canterbury.ac.nz> Terry Reedy wrote: > Math gets away with overloading = Some programming languages get away with it too, e.g. BASIC. But if you do that, you give up the possibility of chained assignments, e.g. x = y = z = 0 to set x, y and z to 0. -- Greg From greg.ewing at canterbury.ac.nz Fri Feb 14 00:04:58 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 14 Feb 2014 12:04:58 +1300 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> Message-ID: <52FD4F9A.1010307@canterbury.ac.nz> Boris Borcic wrote: > Are you trying to say "X in Y and" forms a complete expression? No, I think he's saying that "X in Y" forms a complete expression, but only if it's *not* followed by "or in". If it is, you need to back up and re-interpret it as "X in ". A syntax along the lines of X in either Y or Z would avoid the problem. But we don't have a good candidate for 'either' that's already a keyword. -- Greg From greg.ewing at canterbury.ac.nz Fri Feb 14 00:10:34 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 14 Feb 2014 12:10:34 +1300 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> Message-ID: <52FD50EA.7060408@canterbury.ac.nz> Serhiy Storchaka wrote: > You forgot keyword "are". > > keys are in car or in pocket Hmmm, that has possibilities: car or pocket are around keys -- Greg From carl at oddbird.net Fri Feb 14 00:23:02 2014 From: carl at oddbird.net (Carl Meyer) Date: Thu, 13 Feb 2014 16:23:02 -0700 Subject: [Python-ideas] a in x or in y In-Reply-To: <52FD4F9A.1010307@canterbury.ac.nz> References: <20140213105716.GA3799@ando> <52FD4F9A.1010307@canterbury.ac.nz> Message-ID: <52FD53D6.5050706@oddbird.net> On 02/13/2014 04:04 PM, Greg Ewing wrote: [snip] > A syntax along the lines of > > X in either Y or Z > > would avoid the problem. But we don't have a good > candidate for 'either' that's already a keyword. This makes me think of X in any(Y, Z) which is equally concise, and I think clearer and more readable (for garden-path reasons) than X in Y or in Z It's also technically not ambiguous, since currently any() only takes one argument, and "X in any(Y)" is useless. Still a confusing overload of the meaning of any(). Implementation would be a bit ugly; it would need to return an object that implements every operator magic method, and "distributes" the operation across the objects passed to it. Equivalent could be done for all(). Or different names could be chosen to avoid the overloads, though I'm not sure what those names would be. Probably there be dragons, but it might be fun to experiment with (and doesn't require changes to the language, only new/modified builtins, or even just library additions). Carl From greg.ewing at canterbury.ac.nz Fri Feb 14 00:26:01 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 14 Feb 2014 12:26:01 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: <52FD5489.80800@canterbury.ac.nz> Terry Reedy wrote: > >>> (seq or ['default'])[0] > 'default' > > (Python's precedence rules make the parentheses optional). Um, no, they don't: >>> seq = ['a'] >>> (seq or ['default'])[0] 'a' >>> seq or ['default'][0] ['a'] In any case, this only works for an extremely special case (an index of 0), and can hardly be seen as an alternative to the proposed except-expression. >>>> {}.get('key', 'default') > 'default' >>>> {} or {'key':'default'}['key'] > 'default' This is an even *more* special case, since it only works if you know that the only way the dict can possibly fail to contain the key is if it's completely empty. -- Greg From amber.yust at gmail.com Fri Feb 14 00:26:02 2014 From: amber.yust at gmail.com (Amber Yust) Date: Thu, 13 Feb 2014 23:26:02 +0000 Subject: [Python-ideas] a in x or in y References: <20140213105716.GA3799@ando> <52FD4F9A.1010307@canterbury.ac.nz> <52FD53D6.5050706@oddbird.net> Message-ID: The issue I see with "X in any(Y, Z)" is that it's unclear to the reader how it differs from "X in (Y,Z)". On Thu Feb 13 2014 at 3:23:41 PM, Carl Meyer wrote: > On 02/13/2014 04:04 PM, Greg Ewing wrote: > [snip] > > A syntax along the lines of > > > > X in either Y or Z > > > > would avoid the problem. But we don't have a good > > candidate for 'either' that's already a keyword. > > This makes me think of > > X in any(Y, Z) > > which is equally concise, and I think clearer and more readable (for > garden-path reasons) than > > X in Y or in Z > > It's also technically not ambiguous, since currently any() only takes > one argument, and "X in any(Y)" is useless. Still a confusing overload > of the meaning of any(). > > Implementation would be a bit ugly; it would need to return an object > that implements every operator magic method, and "distributes" the > operation across the objects passed to it. Equivalent could be done for > all(). > > Or different names could be chosen to avoid the overloads, though I'm > not sure what those names would be. > > Probably there be dragons, but it might be fun to experiment with (and > doesn't require changes to the language, only new/modified builtins, or > even just library additions). > > Carl > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From haoyi.sg at gmail.com Fri Feb 14 00:32:52 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Thu, 13 Feb 2014 15:32:52 -0800 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> <52FD4F9A.1010307@canterbury.ac.nz> <52FD53D6.5050706@oddbird.net> Message-ID: > X in any(Y, Z) It also doesn't have the desired laziness, unless we special case *any* in the interpreter, or provide a more generic mechanism for lazy parameters without writing *lambda: *everywhere On Thu, Feb 13, 2014 at 3:26 PM, Amber Yust wrote: > The issue I see with "X in any(Y, Z)" is that it's unclear to the reader > how it differs from "X in (Y,Z)". > > > On Thu Feb 13 2014 at 3:23:41 PM, Carl Meyer wrote: > >> On 02/13/2014 04:04 PM, Greg Ewing wrote: >> [snip] >> > A syntax along the lines of >> > >> > X in either Y or Z >> > >> > would avoid the problem. But we don't have a good >> > candidate for 'either' that's already a keyword. >> >> This makes me think of >> >> X in any(Y, Z) >> >> which is equally concise, and I think clearer and more readable (for >> garden-path reasons) than >> >> X in Y or in Z >> >> It's also technically not ambiguous, since currently any() only takes >> one argument, and "X in any(Y)" is useless. Still a confusing overload >> of the meaning of any(). >> >> Implementation would be a bit ugly; it would need to return an object >> that implements every operator magic method, and "distributes" the >> operation across the objects passed to it. Equivalent could be done for >> all(). >> >> Or different names could be chosen to avoid the overloads, though I'm >> not sure what those names would be. >> >> Probably there be dragons, but it might be fun to experiment with (and >> doesn't require changes to the language, only new/modified builtins, or >> even just library additions). >> >> Carl >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zuo at chopin.edu.pl Fri Feb 14 00:37:42 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Fri, 14 Feb 2014 00:37:42 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: <198f96d1fb1ccee5d86dea10b40f5c7b@chopin.edu.pl> My 3 cents (not proposed yet, AFAIK): 1. Simple Variant: get_it() except (IndexError: None) 2. Variant with 'as': get_it() except (OSError as exc: exc.errno) 3. Variant with multiple exceptions: get_it() except (FileNotFound: 0, OSError as exc: exc.errno, Exception: None) Cheers. *j From greg.ewing at canterbury.ac.nz Fri Feb 14 00:37:50 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 14 Feb 2014 12:37:50 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> Message-ID: <52FD574E.5020403@canterbury.ac.nz> Amber Yust wrote: > Actually. What if we just reused 'try'? > > foo = bar() except BazException try 'qux' This suggests that trying 'qux' may somehow fail to work, which is not the intended meaning at all. -- Greg From rosuav at gmail.com Fri Feb 14 00:43:35 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 14 Feb 2014 10:43:35 +1100 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> <52FD4F9A.1010307@canterbury.ac.nz> <52FD53D6.5050706@oddbird.net> Message-ID: On Fri, Feb 14, 2014 at 10:26 AM, Amber Yust wrote: > The issue I see with "X in any(Y, Z)" is that it's unclear to the reader how > it differs from "X in (Y,Z)". X in (Y, Z) <-> X == any(Y, Z) This equivalence is seen in SQL, more or less. I wouldn't want to encourage it, though. ChrisA From greg.ewing at canterbury.ac.nz Fri Feb 14 00:46:29 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 14 Feb 2014 12:46:29 +1300 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> <52FCF84F.9010801@mrabarnett.plus.com> <52FD101C.4060407@mrabarnett.plus.com> Message-ID: <52FD5955.40008@canterbury.ac.nz> No language change required: class either(object): def __init__(self, *args): self.args = args def __contains__(self, x): for a in self.args: if x in a: return True return False pocket = ['hankie', 'pocketknife', 'keys'] car = ['satchel', 'jacket', 'teddybear'] if 'keys' in either(car, pocket): print("Found them") else: print("Lost them") -- Greg From ram at rachum.com Fri Feb 14 00:49:20 2014 From: ram at rachum.com (Ram Rachum) Date: Fri, 14 Feb 2014 01:49:20 +0200 Subject: [Python-ideas] a in x or in y In-Reply-To: <52FD5955.40008@canterbury.ac.nz> References: <20140213105716.GA3799@ando> <52FCF84F.9010801@mrabarnett.plus.com> <52FD101C.4060407@mrabarnett.plus.com> <52FD5955.40008@canterbury.ac.nz> Message-ID: That's nice, but having to import a definition like that would be quite cumbersome. Also, this doesn't allow lazy evaluation. On Fri, Feb 14, 2014 at 1:46 AM, Greg Ewing wrote: > No language change required: > > class either(object): > > def __init__(self, *args): > self.args = args > > def __contains__(self, x): > for a in self.args: > if x in a: > return True > return False > > pocket = ['hankie', 'pocketknife', 'keys'] > car = ['satchel', 'jacket', 'teddybear'] > > if 'keys' in either(car, pocket): > print("Found them") > else: > print("Lost them") > > -- > Greg > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > -- > > --- You received this message because you are subscribed to a topic in the > Google Groups "python-ideas" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/python-ideas/LqFVq8sMMwU/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > python-ideas+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Fri Feb 14 01:34:12 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 13 Feb 2014 16:34:12 -0800 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> <52FCF84F.9010801@mrabarnett.plus.com> <52FD101C.4060407@mrabarnett.plus.com> <52FD5955.40008@canterbury.ac.nz> Message-ID: <52FD6484.7060500@stoneleaf.us> On 02/13/2014 03:49 PM, Ram Rachum wrote: > On Fri, Feb 14, 2014 at 1:46 AM, Greg Ewing wrote: >> >> class either(object): >> >> def __init__(self, *args): >> self.args = args >> >> def __contains__(self, x): >> for a in self.args: >> if x in a: >> return True >> return False > > That's nice, but having to import a definition like that would be quite cumbersome. What? Importing is cumbersome? Since when? > Also, this doesn't allow lazy evaluation. Certainly it does. If the target is in the first thing it returns True at that point. -- ~Ethan~ From ron3200 at gmail.com Fri Feb 14 02:00:44 2014 From: ron3200 at gmail.com (Ron Adam) Date: Thu, 13 Feb 2014 19:00:44 -0600 Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: <1392249059.83235.YahooMailNeo@web181005.mail.ne1.yahoo.com> References: <20140212032919.GP3799@ando> <52FB4915.1040705@gmail.com> <1392249059.83235.YahooMailNeo@web181005.mail.ne1.yahoo.com> Message-ID: On 02/12/2014 05:50 PM, Andrew Barnert wrote: > From: spir > > Sent: Wednesday, February 12, 2014 2:12 AM > > >>> What the compiler need, more than the type, is the "weight" (memory >>> size); so as to be able to (statically) reserve slots in the stack >>> (or register, wherever). [In general, static efficiency comes from >>> knowing the weight of values, also at times their structures (say >>> globally their "formats"), rather than their types.] >>> >>> At first sight, it seems to me that python's optional and "default" >>> args may also be a big obstacle to inlining. (I wonder how they're >>> done for regular func def and cal, by the way.) > > You're still thinking in C terms. > > First, in Python, the stack is a stack of objects, not bytes. They all > have the same weight and structure (in CPython, each one is a PyObject > pointer). > > Meanwhile, a function is an object with attributes, including a tuple of > default values, a code object with its own attributes like a tuple of > argument names, and so on. The interpreter (in particular, the > CALL_FUNCTION bytecode handler) is able to match up arguments and > parameters at runtime. > > Also, a frame is an object with attributes, including a reference to the > previous frame, not just a chunk of bytes on the stack. > > So, there is no "call prologue" compiled into each function. And, more > generally, the problems you're expecting just don't exist. > > The caller just pushes the function and the arguments and does a > CALL_FUNCTION, then gets the return value on the stack. The callee just > starts off with its frame's locals already set up, and runs exactly the > code you wrote and noting more. How it works is all documented, although > scattered in a few different places (the bytecode specs in the dis > module docs, the attributes of function and code objects in the inspect > module docs, the calls and definitions sections of the reference, and > the callable C API). I tried to put a summary together > athttp://stupidpythonideas.blogspot.com/2014/02/arguments-and-parameters-under-covers.html > if it helps. > > So, default parameter values are not an obstacle to inlining at all. > They're handled at runtime by CALL_FUNCTION just like arguments are. The > obstacle to inlining is that you're ultimately trying to inline a > runtime object at compile time, which doesn't make any sense. Elsewhere > in the thread I explained some ways you could do something similar but > sensible, but it's not going to be like C or C++ inlining. Right, Think in byte code not python source code. It should be possibly to define a limited type of function that has a code object which is insert-able into in the byte code, but it would have some pretty limited restrictions. * Take no arguments. * Have all variables declared as non-local. * Not access any closure names. * Insure there is no executable code after any return location. The the call could be removed and the code in the code object could be inserted at that location, and the returns removed so the return value is left on the stack. That isn't a common pattern, so it's not very useful. And probably not worth implementing without expanding on that in some way. Ron From rosuav at gmail.com Fri Feb 14 02:12:59 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 14 Feb 2014 12:12:59 +1100 Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: References: <20140212032919.GP3799@ando> <52FB4915.1040705@gmail.com> <1392249059.83235.YahooMailNeo@web181005.mail.ne1.yahoo.com> Message-ID: On Fri, Feb 14, 2014 at 12:00 PM, Ron Adam wrote: > It should be possibly to define a limited type of function that has a code > object which is insert-able into in the byte code, but it would have some > pretty limited restrictions. > > * Have all variables declared as non-local. By "non-local" do you mean specifically the Python 3 "nonlocal" keyword, or is it acceptable to declare globals? ChrisA From ethan at stoneleaf.us Fri Feb 14 01:52:59 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 13 Feb 2014 16:52:59 -0800 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> <52FD4F9A.1010307@canterbury.ac.nz> <52FD53D6.5050706@oddbird.net> Message-ID: <52FD68EB.8040303@stoneleaf.us> On 02/13/2014 03:32 PM, Haoyi Li wrote: > Carl Meyer wrote: >> This makes me think of >> >> X in any(Y, Z) > > It also doesn't have the desired laziness, unless we special case *any* in the interpreter, or provide a more generic > mechanism for lazy parameters without writing *lambda: *everywhere Well, currently any() and all() return True/False. To overload like this would require returning an object with a __contains__ like Greg's either() (which is lazy), and a __bool__ that does what any() and all() currently do. I can see it being confusing, and it also doesn't get the chained comparisons like "or in" and friends do. -- ~Ethan~ From ram.rachum at gmail.com Thu Feb 13 22:59:31 2014 From: ram.rachum at gmail.com (Ram Rachum) Date: Thu, 13 Feb 2014 13:59:31 -0800 (PST) Subject: [Python-ideas] if expensive_computation() as x: Message-ID: Hi everybody, Please excuse the recent torrent of crazy ideas :) I was reading code of the Shpaml project, trying to make a patch, while I saw code that looked inelegant to me. I considered how to improve it, but then saw I don't know how to. The code paraphrased is this: if expensive_computation_0(): x = expensive_computation_0() # Do something with x... elif expensive_computation_1(): x = expensive_computation_1() # Do something with x... elif expensive_computation_2(): x = expensive_computation_2() # Do something with x... The problem here is that we're doing the expensive computation twice. That's because we first need to check its truth value, and if it's true we need to actually use the value and do stuff with it. One solution would be to calculate it and put it in a variable before checking truth value. The problem with that is, besides being quite verbose, it doesn't work with the `elif` lines. You want to calculate the values only if the `elif` branch is being taken. (i.e. you can't do it before the `if` because then you might be calculating it needlessly since maybe the first condition would be true.) Obviously this can be "solved", i.e. rewritten in a way that wouldn't call the expensive operation twice, but the solution would probably be quite verbose, which is something we don't want. My suggestion: if expensive_computation_0() as x: # Do something with x... elif expensive_computation_1() as x: # Do something with x... elif expensive_computation_2() as x: # Do something with x... If you'd like to bind to a variable only a part of the condition, this would work too: if x<5 with expensive_computation_0() as x: # Do something with x What do you think? Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Fri Feb 14 03:58:54 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 14 Feb 2014 13:58:54 +1100 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: Message-ID: Responding to your post in different order to the original. On Fri, Feb 14, 2014 at 8:59 AM, Ram Rachum wrote: > If you'd like to bind to a variable only a part of the condition, this would > work too: > > if x<5 with expensive_computation_0() as x: > # Do something with x Definitely don't like this syntax - while it might be useful to snapshot part of a condition (I've done it in C plenty of times), this notation feels clumsy. However... > My suggestion: > > if expensive_computation_0() as x: > # Do something with x... > elif expensive_computation_1() as x: > # Do something with x... > elif expensive_computation_2() as x: > # Do something with x... ... this simpler form does look reasonable. The "as" part will *only* come at the end of the expression, and it *always* applies to the whole expression, so it's fairly clear. There is another cheat you can do, though, and that's to use break or return instead of an elif chain. Going back to your original: > The code paraphrased is this: > > if expensive_computation_0(): > x = expensive_computation_0() > # Do something with x... > elif expensive_computation_1(): > x = expensive_computation_1() > # Do something with x... > elif expensive_computation_2(): > x = expensive_computation_2() > # Do something with x... the alternative would be something like this: while "allow_break": x = expensive_computation_0(): if x: # Do something with x... break x = expensive_computation_1(): if x: # Do something with x... break x = expensive_computation_2(): if x: # Do something with x... break # whatever your 'else' clause would be, if any break Or, using a function instead: def allow_return(): nonlocal everything, you, need x = expensive_computation_0(): if x: # Do something with x... return x = expensive_computation_1(): if x: # Do something with x... return x = expensive_computation_2(): if x: # Do something with x... return allow_return() Both of them are abusing their keywords into gotos, but ultimately, that's what if/elif/elif is anyway - as soon as you finish one elif, you goto the end of the block. It's not perfect by any means, but it works. ChrisA From nathan at cmu.edu Fri Feb 14 04:19:46 2014 From: nathan at cmu.edu (Nathan Schneider) Date: Thu, 13 Feb 2014 22:19:46 -0500 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: Message-ID: On Thu, Feb 13, 2014 at 9:58 PM, Chris Angelico wrote: > Responding to your post in different order to the original. > > On Fri, Feb 14, 2014 at 8:59 AM, Ram Rachum wrote: > > If you'd like to bind to a variable only a part of the condition, this > would > > work too: > > > > if x<5 with expensive_computation_0() as x: > > # Do something with x > > Definitely don't like this syntax - while it might be useful to > snapshot part of a condition (I've done it in C plenty of times), this > notation feels clumsy. However... > > > I agree that a non-clunky way to extract variables from conditions with an operator would be nice. Maybe a better syntax would be: if (expensive_computation_0() as x)<5: # Do something with x And likewise for `while` loops, while (expensive_computation_0() as x)<5: # Do something with x > My suggestion: > > > > if expensive_computation_0() as x: > > # Do something with x... > > elif expensive_computation_1() as x: > > # Do something with x... > > elif expensive_computation_2() as x: > > # Do something with x... > > ... this simpler form does look reasonable. The "as" part will *only* > come at the end of the expression, and it *always* applies to the > whole expression, so it's fairly clear. > Agreed, this looks reasonable to me. These are special cases of PEP 379, "Adding an Assignment Expression" ( http://www.python.org/dev/peps/pep-0379/) from 2009, which has been withdrawn. Perhaps it would be better received if restricted to if/while conditions. Nathan -------------- next part -------------- An HTML attachment was scrubbed... URL: From grosser.meister.morti at gmx.net Fri Feb 14 04:29:05 2014 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Fri, 14 Feb 2014 04:29:05 +0100 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: Message-ID: <52FD8D81.40906@gmx.net> Am 2014-02-14 03:58, schrieb Chris Angelico: > Responding to your post in different order to the original. > > On Fri, Feb 14, 2014 at 8:59 AM, Ram Rachum wrote: >> If you'd like to bind to a variable only a part of the condition, this would >> work too: >> >> if x<5 with expensive_computation_0() as x: >> # Do something with x > > Definitely don't like this syntax - while it might be useful to > snapshot part of a condition (I've done it in C plenty of times), this > notation feels clumsy. However... > >> My suggestion: >> >> if expensive_computation_0() as x: >> # Do something with x... >> elif expensive_computation_1() as x: >> # Do something with x... >> elif expensive_computation_2() as x: >> # Do something with x... > > ... this simpler form does look reasonable. The "as" part will *only* > come at the end of the expression, and it *always* applies to the > whole expression, so it's fairly clear. > > There is another cheat you can do, though, and that's to use break or > return instead of an elif chain. Going back to your original: > >> The code paraphrased is this: >> >> if expensive_computation_0(): >> x = expensive_computation_0() >> # Do something with x... >> elif expensive_computation_1(): >> x = expensive_computation_1() >> # Do something with x... >> elif expensive_computation_2(): >> x = expensive_computation_2() >> # Do something with x... > > the alternative would be something like this: > > while "allow_break": > x = expensive_computation_0(): > if x: > # Do something with x... > break > x = expensive_computation_1(): > if x: > # Do something with x... > break > x = expensive_computation_2(): > if x: > # Do something with x... > break > # whatever your 'else' clause would be, if any > break > > Or, using a function instead: > > def allow_return(): > nonlocal everything, you, need > x = expensive_computation_0(): > if x: > # Do something with x... > return > x = expensive_computation_1(): > if x: > # Do something with x... > return > x = expensive_computation_2(): > if x: > # Do something with x... > return > allow_return() > > Both of them are abusing their keywords into gotos, but ultimately, > that's what if/elif/elif is anyway - as soon as you finish one elif, > you goto the end of the block. It's not perfect by any means, but it > works. > Or if "Do something with x" is always the same: x = expensive_computation_0() or expensive_computation_1() or expensive_computation_2() if x: # Do something with x... From jelle.zijlstra at gmail.com Fri Feb 14 04:34:35 2014 From: jelle.zijlstra at gmail.com (Jelle Zijlstra) Date: Thu, 13 Feb 2014 19:34:35 -0800 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: Message-ID: Worth noting that Go has a similar syntax: if x := expensive_computation(); x < 5 { fmt.Println("x is smaller than 5: %d", x) } else { fmt.Println("also have access to x here: %d", x) } 2014-02-13 19:19 GMT-08:00 Nathan Schneider : > > On Thu, Feb 13, 2014 at 9:58 PM, Chris Angelico wrote: >> >> Responding to your post in different order to the original. >> >> On Fri, Feb 14, 2014 at 8:59 AM, Ram Rachum wrote: >> > If you'd like to bind to a variable only a part of the condition, this >> > would >> > work too: >> > >> > if x<5 with expensive_computation_0() as x: >> > # Do something with x >> >> Definitely don't like this syntax - while it might be useful to >> snapshot part of a condition (I've done it in C plenty of times), this >> notation feels clumsy. However... >> >> > > I agree that a non-clunky way to extract variables from conditions with an > operator would be nice. Maybe a better syntax would be: > > if (expensive_computation_0() as x)<5: > > # Do something with x > > And likewise for `while` loops, > > while (expensive_computation_0() as x)<5: > > # Do something with x > >> > My suggestion: >> > >> > if expensive_computation_0() as x: >> > # Do something with x... >> > elif expensive_computation_1() as x: >> > # Do something with x... >> > elif expensive_computation_2() as x: >> > # Do something with x... >> >> ... this simpler form does look reasonable. The "as" part will *only* >> come at the end of the expression, and it *always* applies to the >> whole expression, so it's fairly clear. > > > Agreed, this looks reasonable to me. > > These are special cases of PEP 379, "Adding an Assignment Expression" > (http://www.python.org/dev/peps/pep-0379/) from 2009, which has been > withdrawn. Perhaps it would be better received if restricted to if/while > conditions. > > Nathan > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From ethan at stoneleaf.us Fri Feb 14 04:47:46 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 13 Feb 2014 19:47:46 -0800 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: Message-ID: <52FD91E2.7010602@stoneleaf.us> On 02/13/2014 01:59 PM, Ram Rachum wrote: > Hi everybody, > > Please excuse the recent torrent of crazy ideas :) > > I was reading code of the Shpaml project, trying to make a patch, while I saw code that looked inelegant to me. I > considered how to improve it, but then saw I don't know how to. > > The code paraphrased is this: > > if expensive_computation_0(): > x = expensive_computation_0() > # Do something with x... > elif expensive_computation_1(): > x = expensive_computation_1() > # Do something with x... > elif expensive_computation_2(): > x = expensive_computation_2() > # Do something with x... > > The problem here is that we're doing the expensive computation twice. That's because we first need to check its truth > value, and if it's true we need to actually use the value and do stuff with it. Use a pocket function: def pocket(value=None, _storage=[]): if value is not None: _storage[:] = [value] return _storage[0] and then: if pocket(expensive_computation_0()): x = pocket() # Do something with x... elif pocket(expensive_computation_1()): x = pocket() # Do something with x... elif pocket(expensive_computation_2()): x = pocket() # Do something with x... -- ~Ethan~ From rosuav at gmail.com Fri Feb 14 05:22:59 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 14 Feb 2014 15:22:59 +1100 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: <52FD8D81.40906@gmx.net> References: <52FD8D81.40906@gmx.net> Message-ID: On Fri, Feb 14, 2014 at 2:29 PM, Mathias Panzenb?ck wrote: > Or if "Do something with x" is always the same: > > x = expensive_computation_0() or expensive_computation_1() or > expensive_computation_2() > > if x: > # Do something with x... I would assume that it's not the same... otherwise this method doesn't just obviate the need for comparison-assignment, it also deduplicates a block of code. I think most programmers are smart enough to figure out that that's important :) ChrisA From michelelacchia at gmail.com Fri Feb 14 07:49:02 2014 From: michelelacchia at gmail.com (Michele Lacchia) Date: Fri, 14 Feb 2014 07:49:02 +0100 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: Message-ID: You could gather all the functions in a list and then iterate and break when needed: for func in [f1, f2, f3, f4]: x = func() if x: # bla bla break Sorry for the indentation if it's not right, I'm writing from my phone. Il 14/feb/2014 03:40 "Ram Rachum" ha scritto: > Hi everybody, > > Please excuse the recent torrent of crazy ideas :) > > I was reading code of the Shpaml project, trying to make a patch, while I > saw code that looked inelegant to me. I considered how to improve it, but > then saw I don't know how to. > > The code paraphrased is this: > > if expensive_computation_0(): > x = expensive_computation_0() > # Do something with x... > elif expensive_computation_1(): > x = expensive_computation_1() > # Do something with x... > elif expensive_computation_2(): > x = expensive_computation_2() > # Do something with x... > > The problem here is that we're doing the expensive computation twice. > That's because we first need to check its truth value, and if it's true we > need to actually use the value and do stuff with it. > > One solution would be to calculate it and put it in a variable before > checking truth value. The problem with that is, besides being quite > verbose, it doesn't work with the `elif` lines. You want to calculate the > values only if the `elif` branch is being taken. (i.e. you can't do it > before the `if` because then you might be calculating it needlessly since > maybe the first condition would be true.) > > Obviously this can be "solved", i.e. rewritten in a way that wouldn't call > the expensive operation twice, but the solution would probably be quite > verbose, which is something we don't want. > > My suggestion: > > if expensive_computation_0() as x: > # Do something with x... > elif expensive_computation_1() as x: > # Do something with x... > elif expensive_computation_2() as x: > # Do something with x... > > If you'd like to bind to a variable only a part of the condition, this > would work too: > > if x<5 with expensive_computation_0() as x: > # Do something with x > > What do you think? > > > Ram. > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Fri Feb 14 07:51:02 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 14 Feb 2014 17:51:02 +1100 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: Message-ID: On Fri, Feb 14, 2014 at 5:49 PM, Michele Lacchia wrote: > You could gather all the functions in a list and then iterate and break when > needed: > > for func in [f1, f2, f3, f4]: > x = func() > if x: > # bla bla > break > > Sorry for the indentation if it's not right, I'm writing from my phone. The indentation's fine, but this still assumes that the 'if x' block in each case is the same. ChrisA From denis.spir at gmail.com Fri Feb 14 08:41:23 2014 From: denis.spir at gmail.com (spir) Date: Fri, 14 Feb 2014 08:41:23 +0100 Subject: [Python-ideas] except expression In-Reply-To: <52FD4648.4060603@canterbury.ac.nz> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> <52FD4648.4060603@canterbury.ac.nz> Message-ID: <52FDC8A3.4000409@gmail.com> On 02/13/2014 11:25 PM, Greg Ewing wrote: > spir wrote: >> [By the way, this shows that: >> x = b if cond else a >> should really be: >> x = a else b if cond >> The difference being that the standard case is expressed first, the >> exceptional one being then introduced as an special variant.] > > I don't think it shows that at all. Which is the normal > case and which is the exceptional one depends entirely > on how the condition is expressed. Say it differently: the condition should single out the special case, not the normal one; d From denis.spir at gmail.com Fri Feb 14 09:05:01 2014 From: denis.spir at gmail.com (spir) Date: Fri, 14 Feb 2014 09:05:01 +0100 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: Message-ID: <52FDCE2D.2010903@gmail.com> On 02/13/2014 10:59 PM, Ram Rachum wrote: > Hi everybody, > > Please excuse the recent torrent of crazy ideas :) > > I was reading code of the Shpaml project, trying to make a patch, while I > saw code that looked inelegant to me. I considered how to improve it, but > then saw I don't know how to. > > The code paraphrased is this: > > if expensive_computation_0(): > x = expensive_computation_0() > # Do something with x... > elif expensive_computation_1(): > x = expensive_computation_1() > # Do something with x... > elif expensive_computation_2(): > x = expensive_computation_2() > # Do something with x... Such a pattern is rather common, eg in some kind of hand-baked parsing. (Say you have match funcs returning False if no match, True if match but you don't want/need the result, some "form" having a truth value of True if you need the form; then you are matching a pattern choice). Solution 0: for a little and particular choice: x = expensive_computation_0() if not x: x = expensive_computation_1() if not x: x = expensive_computation_2() # assert(x) # or whatever check Alternative: x = expensive_computation_0() \ or expensive_computation_1() \ or expensive_computation_2() # assert(x) Works due to lazy evaluation of logical operations. Solution 1: for a big choice, or the general case: for comp in computations: x = comp() if x: break # assert(x) However, one must notice that in some way we are here abusing the logical idea of truth value (of a result, specifically) to make it indcate the success of a computation. In other words, we are carrying two pieces of information (result+success) in the same single piece of data. And the loop version only works if all comp's take no param, or the same one(s), or you have a parallel array of param sets (in parsing, all match funcs take source & index). d From denis.spir at gmail.com Fri Feb 14 09:12:45 2014 From: denis.spir at gmail.com (spir) Date: Fri, 14 Feb 2014 09:12:45 +0100 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: Message-ID: <52FDCFFD.5080103@gmail.com> On 02/14/2014 04:19 AM, Nathan Schneider wrote: > On Thu, Feb 13, 2014 at 9:58 PM, Chris Angelico wrote: > >> Responding to your post in different order to the original. >> >> On Fri, Feb 14, 2014 at 8:59 AM, Ram Rachum wrote: >>> If you'd like to bind to a variable only a part of the condition, this >> would >>> work too: >>> >>> if x<5 with expensive_computation_0() as x: >>> # Do something with x >> >> Definitely don't like this syntax - while it might be useful to >> snapshot part of a condition (I've done it in C plenty of times), this >> notation feels clumsy. However... >> >> >> > I agree that a non-clunky way to extract variables from conditions with an > operator would be nice. Maybe a better syntax would be: > > if (expensive_computation_0() as x)<5: > # Do something with x > > And likewise for `while` loops, > > while (expensive_computation_0() as x)<5: > # Do something with x > >> My suggestion: >>> >>> if expensive_computation_0() as x: >>> # Do something with x... >>> elif expensive_computation_1() as x: >>> # Do something with x... >>> elif expensive_computation_2() as x: >>> # Do something with x... >> >> ... this simpler form does look reasonable. The "as" part will *only* >> come at the end of the expression, and it *always* applies to the >> whole expression, so it's fairly clear. >> > > Agreed, this looks reasonable to me. > > These are special cases of PEP 379, "Adding an Assignment Expression" ( > http://www.python.org/dev/peps/pep-0379/) from 2009, which has been > withdrawn. Perhaps it would be better received if restricted to if/while > conditions. > > Nathan Isn't this asking for a python variant of C's X x; if (x = f()) {...} for (x = f()) {...} ? Remember all the critics around such constructs? (IIRC partly, but not only, due to the misuse of '=' to mean assignment; the other part is that it is hard to think right, intuitively the mix of a computation [f()] and an action [assignment] is a single construct; this is also partly why people never check the fake error-results of "action-functions") d From denis.spir at gmail.com Fri Feb 14 09:24:12 2014 From: denis.spir at gmail.com (spir) Date: Fri, 14 Feb 2014 09:24:12 +0100 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: <52FDCE2D.2010903@gmail.com> References: <52FDCE2D.2010903@gmail.com> Message-ID: <52FDD2AC.7020206@gmail.com> On 02/14/2014 09:05 AM, spir wrote: > On 02/13/2014 10:59 PM, Ram Rachum wrote: >> Hi everybody, >> >> Please excuse the recent torrent of crazy ideas :) >> >> I was reading code of the Shpaml project, trying to make a patch, while I >> saw code that looked inelegant to me. I considered how to improve it, but >> then saw I don't know how to. >> >> The code paraphrased is this: >> >> if expensive_computation_0(): >> x = expensive_computation_0() >> # Do something with x... >> elif expensive_computation_1(): >> x = expensive_computation_1() >> # Do something with x... >> elif expensive_computation_2(): >> x = expensive_computation_2() >> # Do something with x... > > Such a pattern is rather common, eg in some kind of hand-baked parsing. (Say you > have match funcs returning False if no match, True if match but you don't > want/need the result, some "form" having a truth value of True if you need the > form; then you are matching a pattern choice). > > Solution 0: for a little and particular choice: > > x = expensive_computation_0() > if not x: > x = expensive_computation_1() > if not x: > x = expensive_computation_2() > # assert(x) # or whatever check > > Alternative: > x = expensive_computation_0() \ > or expensive_computation_1() \ > or expensive_computation_2() > # assert(x) > > Works due to lazy evaluation of logical operations. > > > Solution 1: for a big choice, or the general case: > > for comp in computations: > x = comp() > if x: break > # assert(x) > > Oh, I did not get the action performed is supposed to be different in each case. (Note: you should have used indices 1, 2, 3... for actions as well.) Then, we need a // array of actions. Something like: for i in range(len(comps)): # or zip x = comps[i]() if x: actions[i]() break # assert(x) But it still assumes all computations and all actions take the same input. Else, we're left with unrolling the loop: but how to break outside no loop? The solution may be to export the whole series comps/actions into a func and return from it. def f (): ... g() ... def g (): x = comp1() if x: action1(x) return x = comp2() if x: action2(x) return ... d From rosuav at gmail.com Fri Feb 14 09:36:47 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 14 Feb 2014 19:36:47 +1100 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: <52FDD2AC.7020206@gmail.com> References: <52FDCE2D.2010903@gmail.com> <52FDD2AC.7020206@gmail.com> Message-ID: On Fri, Feb 14, 2014 at 7:24 PM, spir wrote: > But it still assumes all computations and all actions take the same input. > Else, we're left with unrolling the loop: but how to break outside no loop? > The solution may be to export the whole series comps/actions into a func and > return from it. You can break out of a loop that's guaranteed to execute only once anyway. In C, that's sometimes spelled "do {......} while (0);", but in Python you just put a hard break at the end of it. That or 'return' from a function, either way works. ChrisA From g.brandl at gmx.net Fri Feb 14 10:30:16 2014 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 14 Feb 2014 10:30:16 +0100 Subject: [Python-ideas] except expression In-Reply-To: <52FDC8A3.4000409@gmail.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> <52FD4648.4060603@canterbury.ac.nz> <52FDC8A3.4000409@gmail.com> Message-ID: Am 14.02.2014 08:41, schrieb spir: > On 02/13/2014 11:25 PM, Greg Ewing wrote: >> spir wrote: >>> [By the way, this shows that: >>> x = b if cond else a >>> should really be: >>> x = a else b if cond >>> The difference being that the standard case is expressed first, the >>> exceptional one being then introduced as an special variant.] >> >> I don't think it shows that at all. Which is the normal >> case and which is the exceptional one depends entirely >> on how the condition is expressed. > > Say it differently: the condition should single out the special case, not the > normal one; And that is also just your opinion. Georg From steve at pearwood.info Fri Feb 14 10:41:15 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 14 Feb 2014 20:41:15 +1100 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> <52FCF84F.9010801@mrabarnett.plus.com> <52FD101C.4060407@mrabarnett.plus.com> <52FD5955.40008@canterbury.ac.nz> Message-ID: <20140214094115.GA4281@ando> On Fri, Feb 14, 2014 at 01:49:20AM +0200, Ram Rachum wrote about a custom "either" class: > That's nice, but having to import a definition like that would be quite > cumbersome. No more cumbersome than any other import. We don't insist that every itertools function or maths routine be a built-in, and people manage :-) The barrier to a new built-in is higher than the barrier to a new module or function in a module. I'm not aware of any concrete examples where this has happened, but in principle, a really popular function might be moved from a module to built-ins. > Also, this doesn't allow lazy evaluation. That is a bigger problem with the suggestion. -- Steven From tjreedy at udel.edu Fri Feb 14 11:18:55 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 14 Feb 2014 05:18:55 -0500 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: Message-ID: On 2/13/2014 4:59 PM, Ram Rachum wrote: > Hi everybody, > > Please excuse the recent torrent of crazy ideas :) > > I was reading code of the Shpaml project, trying to make a patch, while > I saw code that looked inelegant to me. I considered how to improve it, > but then saw I don't know how to. > > The code paraphrased is this: > > if expensive_computation_0(): > x = expensive_computation_0() > # Do something with x... > elif expensive_computation_1(): > x = expensive_computation_1() > # Do something with x... > elif expensive_computation_2(): > x = expensive_computation_2() > # Do something with x... I do not really understand the fear of indents that would cause one to repeat calculations rather than write the actual logic. x = expensive_computation_0(): if x: # Do something with x... else: x = expensive_computation_1() if x: # Do something with x... else: x = expensive_computation_2() # Do something with x... If the code is already indented so much that 8 more spaces is a burden, then temporarily use 1 space indents. If do_something is the same, I would use 'or' as already posted. Python 'or' expressions are flow-control expression, not just logic expresssion. If there are more nested clauses, a separate function or "while 'fake loop': ... break" is fine. -- Terry Jan Reedy From steve at pearwood.info Fri Feb 14 11:20:54 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 14 Feb 2014 21:20:54 +1100 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> Message-ID: <20140214102053.GB4281@ando> On Thu, Feb 13, 2014 at 10:27:19PM +1100, Chris Angelico wrote: > Generalizing the syntax, I'd see this as: > > operand1 binary-op1 operand2 {and|or} binary-op2 operand3 [...] > The way I see it, there should ideally be no syntactic rule against > using different operators on the two sides: > > input("> ") in legalcommands and not in forbiddencommands > value > 20 or in {3,5,7,11} I would rather be conservative about adding new syntax in this way. It's easier to generalise later than to make it less general. I don't mind chaining "in" or "not in", I mind a bit that one might extend that to other comparisons, and I *really strongly dislike* that one might write something like this without the compiler complaining: x + 23 or * 2 Although we might declare that this is to understood as "x+23 or x*2", I think it's ugly and weird and doesn't read like executable pseudo-code. To me, it feels almost Forth-like (and I like Forth, as Forth, just not in Python code). -- Steven From rosuav at gmail.com Fri Feb 14 11:40:03 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 14 Feb 2014 21:40:03 +1100 Subject: [Python-ideas] a in x or in y In-Reply-To: <20140214102053.GB4281@ando> References: <20140213105716.GA3799@ando> <20140214102053.GB4281@ando> Message-ID: On Fri, Feb 14, 2014 at 9:20 PM, Steven D'Aprano wrote: > I would rather be conservative about adding new syntax in this way. It's > easier to generalise later than to make it less general. I don't mind > chaining "in" or "not in", I mind a bit that one might extend that to > other comparisons, and I *really strongly dislike* that one might write > something like this without the compiler complaining: > > x + 23 or * 2 Maybe, but I find it easier to explain if it's simply "and/or followed by a binary operator" rather than specifically a function of "[not] in". Maybe require that it be only comparison operators? That excludes the example you give (which I agree is insane), and also prevents the ambiguity of + and - in their unary forms, but would allow this: inside = 1 < x < 5 outside = x <= 1 or >= 5 In each case, x is written only once. We already have chained comparisons which implicitly require both conditions; this would allow an either-or without negating all conditions and putting a big fat "not" around the outside of it. ChrisA From abarnert at yahoo.com Fri Feb 14 12:24:10 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 14 Feb 2014 03:24:10 -0800 Subject: [Python-ideas] a in x or in y In-Reply-To: References: <20140213105716.GA3799@ando> <52FCF84F.9010801@mrabarnett.plus.com> <52FD101C.4060407@mrabarnett.plus.com> <20140213211135.GB3799@ando> Message-ID: <7E718AC2-3DD1-421F-8F95-6D4D2EF3A9EE@yahoo.com> On Feb 13, 2014, at 13:20, Haoyi Li wrote: > Ah, I did not think about the laziness. That indeed is a pain since we can't create our own custom lazy operators/methods. > > I would say the correct answer is that we should let people define their own lazy operators, and then define or_in or whatever as a lazy operator/method, but I'm sure others would disagree. Well, first you need to come up with a way to allow people to define new operators in the first place. Because the parser can't know what you're going to define at runtime, you can't really do fancy things like specifying precedence or association direction, and they all have to fit some easily detectable pattern. So, let's pick a straw man: a `foo` b always means foo(a, b), and has higher precedence than all other binary operators, and left associates. As long as foo has to be an identifier, not an arbitrary expression, this would be pretty simple to add to the grammar. To add short circuiting, just do this: a ``foo`` b is the same thing as foo(lambda: a, lambda: b). It has the same precedence as single-backtick operators and left associates. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Fri Feb 14 13:15:56 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 14 Feb 2014 23:15:56 +1100 Subject: [Python-ideas] a in x or in y In-Reply-To: <52FD6484.7060500@stoneleaf.us> References: <20140213105716.GA3799@ando> <52FCF84F.9010801@mrabarnett.plus.com> <52FD101C.4060407@mrabarnett.plus.com> <52FD5955.40008@canterbury.ac.nz> <52FD6484.7060500@stoneleaf.us> Message-ID: <20140214121556.GC4281@ando> On Thu, Feb 13, 2014 at 04:34:12PM -0800, Ethan Furman wrote: > On 02/13/2014 03:49 PM, Ram Rachum wrote: > >On Fri, Feb 14, 2014 at 1:46 AM, Greg Ewing wrote: > >> > >>class either(object): > >> > >> def __init__(self, *args): > >> self.args = args > >> > >> def __contains__(self, x): > >> for a in self.args: > >> if x in a: > >> return True > >> return False [...] > >Also, this doesn't allow lazy evaluation. > > Certainly it does. If the target is in the first thing it returns True at > that point. No, Ram is correct. either() short circuits the `x in a` tests, but it evaluates the individual args eagerly, not lazily. For simplicity, let's just talk about two elements, rather than arbitrary numbers, and compare lazy and non-lazy evaluation. Contrast how Python's short-circuiting `and` works compared to this not-quite equivalent: x and 1/x # 1/x is only evaluated if x is not zero def and_(a, b): if a: return b return a and_(x, 1/x) # 1/x is evaluated before and_ even sees it Because Python's short-circuiting operators are syntax, they can avoid evaluating the operand they don't need. The same doesn't apply to Greg's "either" class, you have to evaluate all the terms first. It can short-circuit calling the __contains__ methods, but that's all. -- Steven From rob.cliffe at btinternet.com Fri Feb 14 13:33:09 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Fri, 14 Feb 2014 12:33:09 +0000 Subject: [Python-ideas] except expression In-Reply-To: <52FD574E.5020403@canterbury.ac.nz> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> Message-ID: <52FE0D05.8070900@btinternet.com> I propose the syntax x = try entries[0] except IndexError: None Advantages: (1) Similar to the current try ... except syntax, so meaning immediately obvious. (2) Allows all the current (and future??) syntax of except clauses: x = try get_it() except IOError as e: e.errno x = try get_it() except (OSError, IOError): None x = try entries[0] except NameError: ProgramError[1] except IndexError: None (3) Unambiguous: in the last example the second "except" traps an IndexError that occurs during the evaluation of "entries[0]", not during the evaluation of "ProgramError[1]" (the latter would be written with a second "try" before "ProgramError[1], not that I would recommend doing it). An "except" refers to the expression following the nearest preceding "try". I.e. there is no "dangling except" problem. (4) Straightforward to parse. The leading "try" immediately tells the parser what to expect. And the "except"s and colons are unambiguous delimiters. Ditto for a human reader. (5) No new keyword, or strained use of an existing keyword, needed. It would be illegal to have a "try" expression without at least one "except", just as is it currently illegal to have a "try" statement without at least one "except" or "finally". Rob Cliffe On 13/02/2014 23:37, Greg Ewing wrote: > Amber Yust wrote: >> Actually. What if we just reused 'try'? >> >> foo = bar() except BazException try 'qux' > > This suggests that trying 'qux' may somehow fail to work, > which is not the intended meaning at all. > From ron3200 at gmail.com Fri Feb 14 16:33:03 2014 From: ron3200 at gmail.com (Ron Adam) Date: Fri, 14 Feb 2014 09:33:03 -0600 Subject: [Python-ideas] combining two threads: switch statements and inline functions In-Reply-To: References: <20140212032919.GP3799@ando> <52FB4915.1040705@gmail.com> <1392249059.83235.YahooMailNeo@web181005.mail.ne1.yahoo.com> Message-ID: On 02/13/2014 07:12 PM, Chris Angelico wrote: > On Fri, Feb 14, 2014 at 12:00 PM, Ron Adam wrote: >> >It should be possibly to define a limited type of function that has a code >> >object which is insert-able into in the byte code, but it would have some >> >pretty limited restrictions. >> > >> > * Have all variables declared as non-local. > By "non-local" do you mean specifically the Python 3 "nonlocal" > keyword, or is it acceptable to declare globals? Yes, the Python 3 "nonlocal". but I took a look at the bytecode, and I think it's not worth doing because you need to define the function in a function with names that match the function it will be used in. The globals case is easier... >>> from dis import dis >>> def foo(): ... global x, y ... return x + y ... >>> dis(foo) 3 0 LOAD_GLOBAL 0 (x) 3 LOAD_GLOBAL 1 (y) 6 BINARY_ADD 7 RETURN_VALUE >>> def bar(x, y): ... return foo() ... >>> dis(bar) 2 0 LOAD_GLOBAL 0 (foo) 3 CALL_FUNCTION 0 (0 positional, 0 keyword pair) 6 RETURN_VALUE In addition to replacing the fist 5 bytecodes here. There may be some index'es that need to be corrected in the surrounding code. Cheers, Ron From ethan at stoneleaf.us Fri Feb 14 15:50:25 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 14 Feb 2014 06:50:25 -0800 Subject: [Python-ideas] a in x or in y In-Reply-To: <20140214121556.GC4281@ando> References: <20140213105716.GA3799@ando> <52FCF84F.9010801@mrabarnett.plus.com> <52FD101C.4060407@mrabarnett.plus.com> <52FD5955.40008@canterbury.ac.nz> <52FD6484.7060500@stoneleaf.us> <20140214121556.GC4281@ando> Message-ID: <52FE2D31.5090405@stoneleaf.us> On 02/14/2014 04:15 AM, Steven D'Aprano wrote: > On Thu, Feb 13, 2014 at 04:34:12PM -0800, Ethan Furman wrote: >> On 02/13/2014 03:49 PM, Ram Rachum wrote: >>> On Fri, Feb 14, 2014 at 1:46 AM, Greg Ewing wrote: >>>> >>>> class either(object): >>>> >>>> def __init__(self, *args): >>>> self.args = args >>>> >>>> def __contains__(self, x): >>>> for a in self.args: >>>> if x in a: >>>> return True >>>> return False > [...] >>> Also, this doesn't allow lazy evaluation. >> >> Certainly it does. If the target is in the first thing it returns True at >> that point. > > No, Ram is correct. either() short circuits the `x in a` tests, but it > evaluates the individual args eagerly, not lazily. Ah, thanks. Ram, my apologies. -- ~Ethan~ From random832 at fastmail.us Fri Feb 14 21:31:19 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Fri, 14 Feb 2014 15:31:19 -0500 Subject: [Python-ideas] if expensive_computation() as x: Message-ID: <1392409879.26793.83563621.50596AD8@webmail.messagingengine.com> Isn't this whole concept just a way of sneaking in assignments-as-expressions? I'd almost say it would be preferable to _actually_ allow assignments as expressions [maybe with an alternate operator like := if we're too worried about people making a mistake in a condition], rather than invent a new kind of scope and come up with rules as to where this new syntax can be used. From steve at pearwood.info Fri Feb 14 21:38:14 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 15 Feb 2014 07:38:14 +1100 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: Message-ID: <20140214203814.GD4281@ando> On Fri, Feb 14, 2014 at 07:49:02AM +0100, Michele Lacchia wrote: > You could gather all the functions in a list and then iterate and break > when needed: > > for func in [f1, f2, f3, f4]: > x = func() > if x: > # bla bla > break To me, that is the cleanest, most obvious solution to avoid repeating yourself. Must nicer than the suggestion to add name binding to arbitrary expressions. The only downside is that it assumes the "bla bla" part is identical for each function. Coming back to Ram's suggestion: > > if expensive_computation_0(): > > x = expensive_computation_0() > > # Do something with x... > > elif expensive_computation_1(): > > x = expensive_computation_1() > > # Do something with x... > > elif expensive_computation_2(): > > x = expensive_computation_2() > > # Do something with x... > > > > The problem here is that we're doing the expensive computation twice. It's not really a language problem. The problem is not the lack of syntax, but the poor way the code is written, avoiding taking advantage of Python's already existing features. Put the code inside a function, assign the expensive computations once each time, and return as needed: def perform_expensive_calculation(): x = expensive_computation_0() if x: # Do something with x... return x = expensive_computation_1() if x: # Do something with x... return This requires no new syntax, and it is easy to understand. If the "do something" parts include returning a value, you can drop the bare returns as well. > > Obviously this can be "solved", i.e. rewritten in a way that wouldn't call > > the expensive operation twice, but the solution would probably be quite > > verbose, which is something we don't want. It's not at all verbose. Apart from the extra returns, which are only needed if you're not otherwise returning from the "do something" parts, it is no more verbose than what you already have. > > My suggestion: > > > > if expensive_computation_0() as x: > > # Do something with x... If we must have an alternative name binding of expressions, I don't mind this too much. But I'm not sold that this is needed. > > If you'd like to bind to a variable only a part of the condition, this > > would work too: > > > > if x<5 with expensive_computation_0() as x: > > # Do something with x I think that's hard to understand, unnecessary and clashes with the existing use of "with". It also leads to the possibility of abuse: x < 5 with (y + 1)/y with z*(z+1) with function(arg) as z as y as x -- Steven From steve at pearwood.info Fri Feb 14 21:48:20 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 15 Feb 2014 07:48:20 +1100 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: Message-ID: <20140214204820.GE4281@ando> On Fri, Feb 14, 2014 at 05:18:55AM -0500, Terry Reedy wrote: > I do not really understand the fear of indents that would cause one to > repeat calculations rather than write the actual logic. > > x = expensive_computation_0(): > if x: > # Do something with x... > else: > x = expensive_computation_1() > if x: > # Do something with x... That's really not very nice looking. It's okay with one or two levels, three at the most, but avoiding that sort of thing is why we have elif in the first place. So I wouldn't call it a *fear* of indents, more an dislike of excessive indentation. -- Steven From rosuav at gmail.com Fri Feb 14 22:06:34 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 15 Feb 2014 08:06:34 +1100 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: <20140214204820.GE4281@ando> References: <20140214204820.GE4281@ando> Message-ID: On Sat, Feb 15, 2014 at 7:48 AM, Steven D'Aprano wrote: > On Fri, Feb 14, 2014 at 05:18:55AM -0500, Terry Reedy wrote: > >> I do not really understand the fear of indents that would cause one to >> repeat calculations rather than write the actual logic. >> >> x = expensive_computation_0(): >> if x: >> # Do something with x... >> else: >> x = expensive_computation_1() >> if x: >> # Do something with x... > > That's really not very nice looking. It's okay with one or two levels, > three at the most, but avoiding that sort of thing is why we have elif > in the first place. So I wouldn't call it a *fear* of indents, more an > dislike of excessive indentation. More to the point, excessive _and inappropriate_ indentation. An if/elif/elif/elif/else chain puts all the conditions at the same level, and all the bodies at the same level (one further in than the conditions). The intention is that they're all peers, not that they're nested inside each other. Maybe the computer physically executes it as a series of nested conditions (more likely, it's a series of conditions with big fat GOTOs to get out of them), but logically and conceptually, it's not that, so it shouldn't be written that way. Since Python, unlike C, doesn't let you assign inside a condition (and C doesn't let you declare inside a condition - nor does C++, though at least there you can declare inside a for loop, which is the most common case), you need to warp your code somewhat around what the language supports; in my opinion, the less warping, the better, which means either a dummy while loop or a nested function. (The nested function might be clearer in a lot of situations, but having to declare too many nonlocals may damage that, which would be a point in favour of while/break. It'd be nice to just say "nonlocal *" meaning "everything that looks local isn't", but that's probably a nightmare for the interpreter.) ChrisA From steve at pearwood.info Fri Feb 14 22:20:14 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 15 Feb 2014 08:20:14 +1100 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: <1392409879.26793.83563621.50596AD8@webmail.messagingengine.com> References: <1392409879.26793.83563621.50596AD8@webmail.messagingengine.com> Message-ID: <20140214212012.GF4281@ando> On Fri, Feb 14, 2014 at 03:31:19PM -0500, random832 at fastmail.us wrote: > Isn't this whole concept just a way of sneaking in > assignments-as-expressions? Certainly is. > I'd almost say it would be preferable to > _actually_ allow assignments as expressions [maybe with an alternate > operator like := if we're too worried about people making a mistake in a > condition], -1 on this. Python is not C and should not emulate it's mistakes, no matter how tempting they seem :-) -- Steven From zuo at chopin.edu.pl Fri Feb 14 22:29:26 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Fri, 14 Feb 2014 22:29:26 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FC9A20.9040303@egenix.com> Message-ID: <432739f63ba844056cc98429f341a156@chopin.edu.pl> 13.02.2014 13:50, Nick Coghlan wrote: > Some of the specific syntactic proposals: > > x = op() except default if Exception > x = op() except default for Exception > x = op() except default from Exception > x = op() except Exception return default > > x = op() except exc.attr if Exception as exc > x = op() except exc.attr for Exception as exc > x = op() except exc.attr from Exception as exc > x = op() except Exception as exc return exc.attr Please append also my proposal from another branch of this thread: > 1. Simple Variant: > > get_it() except (IndexError: None) > > 2. Variant with 'as': > > get_it() except (OSError as exc: exc.errno) > > 3. Variant with multiple exceptions: > > get_it() except (FileNotFound: 0, > OSError as exc: exc.errno, > Exception: None) Cheers. *j From tjreedy at udel.edu Fri Feb 14 23:02:10 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 14 Feb 2014 17:02:10 -0500 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: <20140214204820.GE4281@ando> Message-ID: On 2/14/2014 4:06 PM, Chris Angelico wrote: > On Sat, Feb 15, 2014 at 7:48 AM, Steven D'Aprano wrote: >> On Fri, Feb 14, 2014 at 05:18:55AM -0500, Terry Reedy wrote: >> >>> I do not really understand the fear of indents that would cause one to >>> repeat calculations rather than write the actual logic. In C, where (syntactically insignificant) 8-space tabs are standard, I can, but not in Python where 4 spaces are standard and one can use less. >>> x = expensive_computation_0(): >>> if x: >>> # Do something with x... >>> else: >>> x = expensive_computation_1() >>> if x: >>> # Do something with x... >> >> That's really not very nice looking. It is an accurate representation of the logic, so you are saying that the logic is not nice looking. Ok. > It's okay with one or two levels, three at the most, As I said, but you clipped. > More to the point, excessive _and inappropriate_ indentation. An > if/elif/elif/elif/else chain puts all the conditions at the same > level, and all the bodies at the same level (one further in than the > conditions). The intention is that they're all peers, But they are *not* peers. > not that they're nested inside each other. But logically, they are. > Since Python, unlike C, doesn't let you assign inside a condition The C construction with a side-effect expression as a condition is a bit of a contortion. Its main virtue is avoidance of indents. -- Terry Jan Reedy From random832 at fastmail.us Fri Feb 14 23:10:53 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Fri, 14 Feb 2014 17:10:53 -0500 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: <20140214212012.GF4281@ando> References: <1392409879.26793.83563621.50596AD8@webmail.messagingengine.com> <20140214212012.GF4281@ando> Message-ID: <1392415853.21331.83593701.31EBD0BE@webmail.messagingengine.com> On Fri, Feb 14, 2014, at 16:20, Steven D'Aprano wrote: > -1 on this. Python is not C and should not emulate it's mistakes, no > matter how tempting they seem :-) = as assignment is one of C's mistakes. I propose abandoning it in Python 4000. Absent that, assignments as expressions is no longer obviously a mistake. Other than the specific syntax, the only difference between this proposal and assignment expressions is that this proposal also invents a new form of block scope. From greg.ewing at canterbury.ac.nz Fri Feb 14 23:20:13 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 15 Feb 2014 11:20:13 +1300 Subject: [Python-ideas] except expression In-Reply-To: <52FE0D05.8070900@btinternet.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> Message-ID: <52FE969D.5000002@canterbury.ac.nz> Here's another one: things[i] (except IndexError: 42) This has the advantage of putting the colon inside parens, where it's less likely to get confused with other uses of colons in the same line (by humans, if not by the computer). Also it might be useful to be able to say things.remove(i) (except ValueError: pass) which would be equivalent to things.remove(i) (except ValueError: None) but would read more smoothly in cases where you're not interested in the value of the expression. -- Greg From ethan at stoneleaf.us Fri Feb 14 23:09:10 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 14 Feb 2014 14:09:10 -0800 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: <20140214204820.GE4281@ando> Message-ID: <52FE9406.3060509@stoneleaf.us> On 02/14/2014 02:02 PM, Terry Reedy wrote: > On 2/14/2014 4:06 PM, Chris Angelico wrote: >> >> More to the point, excessive _and inappropriate_ indentation. An >> if/elif/elif/elif/else chain puts all the conditions at the same >> level, and all the bodies at the same level (one further in than the >> conditions). The intention is that they're all peers, > > But they are *not* peers. > >> not that they're nested inside each other. > > But logically, they are. Maybe `logic`ally, but not `human`ly. ;) For me at least, nested implies that to get to B, A has to be true: therefore B is inside A. Otherwise it's simply a matter of prioritizing - LIFO, FIFO, random(), or whatever, and the only requirement to try the next is that the last did not meet our criteria. -- ~Ethan~ From steve at pearwood.info Sat Feb 15 00:58:39 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 15 Feb 2014 10:58:39 +1100 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: <1392415853.21331.83593701.31EBD0BE@webmail.messagingengine.com> References: <1392409879.26793.83563621.50596AD8@webmail.messagingengine.com> <20140214212012.GF4281@ando> <1392415853.21331.83593701.31EBD0BE@webmail.messagingengine.com> Message-ID: <20140214235839.GG4281@ando> On Fri, Feb 14, 2014 at 05:10:53PM -0500, random832 at fastmail.us wrote: > On Fri, Feb 14, 2014, at 16:20, Steven D'Aprano wrote: > > -1 on this. Python is not C and should not emulate it's mistakes, no > > matter how tempting they seem :-) > > = as assignment is one of C's mistakes. I propose abandoning it in > Python 4000. > > Absent that, assignments as expressions is no longer obviously a > mistake. I think it is. Mathematicians have been performing algorithmic manipulations rather similar to programming for thousands of years, and I do not believe that they have anything like assignment as expressions. (Or if they do, they're in advanced fields which I have not come across.) Instead, they either pre- or post-define the variables they need. # pre-define y = something complicated related to x z = function(y + 1/y) # post-define z = function(y + 1/y) where y = something complicated related to x They don't "in-define": z = function(something complicated related to x as y + 1/y) Mathematicians are notorious for inventing user-hostile but terse syntax. If they don't do definitions-as-expressions, there is likely to be a *really* good reason for it. C using = for assignment-as-expression just compounds the problem, it doesn't create the fundamental problem that sticking assignments in the middle of an expression makes the expression hard for human beings to parse. -- Steven From mertz at gnosis.cx Sat Feb 15 01:11:31 2014 From: mertz at gnosis.cx (David Mertz) Date: Fri, 14 Feb 2014 16:11:31 -0800 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: <20140214235839.GG4281@ando> References: <1392409879.26793.83563621.50596AD8@webmail.messagingengine.com> <20140214212012.GF4281@ando> <1392415853.21331.83593701.31EBD0BE@webmail.messagingengine.com> <20140214235839.GG4281@ando> Message-ID: A problem with the "list of expensive functions" style is that not everything we want to do will necessarily be a pure function. I.e. if it is, this is great: fs = [expensive_1, expensive_2, expensive_3] for f in fs: x = f(known, args, here) if x: break But sometimes you want more general expressions that might be expensive. Nonetheless, using while/break gets us the early exit just as well: while True: x = expensive_1() + 1 if x: break x = expensive_2() // 2 if x: break x = expensive_3() % 3 if x: break x = default_value break Honestly that isn't very verbose. And also, while the def/return approach is similar, it *does* require passing in all the relevant lexical elements needed into the function (well, or using a closure), and that's a bit more bookkeeping possibly. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sat Feb 15 01:46:27 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 15 Feb 2014 10:46:27 +1000 Subject: [Python-ideas] except expression In-Reply-To: <432739f63ba844056cc98429f341a156@chopin.edu.pl> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FC9A20.9040303@egenix.com> <432739f63ba844056cc98429f341a156@chopin.edu.pl> Message-ID: On 15 February 2014 07:29, Jan Kaliszewski wrote: > 13.02.2014 13:50, Nick Coghlan wrote: > >> Some of the specific syntactic proposals: >> >> x = op() except default if Exception >> x = op() except default for Exception >> x = op() except default from Exception >> x = op() except Exception return default >> >> x = op() except exc.attr if Exception as exc >> x = op() except exc.attr for Exception as exc >> x = op() except exc.attr from Exception as exc >> x = op() except Exception as exc return exc.attr > > > Please append also my proposal from another branch of this thread There's currently no volunteer to write a PEP - my post was just an illustration of the kinds of things such a PEP would need to consider. Until there is such a volunteer, the many syntax variants will remain scattered throughout the thread, and we'll likely end up rehashing the discussion in a couple of years time. Note that writing a PEP isn't that complicated - the general process is covered in PEP 1, and the PEP editors take care of most of the details of checking that the markup is correct and then actually posting it on python.org (by committing it to the PEPs repo). So, if anyone would like to write up this discussion, it will involve a few things: 1. Having enough time to follow the discussion and update it with new variants and any new open questions that come up 2. Being willing to read old PEPs for similar ideas (notably PEP 308 which added conditional expressions, but also those for yield from, the with statement, etc), and use those as a guide to the kind of things that need to be accounted for in a new syntax proposal 3. Being able to write up the proposal in such a way that it presents a clear rationale (based on the thread - in particular noting the uneven impact the lack of such a feature is having on function API designs), a clear *semantic* proposal (what does the new construct actually do, what is the scope of any name bindings involved, what scope do subexpressions evaluate in, etc), and then a presentation of the (many) syntactic proposals. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From rosuav at gmail.com Sat Feb 15 01:49:31 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 15 Feb 2014 11:49:31 +1100 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: <20140214204820.GE4281@ando> Message-ID: On Sat, Feb 15, 2014 at 9:02 AM, Terry Reedy wrote: >> More to the point, excessive _and inappropriate_ indentation. An >> if/elif/elif/elif/else chain puts all the conditions at the same >> level, and all the bodies at the same level (one further in than the >> conditions). The intention is that they're all peers, > > > But they are *not* peers. > > >> not that they're nested inside each other. > > > But logically, they are. > C has a switch statement that looks like this: switch (expr) { case value1: code1; break; case value2: code2; break; case value3: code3; break; case value4: code4; break; default: code5; break; } Are the case statements peers? The intention is usually that they are. One does not indent each case statement equally to its preceding code block, ever-increasing down the page. What if we rewrite this with nothing but if and else? x = expr; if (x == value1) code1; else if (x == value2) code2; else if (x == value3) code3; else if (x == value4) code4; else code5; Tell me, please, why code4 and code5 are at equal indentation, but all the other code blocks are different. In Python, we could use a dictionary: switch = { value1: lambda: code1, value2: lambda: code2, value3: lambda: code3, value4: lambda: code4, None: lambda: code5 } switch.get(expr, switch[None])() Once again, they're all peers. And if you define all the functions out of line, they'll still be peers, both in definition and in execution. Not one of them depends on any other. There is no sane and logical reason to indent each one further than the others, because *to the programmer* they are peers. And that's my point. No matter how the *computer* sees them, the *programmer* sees them as peers. Adding an additional option to the list should not change the indentation level of any other, because the logical structure of the code hasn't changed. ChrisA From ncoghlan at gmail.com Sat Feb 15 02:12:38 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 15 Feb 2014 11:12:38 +1000 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: <20140214204820.GE4281@ando> Message-ID: On 15 February 2014 10:49, Chris Angelico wrote: > And that's my point. No matter how the *computer* sees them, the > *programmer* sees them as peers. Adding an additional option to the > list should not change the indentation level of any other, because the > logical structure of the code hasn't changed. You snipped the salient point in all this: the relevant case is one where the programmer wants to check *different* things, and *only* wants to calculate them if the earlier things are false. If they were true peers, this approach would work: x = alternative_1() y = alternative_2() z = alternative_3() if x: # Do something with x elif y: # Do something with y elif z: # Do something with z Ram's concern is that the calculation of y and z isn't needed if x is true, and these calculations are considered to expensive to execute unconditionally, so calculating all three up front isn't wanted. The natural shortcircuiting translation of that is nested if statements, not an elif chain, because these *aren't* peers, the programmer actually wants execution of the later calculations to be conditional on the results of the earlier ones: x = alternative_1() if x: # Do something with x else: y = alternative_2() if y: # Do something with y else: z = alternative_3() if z: # Do something with z If the response to this recommendation is "but I need them all to be bound to x, because x gets reused later in the function and these are just different ways of calculating x", then we have *definitely* reached the point where the correct answer is not "make this kind of spaghetti code easier to write", but instead "refactor the function so that calculating x is a clearly distinct named operation": x = calculate_x() And if the objection to *that* is "but I don't like factoring out functions that are only used once", then the appropriate response is to keep looking for a better way to handle one-shot function definitions (along the lines of PEPs 403 and 3150), not attempting to reduce the number of cases where factoring out a one shot function is appropriate by encouraging more spaghetti code. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From rosuav at gmail.com Sat Feb 15 02:22:29 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 15 Feb 2014 12:22:29 +1100 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: <20140214204820.GE4281@ando> Message-ID: On Sat, Feb 15, 2014 at 12:12 PM, Nick Coghlan wrote: > On 15 February 2014 10:49, Chris Angelico wrote: >> And that's my point. No matter how the *computer* sees them, the >> *programmer* sees them as peers. Adding an additional option to the >> list should not change the indentation level of any other, because the >> logical structure of the code hasn't changed. > > You snipped the salient point in all this: the relevant case is one > where the programmer wants to check *different* things, and *only* > wants to calculate them if the earlier things are false. If they were > true peers, this approach would work: > > x = alternative_1() > y = alternative_2() > z = alternative_3() > > if x: > # Do something with x > elif y: > # Do something with y > elif z: > # Do something with z I believe they're still peers; they just happen to be unable to fit into the physical structure of Python that way. Her's a concrete example: You have a series of regular expressions, and you want to process whichever one matches. No line of text is allowed to match more than one regex (to prevent excessive processing), but any line could match any regex. if re1.match(line) as match: # do stuff with the match object elif re2.match(line) as match: # do different stuff, using the matched substrings # etc etc etc Adding another one is as simple as determining its position in the sequence (since earlier matches take precedence over later ones). Removing one is just deleting its elif and corresponding block of code. They are peers. The fact that Python doesn't currently have syntax that allows this *in an elif chain* doesn't change this; if you were to write it as pseudo-code, you would write them as peers. That's why the function-with-return or loop-with-break still looks better, because it structures the code the way that logically makes sense - flat, not nested. ChrisA From ncoghlan at gmail.com Sat Feb 15 02:40:25 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 15 Feb 2014 11:40:25 +1000 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: <20140214204820.GE4281@ando> Message-ID: On 15 February 2014 11:22, Chris Angelico wrote: > On Sat, Feb 15, 2014 at 12:12 PM, Nick Coghlan wrote: >> On 15 February 2014 10:49, Chris Angelico wrote: >>> And that's my point. No matter how the *computer* sees them, the >>> *programmer* sees them as peers. Adding an additional option to the >>> list should not change the indentation level of any other, because the >>> logical structure of the code hasn't changed. >> >> You snipped the salient point in all this: the relevant case is one >> where the programmer wants to check *different* things, and *only* >> wants to calculate them if the earlier things are false. If they were >> true peers, this approach would work: >> >> x = alternative_1() >> y = alternative_2() >> z = alternative_3() >> >> if x: >> # Do something with x >> elif y: >> # Do something with y >> elif z: >> # Do something with z > > I believe they're still peers; they just happen to be unable to fit > into the physical structure of Python that way. Her's a concrete > example: You have a series of regular expressions, and you want to > process whichever one matches. No line of text is allowed to match > more than one regex (to prevent excessive processing), but any line > could match any regex. > > if re1.match(line) as match: > # do stuff with the match object > elif re2.match(line) as match: > # do different stuff, using the matched substrings > # etc etc etc This use case is a large part of why the "first" PyPI project exists and why itertools.first_true has been proposed for inclusion in the stdlib: http://bugs.python.org/issue18652 (a patch would help greatly in getting that to actually happen for 3.5) The re.match case would then be written as: match = first_true(pattern.match for pattern in patterns) If the handling is identical, there's no need for an if/elif chain at all, otherwise the revised if/elif chain is based on the attributes of the match object. > Adding another one is as simple as determining its position in the > sequence (since earlier matches take precedence over later ones). And with first_true, it's similarly simple (especially in the case where the patterns are different, but the handling is predominantly the same, since then it's just a matter of adding a new entry to the list of patterns). > Removing one is just deleting its elif and corresponding block of > code. They are peers. The fact that Python doesn't currently have > syntax that allows this *in an elif chain* doesn't change this; if you > were to write it as pseudo-code, you would write them as peers. That's > why the function-with-return or loop-with-break still looks better, > because it structures the code the way that logically makes sense - > flat, not nested. That doesn't mean embedded assignments are the answer though - they're a sledgehammer solution that is tempting as a workaround for larger structural issues. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Sat Feb 15 02:41:37 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 15 Feb 2014 11:41:37 +1000 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: <20140214204820.GE4281@ando> Message-ID: On 15 February 2014 11:40, Nick Coghlan wrote: > The re.match case would then be written as: > > match = first_true(pattern.match for pattern in patterns) > Oops, that was supposed to be: match = first_true(pattern.match(line) for pattern in patterns) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From rosuav at gmail.com Sat Feb 15 02:43:36 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 15 Feb 2014 12:43:36 +1100 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: <20140214204820.GE4281@ando> Message-ID: On Sat, Feb 15, 2014 at 12:40 PM, Nick Coghlan wrote: > If the handling is identical, there's no need for an if/elif chain at > all, otherwise the revised if/elif chain is based on the attributes of > the match object. If the handling's identical, short-circuiting 'or' will do the job. Assume it's not. >> Removing one is just deleting its elif and corresponding block of >> code. They are peers. The fact that Python doesn't currently have >> syntax that allows this *in an elif chain* doesn't change this; if you >> were to write it as pseudo-code, you would write them as peers. That's >> why the function-with-return or loop-with-break still looks better, >> because it structures the code the way that logically makes sense - >> flat, not nested. > > That doesn't mean embedded assignments are the answer though - they're > a sledgehammer solution that is tempting as a workaround for larger > structural issues. I agree that embedded assignment isn't necessarily the answer. I just think that a solution that has them all at the same indentation level makes more sense than one that nests them inside each other, for exactly the same reason that we have 'elif' in the first place. ChrisA From rosuav at gmail.com Sat Feb 15 05:24:31 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 15 Feb 2014 15:24:31 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FC9A20.9040303@egenix.com> <432739f63ba844056cc98429f341a156@chopin.edu.pl> Message-ID: On Sat, Feb 15, 2014 at 11:46 AM, Nick Coghlan wrote: > There's currently no volunteer to write a PEP - my post was just an > illustration of the kinds of things such a PEP would need to consider. > Until there is such a volunteer, the many syntax variants will remain > scattered throughout the thread, and we'll likely end up rehashing the > discussion in a couple of years time. I'll do up a PEP. If nothing else, it can be rejected and thus retain on record what's been said here. It'll be my first PEP. Let's see how bad a job I make of it :) ChrisA From stephen at xemacs.org Thu Feb 13 05:57:56 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 13 Feb 2014 13:57:56 +0900 Subject: [Python-ideas] except expression In-Reply-To: <85ha83am9x.fsf@benfinney.id.au> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> Message-ID: <87eh37y5gr.fsf@uwakimon.sk.tsukuba.ac.jp> Ben Finney writes: > Ben Finney writes: > > Ram Rachum writes: > > > Here's an idea that would help shortening code. Shortening code is not a win in Python in general. Other things equal, OK, but clarity comes first. ITSM Raymond's comment about simplifying *other* APIs is the right way to get support from python-dev. > > > Allow a ternary expression based on except, like so: > > > > > > first_entry = entries[0] except IndexError else None > > > item = my_queue.get() except queue.Empty else None > > > response_text = request('http://whatever.com').text except HttpError else "Can't access data" Why not spell it the same way as in a try statement? response_text = request('http://whatever.com').text except HttpError as "can't access data" The "as" clause would be required, so "as" always binds to the immediately preceding "except", and iterated it should associate as "(this except ErrorA as that) except ErrorB as other" rather than "this except ErrorA as (that except ErrorB as other)" IMO. I don't think it reads very well, though. The statement form (1) emphasizes the main operation compared to the "try", and (2) suggests that catching an exception is quite a heavy operation compared to execution without an exception, which is true. > > That is more obscure, to my eye, than laying out the control branches: > try: > response_text = request('http://whatever.com').text > except HttpError: > "Can't access data" This has incorrect semantics. The correct semantics would be try: response_text = request('http://whatever.com').text except HttpError: response_text = "Can't access data" I assume. From rosuav at gmail.com Sat Feb 15 06:37:57 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 15 Feb 2014 16:37:57 +1100 Subject: [Python-ideas] except expression In-Reply-To: <87eh37y5gr.fsf@uwakimon.sk.tsukuba.ac.jp> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <87eh37y5gr.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Thu, Feb 13, 2014 at 3:57 PM, Stephen J. Turnbull wrote: > Why not spell it the same way as in a try statement? > > response_text = request('http://whatever.com').text except HttpError as "can't access data" > > The "as" clause would be required, so "as" always binds to the > immediately preceding "except", and iterated it should associate as > "(this except ErrorA as that) except ErrorB as other" rather than > "this except ErrorA as (that except ErrorB as other)" IMO. > > I don't think it reads very well, though. The statement form (1) > emphasizes the main operation compared to the "try", and (2) suggests > that catching an exception is quite a heavy operation compared to > execution without an exception, which is true. Compared to the block try/except syntax, "as" would have to mean "bind the exception to this name before going into the exception-handling block", which is quite different from your proposal. I can mention it in the PEP if you like, though. ChrisA From ben+python at benfinney.id.au Sat Feb 15 07:09:17 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 15 Feb 2014 17:09:17 +1100 Subject: [Python-ideas] except expression References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <87eh37y5gr.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <85r4747vqq.fsf@benfinney.id.au> "Stephen J. Turnbull" writes: > Ben Finney writes: > > try: > > response_text = request('http://whatever.com').text > > except HttpError: > > "Can't access data" > > This has incorrect semantics. The correct semantics would be > > try: > response_text = request('http://whatever.com').text > except HttpError: > response_text = "Can't access data" > > I assume. You assume correctly; I wrote the above code too hastily. Thanks for the correction. -- \ ?Shepherds ? look after their sheep so they can, first, fleece | `\ them and second, turn them into meat. That's much more like the | _o__) priesthood as I know it.? ?Christopher Hitchens, 2008-10-29 | Ben Finney From stephen at xemacs.org Sat Feb 15 09:36:44 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 15 Feb 2014 17:36:44 +0900 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <1392256356.27133.YahooMailNeo@web181003.mail.ne1.yahoo.com> <52FC9A20.9040303@egenix.com> Message-ID: <87y51cwz4z.fsf@uwakimon.sk.tsukuba.ac.jp> Ram Rachum writes: > My favorite syntax so far is: >? ? x except Exception as e pass e.y I think that this syntax is overengineering (regardless of the keywords used). That is, I think it's reasonable to have a syntax like ?>? ? x except Exception pass y for the purpose of passing a specific default (especially because it's a big step toward getting rid of the nuisance default parameters in many APIs). But the extended syntax here allows e.y to be the result of an arbitrary calculation. I think it's better to use the heavier statement syntax in that case. While I'm no authority on "Pythonicity", somehow packing so much into the syntax ? x except Exception as e pass e.y seems to the be kind of thing that the Zen refers to as "complicated". From stephen at xemacs.org Sat Feb 15 10:03:06 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 15 Feb 2014 18:03:06 +0900 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <87eh37y5gr.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87wqgwwxx1.fsf@uwakimon.sk.tsukuba.ac.jp> Chris Angelico writes: > On Thu, Feb 13, 2014 at 3:57 PM, Stephen J. Turnbull wrote: > > Why not spell it the same way as in a try statement? > > > > response_text = request('http://whatever.com').text except HttpError as "can't access data" > Compared to the block try/except syntax, "as" would have to mean "bind > the exception to this name before going into the exception-handling > block", which is quite different from your proposal. True. > I can mention it in the PEP if you like, though. No, please don't. Your point kills it. My main thing is to avoid encouraging use of "may_raise() except ExcType as exc WHATEVER default_value". I don't think an expression should return multiple values through separate channels this way, and as posted elsewhere I think use of exc *in* the expression is "complicated". From rosuav at gmail.com Sat Feb 15 10:26:20 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 15 Feb 2014 20:26:20 +1100 Subject: [Python-ideas] except expression In-Reply-To: <87wqgwwxx1.fsf@uwakimon.sk.tsukuba.ac.jp> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <87eh37y5gr.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqgwwxx1.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Sat, Feb 15, 2014 at 8:03 PM, Stephen J. Turnbull wrote: > My main thing is to avoid encouraging use of "may_raise() except > ExcType as exc WHATEVER default_value". I don't think an expression > should return multiple values through separate channels this way, and > as posted elsewhere I think use of exc *in* the expression is > "complicated". Understood, but I can well imagine there'll be plenty of cases where it makes sense to fold "return value or exception message" into a single value, possibly including a tag of some sort. ret = urlopen(addr) except HTTPError as e pass "Oops - "+e.reason So it makes good sense to grab the exception and use it in the expression. Anyway, the PEP's written and submitted to the pep editors, so as soon as something's posted, the bikeshedding can begin anew :) ChrisA From rosuav at gmail.com Sat Feb 15 12:56:32 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 15 Feb 2014 22:56:32 +1100 Subject: [Python-ideas] Better repr() for sentinel objects Message-ID: A common pattern, both in the stdlib and in user code, is the dedicated sentinel object to recognize an omitted argument: no_timeout = object() def get_data(source, timeout=no_timeout): if timeout is not no_timeout: source.set_alarm(timeout) return source.pop() This is a bit unclear in the function signature, as seen in help(): """ Help on function get_data in module __main__: get_data(source, timeout=) """ The stdlib does this: """ Help on function create_connection in module socket: create_connection(address, timeout=, source_address=None) """ # chomp docstring """ Help on function urlopen in module urllib.request: urlopen(url, data=None, timeout=, *, cafile=None, capath=None, cadefault=False) """ # no docstring to chomp, actually It's not particularly useful to get the object's address. Proposal: A self-documenting Sentinel class which serves the exact same purpose. class Sentinel: def __init__(self, desc): self.desc = desc def __repr__(self): return "Sentinel(" + repr(self.desc) + ")" This can then be used just like object(), only it retains something for the benefit of its repr: no_timeout = Sentinel("No timeout") # Same function definition """ Help on function get_data in module __main__: get_data(source, timeout=Sentinel('No timeout')) """ I don't know how this interacts with Argument Clinic and C-written functions. If there's something that's being done for those that would make sentinels come out a particular way in their help() info, ideally this should be displayed the same way (or at least similarly). Thoughts? ChrisA From fuzzyman at gmail.com Sat Feb 15 13:37:39 2014 From: fuzzyman at gmail.com (Michael Foord) Date: Sat, 15 Feb 2014 12:37:39 +0000 Subject: [Python-ideas] Better repr() for sentinel objects In-Reply-To: References: Message-ID: On 15 February 2014 11:56, Chris Angelico wrote: > A common pattern, both in the stdlib and in user code, is the > dedicated sentinel object to recognize an omitted argument: > > no_timeout = object() > def get_data(source, timeout=no_timeout): > if timeout is not no_timeout: > source.set_alarm(timeout) > return source.pop() > > This is a bit unclear in the function signature, as seen in help(): > > """ > Help on function get_data in module __main__: > > get_data(source, timeout=) > """ > > The stdlib does this: > """ > Help on function create_connection in module socket: > > create_connection(address, timeout=, > source_address=None) > """ # chomp docstring > > """ > Help on function urlopen in module urllib.request: > > urlopen(url, data=None, timeout=, *, > cafile=None, capath=None, cadefault=False) > """ # no docstring to chomp, actually > > It's not particularly useful to get the object's address. Proposal: A > self-documenting Sentinel class which serves the exact same purpose. > > class Sentinel: > def __init__(self, desc): > self.desc = desc > def __repr__(self): > return "Sentinel(" + repr(self.desc) + ")" > > This can then be used just like object(), only it retains something > for the benefit of its repr: > > no_timeout = Sentinel("No timeout") > # Same function definition > """ > Help on function get_data in module __main__: > > get_data(source, timeout=Sentinel('No timeout')) > """ > > I don't know how this interacts with Argument Clinic and C-written > functions. If there's something that's being done for those that would > make sentinels come out a particular way in their help() info, ideally > this should be displayed the same way (or at least similarly). > > There's always this: Python 3.3.4 (v3.3.4:7ff62415e426, Feb 9 2014, 00:29:34) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from unittest.mock import sentinel >>> sentinel.FOO sentinel.FOO >>> sentinel.FOO is sentinel.FOO True >>> sentinel.BarBamBaz sentinel.BarBamBaz Michael > Thoughts? > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Sat Feb 15 14:54:41 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 16 Feb 2014 00:54:41 +1100 Subject: [Python-ideas] Better repr() for sentinel objects In-Reply-To: References: Message-ID: On Sat, Feb 15, 2014 at 11:37 PM, Michael Foord wrote: > There's always this: > >>>> from unittest.mock import sentinel >>>> sentinel.FOO > sentinel.FOO Looks reasonable. The implementation is pretty much the same as I had, with the additional feature of the object with __getattr__ that generates and caches them. I wouldn't like to depend on unittest, but if that could be lifted out and put into a more plausible place (and then unittest.mock could import it from there, which would mean that nothing visible would change), it'd work. Downside: The names must be unique, so they can't be as descriptive. (If two functions want to use a "No timeout" sentinel, they'd either have to go for unique names, or be sharing a sentinel, which is probably not a good thing.) I suppose the question is: Is it a good thing for callers to be able to invoke the default? Currently, urllib.request.urlopen consciously calls up the default-sentinel for socket.create_connection; if that were done as "sentinel.NO_TIMEOUT", then by definition everyone who types that *will* get the effect of omitting the argument. This is a significantly bigger change than simply "hey wouldn't it be nice if the repr() were more readable", though. ChrisA From skreft at gmail.com Sat Feb 15 16:40:34 2014 From: skreft at gmail.com (Sebastian Kreft) Date: Sat, 15 Feb 2014 16:40:34 +0100 Subject: [Python-ideas] Enhance exceptions by attaching some more information to them Message-ID: More than once I've been in a situation where I wish that some of the stdlib exceptions had a better message or some more information to help me diagnose the problem. For example: a = [1, 2, 3, 4, 5] a[5] IndexError: list index out of range In this case there's no reference to neither the length of the array nor to the offending index. I'm of the idea that we could extend the exceptions by adding some more information to them, so 3rd party libraries could use them for debugging/logging. For example, one such use case would be to have a modified test runner, that in case of exceptions automatically prints some debug information. Another would be a logger system that in case of an exception also logs some debug info that could be relevant to understand and solve the issue. I propose extending (at least) the following exceptions with the following attributes: KeyError: key, object IndexError: index, object AttributeError: attribute, object NameError: name Of course that populating these attributes could be controlled by a flag. I know that some of this information is already present in some exceptions, depending on the context. However, I propose adding these attributes, as in this way a tool could easily and reliably extract the information and work with it, as opposed to have to parse the huge variety of different messages there are. For the first use case mentioned above I have a working prototype, although it does not use this proposal, but a series of hacks (I'm modifying the bytecode to save a reference to the key and object :() and parsing of the exception messages. But I want to share what the output of such a tool could be. ====================================================================== ERROR: test_attribute (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py", line 18, in test_attribute AttributeError: 'str' object has no attribute 'Lower'. Did you mean 'islower', 'lower'? Debug info: Object: '' Type: ====================================================================== ERROR: test_index (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py", line 6, in test_index IndexError: list index out of range Debug info: Object: [1, 2] Object len: 2 Index: 2 ====================================================================== ERROR: test_key (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py", line 10, in test_key KeyError_: 'fooo', did you mean 'foo'? Debug info: Object: {'foo': 1} Key: 'fooo' ====================================================================== ERROR: test_name (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py", line 14, in test_name NameError: global name 'fooo' is not defined. Did you mean 'foo'? ---------------------------------------------------------------------- Ran 4 tests in 0.005s -- Sebastian Kreft -------------- next part -------------- An HTML attachment was scrubbed... URL: From offline at offby1.net Sat Feb 15 18:25:58 2014 From: offline at offby1.net (Chris Rose) Date: Sat, 15 Feb 2014 09:25:58 -0800 Subject: [Python-ideas] A suggested feature for difflib Message-ID: Right now, any alternative SequenceMatcher implementation in difflib requires that the implementor also reimplement the diff formatting methods (unified_diff and context_diff). I propose something along these lines (patch to follow if the idea is sound): class Differ: def __init__(self, linejunk=None, charjunk=None, sequence_matcher_factory=SequenceMatcher): # and so on def compare(self, a, b): cruncher = self.sequence_matcher_factory(self.linejunk, a, b) # and so on The same general idea with unified_diff, ndiff, and context_diff Basically, there are two obvious points of extension in difflib: The formatting of the output opcodes, and the method used to find sequences. The latter is not easily extended while reusing the former, and I'd like to change that. Note that I'm asking because I'd like to do the work, not because I want someone else to do it. Is this a reasonable idea? -- Chris R. ====== Not to be taken literally, internally, or seriously. Twitter: http://twitter.com/offby1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Sat Feb 15 19:11:39 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 16 Feb 2014 05:11:39 +1100 Subject: [Python-ideas] except expression In-Reply-To: <52FE969D.5000002@canterbury.ac.nz> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> Message-ID: <20140215181136.GH4281@ando> On Sat, Feb 15, 2014 at 11:20:13AM +1300, Greg Ewing wrote: > Here's another one: > > things[i] (except IndexError: 42) I believe Jan Kaliszewski independently came up with the same syntax earlier, which is a good sign. Two people suggesting the same thing is promising. Jan also suggested that this syntax allows binding the exception: things[i] (except IndexError as exc: exc.args) and multiple except terms: things[i] (except IndexError as exc: exc.args, except NameError: "missing", except KeyError: None, ) One might also catch multiple exceptions in a single term: things[i] (except IndexError,KeyError as exc: exc.args) It's a tiny bit unusual to have a colon that doesn't introduce a block, but only a tiny bit. There is precedent: lambda x, y: (x+y)/(x*y) This suggests that perhaps the parentheses aren't needed unless you have multiple except parts: things[i] (except IndexError, KeyError as exc: exc.args) things[i] except IndexError, KeyError as exc: exc.args but I think they ought to be compulsory if you use multiple excepts. And of course they are useful for continuing over multiple lines. I think this syntax looks good when used in compound expressions: mylist = [23, lambda x: x+1, things[i] except IndexError: 42, ""] result = function(a, b except NameError: "undefined", c) result = function(a, b, c) except ValueError: float('nan') if something(x) except TypeError: None: block Dicts are problematic. Should the dict colon bind more or less strongly than the except colon? I suggest we follow the same rule as for lambda: adict = {lambda x: x+1: "value", key+1 except TypeError: 23: "value", } Here is a torture-test for the syntax: can we combine it with an if-expression? I think we can, although you may need parentheses to disambiguate the expression: something(x) (except TypeError: a if condition else b) ((something(x) if x else other(x)) (except ValueError: -1) something(x) if x else (other(x) except ValueError: -1) Trying to do too much in a single expression is never exactly *beautiful*, but it can be read. This does look a tiny bit like a function call, especially if you delete the space between the leading expression and the opening bracket: # ugly, don't do this things[i](except IndexError: 42) but that's not actually ambiguous, since the keyword except cannot be an argument to a function. I like this. I think this is the first non-sucky syntax I've seen (sorry to everyone who proposed sucky syntax *wink*). > This has the advantage of putting the colon inside parens, > where it's less likely to get confused with other uses of > colons in the same line (by humans, if not by the computer). > > Also it might be useful to be able to say > > things.remove(i) (except ValueError: pass) > > which would be equivalent to > > things.remove(i) (except ValueError: None) > > but would read more smoothly in cases where you're not > interested in the value of the expression. Certainly not! pass implies that *no return result is generated at all*, which is not possible in Python. Returning None is the right thing to do. -- Steven From flying-sheep at web.de Sat Feb 15 20:24:09 2014 From: flying-sheep at web.de (Philipp A.) Date: Sat, 15 Feb 2014 20:24:09 +0100 Subject: [Python-ideas] except expression In-Reply-To: <20140215181136.GH4281@ando> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> Message-ID: hmm i still don?t like the colon. everywhere else in python, it means ?new block follows? and i?m sure nobody here wants to allow the following? value = hovercraft() except EelsException: remove_eels() no_eels #this is assigned to ?value? if we have too many eels i like it in coffeescript and scala, but it?s not used in python, so we shouldn?t introduce it. neither should we introduce a colon that *can?t* be followed by a block. if we wanted the colon in any case, we?d need to do: winner = germans_win() except EurekaException: return greeks value = hovercraft() except EelsException: remove_eels() return no_eels but i?m still partial to stomach_content = eat() except ThinMintError pass explode() 2014-02-15 19:11 GMT+01:00 Steven D'Aprano : > On Sat, Feb 15, 2014 at 11:20:13AM +1300, Greg Ewing wrote: > > Here's another one: > > > > things[i] (except IndexError: 42) > > I believe Jan Kaliszewski independently came up with the same syntax > earlier, which is a good sign. Two people suggesting the same thing is > promising. > > Jan also suggested that this syntax allows binding the exception: > > things[i] (except IndexError as exc: exc.args) > > and multiple except terms: > > things[i] (except IndexError as exc: exc.args, > except NameError: "missing", > except KeyError: None, > ) > > > One might also catch multiple exceptions in a single term: > > things[i] (except IndexError,KeyError as exc: exc.args) > > > It's a tiny bit unusual to have a colon that doesn't introduce a block, > but only a tiny bit. There is precedent: > > lambda x, y: (x+y)/(x*y) > > > This suggests that perhaps the parentheses aren't needed unless you have > multiple except parts: > > things[i] (except IndexError, KeyError as exc: exc.args) > things[i] except IndexError, KeyError as exc: exc.args > > > but I think they ought to be compulsory if you use multiple excepts. And > of course they are useful for continuing over multiple lines. > > I think this syntax looks good when used in compound expressions: > > mylist = [23, lambda x: x+1, things[i] except IndexError: 42, ""] > > result = function(a, b except NameError: "undefined", c) > > result = function(a, b, c) except ValueError: float('nan') > > if something(x) except TypeError: None: > block > > > Dicts are problematic. Should the dict colon bind more or less strongly > than the except colon? I suggest we follow the same rule as for lambda: > > adict = {lambda x: x+1: "value", > key+1 except TypeError: 23: "value", > } > > > Here is a torture-test for the syntax: can we combine it with an > if-expression? I think we can, although you may need parentheses to > disambiguate the expression: > > something(x) (except TypeError: a if condition else b) > > ((something(x) if x else other(x)) (except ValueError: -1) > > something(x) if x else (other(x) except ValueError: -1) > > > Trying to do too much in a single expression is never exactly > *beautiful*, but it can be read. > > This does look a tiny bit like a function call, especially if you delete > the space between the leading expression and the opening bracket: > > # ugly, don't do this > things[i](except IndexError: 42) > > but that's not actually ambiguous, since the keyword except cannot be an > argument to a function. > > I like this. I think this is the first non-sucky syntax I've seen (sorry > to everyone who proposed sucky syntax *wink*). > > > > > This has the advantage of putting the colon inside parens, > > where it's less likely to get confused with other uses of > > colons in the same line (by humans, if not by the computer). > > > > Also it might be useful to be able to say > > > > things.remove(i) (except ValueError: pass) > > > > which would be equivalent to > > > > things.remove(i) (except ValueError: None) > > > > but would read more smoothly in cases where you're not > > interested in the value of the expression. > > Certainly not! pass implies that *no return result is generated at all*, > which is not possible in Python. Returning None is the right thing to > do. > > > > -- > Steven > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bruce at leapyear.org Sat Feb 15 20:28:42 2014 From: bruce at leapyear.org (Bruce Leban) Date: Sat, 15 Feb 2014 11:28:42 -0800 Subject: [Python-ideas] Better repr() for sentinel objects In-Reply-To: References: Message-ID: A class like this is useful although I've called it NamedObject (by analogy to NamedTuple). It's useful in contexts other than as a sentinel, for example to represent deleted values, but the fundamental aspects are that it's a unique object that has a name. I've done two different things in my case: (1) add __setattr__ so you can't set values on this object (to prevent someone that gets the object from operating on it as if it's some other object). (2) printing str(descr) not repr(descr). Since the standard use case is passing a string to the constructor, it's not valuable for it to print quotes around it. I also use <> instead of NamedObject() but I don't care about that distinction. >>> a = NamedObject('DELETED'), NamedObject('DELETED') >>> a (, ) >>> a[0] == a[1] False Note the last line: this is different from how unittest.mock.sentinel works. I find both cases useful (and particularly find the unittest.mock.sentinel version more useful in unittests). Writing this up I suppose UniqueObject might be a better name for this than NamedObject. --- Bruce Learn how hackers think: http://j.mp/gruyere-security On Sat, Feb 15, 2014 at 3:56 AM, Chris Angelico wrote: > A common pattern, both in the stdlib and in user code, is the > dedicated sentinel object to recognize an omitted argument: > > no_timeout = object() > def get_data(source, timeout=no_timeout): > if timeout is not no_timeout: > source.set_alarm(timeout) > return source.pop() > > This is a bit unclear in the function signature, as seen in help(): > > """ > Help on function get_data in module __main__: > > get_data(source, timeout=) > """ > > The stdlib does this: > """ > Help on function create_connection in module socket: > > create_connection(address, timeout=, > source_address=None) > """ # chomp docstring > > """ > Help on function urlopen in module urllib.request: > > urlopen(url, data=None, timeout=, *, > cafile=None, capath=None, cadefault=False) > """ # no docstring to chomp, actually > > It's not particularly useful to get the object's address. Proposal: A > self-documenting Sentinel class which serves the exact same purpose. > > class Sentinel: > def __init__(self, desc): > self.desc = desc > def __repr__(self): > return "Sentinel(" + repr(self.desc) + ")" > > This can then be used just like object(), only it retains something > for the benefit of its repr: > > no_timeout = Sentinel("No timeout") > # Same function definition > """ > Help on function get_data in module __main__: > > get_data(source, timeout=Sentinel('No timeout')) > """ > > I don't know how this interacts with Argument Clinic and C-written > functions. If there's something that's being done for those that would > make sentinels come out a particular way in their help() info, ideally > this should be displayed the same way (or at least similarly). > > Thoughts? > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amber.yust at gmail.com Sat Feb 15 20:38:06 2014 From: amber.yust at gmail.com (Amber Yust) Date: Sat, 15 Feb 2014 11:38:06 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> Message-ID: Not everywhere. As previously mentioned, lambdas use a colon which is followed by an expression, not a block. On Feb 15, 2014 2:24 PM, "Philipp A." wrote: > hmm i still don?t like the colon. everywhere else in python, it means ?new > block follows? > > and i?m sure nobody here wants to allow the following? > > value = hovercraft() except EelsException: > remove_eels() > no_eels #this is assigned to ?value? if we have too many eels > > i like it in coffeescript and scala, but it?s not used in python, so we > shouldn?t introduce it. > > neither should we introduce a colon that *can?t* be followed by a block. > > if we wanted the colon in any case, we?d need to do: > > winner = germans_win() except EurekaException: return greeks > > value = hovercraft() except EelsException: > remove_eels() > return no_eels > > but i?m still partial to stomach_content = eat() except ThinMintError > pass explode() > > > 2014-02-15 19:11 GMT+01:00 Steven D'Aprano : > >> On Sat, Feb 15, 2014 at 11:20:13AM +1300, Greg Ewing wrote: >> > Here's another one: >> > >> > things[i] (except IndexError: 42) >> >> I believe Jan Kaliszewski independently came up with the same syntax >> earlier, which is a good sign. Two people suggesting the same thing is >> promising. >> >> Jan also suggested that this syntax allows binding the exception: >> >> things[i] (except IndexError as exc: exc.args) >> >> and multiple except terms: >> >> things[i] (except IndexError as exc: exc.args, >> except NameError: "missing", >> except KeyError: None, >> ) >> >> >> One might also catch multiple exceptions in a single term: >> >> things[i] (except IndexError,KeyError as exc: exc.args) >> >> >> It's a tiny bit unusual to have a colon that doesn't introduce a block, >> but only a tiny bit. There is precedent: >> >> lambda x, y: (x+y)/(x*y) >> >> >> This suggests that perhaps the parentheses aren't needed unless you have >> multiple except parts: >> >> things[i] (except IndexError, KeyError as exc: exc.args) >> things[i] except IndexError, KeyError as exc: exc.args >> >> >> but I think they ought to be compulsory if you use multiple excepts. And >> of course they are useful for continuing over multiple lines. >> >> I think this syntax looks good when used in compound expressions: >> >> mylist = [23, lambda x: x+1, things[i] except IndexError: 42, ""] >> >> result = function(a, b except NameError: "undefined", c) >> >> result = function(a, b, c) except ValueError: float('nan') >> >> if something(x) except TypeError: None: >> block >> >> >> Dicts are problematic. Should the dict colon bind more or less strongly >> than the except colon? I suggest we follow the same rule as for lambda: >> >> adict = {lambda x: x+1: "value", >> key+1 except TypeError: 23: "value", >> } >> >> >> Here is a torture-test for the syntax: can we combine it with an >> if-expression? I think we can, although you may need parentheses to >> disambiguate the expression: >> >> something(x) (except TypeError: a if condition else b) >> >> ((something(x) if x else other(x)) (except ValueError: -1) >> >> something(x) if x else (other(x) except ValueError: -1) >> >> >> Trying to do too much in a single expression is never exactly >> *beautiful*, but it can be read. >> >> This does look a tiny bit like a function call, especially if you delete >> the space between the leading expression and the opening bracket: >> >> # ugly, don't do this >> things[i](except IndexError: 42) >> >> but that's not actually ambiguous, since the keyword except cannot be an >> argument to a function. >> >> I like this. I think this is the first non-sucky syntax I've seen (sorry >> to everyone who proposed sucky syntax *wink*). >> >> >> >> > This has the advantage of putting the colon inside parens, >> > where it's less likely to get confused with other uses of >> > colons in the same line (by humans, if not by the computer). >> > >> > Also it might be useful to be able to say >> > >> > things.remove(i) (except ValueError: pass) >> > >> > which would be equivalent to >> > >> > things.remove(i) (except ValueError: None) >> > >> > but would read more smoothly in cases where you're not >> > interested in the value of the expression. >> >> Certainly not! pass implies that *no return result is generated at all*, >> which is not possible in Python. Returning None is the right thing to >> do. >> >> >> >> -- >> Steven >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Sat Feb 15 20:57:31 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 16 Feb 2014 06:57:31 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> Message-ID: <20140215195731.GI4281@ando> On Sat, Feb 15, 2014 at 08:24:09PM +0100, Philipp A. wrote: > hmm i still don?t like the colon. everywhere else in python, it means ?new > block follows? That is incorrect. Did you read my full post? I specifically mentioned both dicts and lambda, both of which use colons in expressions, and don't start a new block. > and i?m sure nobody here wants to allow the following? > > value = hovercraft() except EelsException: > remove_eels() > no_eels #this is assigned to ?value? if we have too many eels That's not the suggested syntax. To start with, if you want to spread the expression over multiple lines, you need either line continuation backslashes, or round brackets: value = hovercraft() (except EelsException: remove_eels() ) Secondly, you then place an unexpected "no_eels" statement after the expression. That will be a syntax error. > i like it in coffeescript and scala, but it?s not used in python, so we > shouldn?t introduce it. What is used in coffeescript and scala? > neither should we introduce a colon that *can?t* be followed by a block. Dicts, lambda. > if we wanted the colon in any case, we?d need to do: > > winner = germans_win() except EurekaException: return greeks Certainly not. Do we write this? mydict = {1: return 'a', 2: return 'b'} > value = hovercraft() except EelsException: > remove_eels() > return no_eels This is not the suggested syntax. I suggest you read my post again, and notice that this is an *expression*. That is the whole point of the thread! If you want a block made up of multiple statements, use a try...except statement. > but i?m still partial to stomach_content = eat() except ThinMintError pass > explode() "pass" is a placeholder null statement, it has no relevance to try...except. You might just as well sensibly say stomach_content = eat() except ThinMintError import explode() stomach_content = eat() except ThinMintError del explode() stomach_content = eat() except ThinMintError class explode() stomach_content = eat() except ThinMintError def explode() stomach_content = eat() except ThinMintError pass explode() In all of these cases, I have picked a random keyword and just tossed it into the expression. None of them make any sense. -- Steven From python at mrabarnett.plus.com Sat Feb 15 22:08:15 2014 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 15 Feb 2014 21:08:15 +0000 Subject: [Python-ideas] except expression In-Reply-To: <20140215195731.GI4281@ando> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <20140215195731.GI4281@ando> Message-ID: <52FFD73F.9010204@mrabarnett.plus.com> On 2014-02-15 19:57, Steven D'Aprano wrote: > On Sat, Feb 15, 2014 at 08:24:09PM +0100, Philipp A. wrote: >> hmm i still don?t like the colon. everywhere else in python, it means ?new >> block follows? > > That is incorrect. Did you read my full post? I specifically mentioned > both dicts and lambda, both of which use colons in expressions, and > don't start a new block. > It's also used in slices. > >> and i?m sure nobody here wants to allow the following? >> >> value = hovercraft() except EelsException: >> remove_eels() >> no_eels #this is assigned to ?value? if we have too many eels > > That's not the suggested syntax. > > To start with, if you want to spread the expression over multiple lines, > you need either line continuation backslashes, or round brackets: > > value = hovercraft() (except EelsException: > remove_eels() > ) > > > Secondly, you then place an unexpected "no_eels" statement after the > expression. That will be a syntax error. > > >> i like it in coffeescript and scala, but it?s not used in python, so we >> shouldn?t introduce it. > > What is used in coffeescript and scala? > > >> neither should we introduce a colon that *can?t* be followed by a block. > > Dicts, lambda. > > >> if we wanted the colon in any case, we?d need to do: >> >> winner = germans_win() except EurekaException: return greeks > > Certainly not. Do we write this? > > mydict = {1: return 'a', 2: return 'b'} > > >> value = hovercraft() except EelsException: >> remove_eels() >> return no_eels > > This is not the suggested syntax. I suggest you read my post again, and > notice that this is an *expression*. That is the whole point of the > thread! If you want a block made up of multiple statements, use a > try...except statement. > > >> but i?m still partial to stomach_content = eat() except ThinMintError pass >> explode() > > "pass" is a placeholder null statement, it has no relevance to > try...except. You might just as well sensibly say > > stomach_content = eat() except ThinMintError import explode() > stomach_content = eat() except ThinMintError del explode() > stomach_content = eat() except ThinMintError class explode() > stomach_content = eat() except ThinMintError def explode() > stomach_content = eat() except ThinMintError pass explode() > > In all of these cases, I have picked a random keyword and just tossed it > into the expression. None of them make any sense. > From bruce at leapyear.org Sat Feb 15 22:08:24 2014 From: bruce at leapyear.org (Bruce Leban) Date: Sat, 15 Feb 2014 13:08:24 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <87eh37y5gr.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqgwwxx1.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: While I'm generally positive about the idea of an except expression (and favor A except E pass B over any of the alternatives), I think it's worth looking in a different direction. One of the motivations of this idea is that there are some use cases that are particularly inconvenient. Another is the proliferation of APIs accepting default values. So instead of a general solution, let me suggest an alternative that addresses specific use cases. I'll write these in terms of an except expression so that they can be directly compared head to head. a[?b] <=> a[b] except (IndexError, TypeError)pass None a.?b <=> a.b except AttributeError pass None a(?b) <=> a(b) except Exception pass None a or? b <=> (a except Exception pass None) or b a and? b <=> (a except Exception pass None) and b del? a.b <=> try: del a.b except AtributeError: pass Of course you can chain these: a[?b][?c] a.?b.?c a or? b or? c Pros/Cons: - no option to specify replacement value on an exception - no option to specify exceptions to catch - doesn't specifically solve the api default value issue -- and the option for that case a(?b) catches all exceptions rather than specific ones --- Bruce Learn how hackers think: http://j.mp/gruyere-security Note: the syntax could just as easily be a?[b], a?.b, etc. -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Sat Feb 15 22:11:41 2014 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 15 Feb 2014 21:11:41 +0000 Subject: [Python-ideas] except expression In-Reply-To: <52FFD73F.9010204@mrabarnett.plus.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <20140215195731.GI4281@ando> <52FFD73F.9010204@mrabarnett.plus.com> Message-ID: <52FFD80D.1010406@mrabarnett.plus.com> On 2014-02-15 21:08, MRAB wrote: > On 2014-02-15 19:57, Steven D'Aprano wrote: >> On Sat, Feb 15, 2014 at 08:24:09PM +0100, Philipp A. wrote: >>> hmm i still don?t like the colon. everywhere else in python, it means ?new >>> block follows? >> >> That is incorrect. Did you read my full post? I specifically mentioned >> both dicts and lambda, both of which use colons in expressions, and >> don't start a new block. >> > It's also used in slices. > [snip] I wonder if the reply is going to be: "Well, _apart_ from dicts, lambdas, and slices, everywhere else in Python...". :-) From amber.yust at gmail.com Sat Feb 15 23:22:15 2014 From: amber.yust at gmail.com (Amber Yust) Date: Sat, 15 Feb 2014 14:22:15 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <85ob2bamy3.fsf@benfinney.id.au> <85ha83am9x.fsf@benfinney.id.au> <87eh37y5gr.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqgwwxx1.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: I don't think adding multiple elements of syntax for an incomplete solution makes sense. On Feb 15, 2014 4:09 PM, "Bruce Leban" wrote: > While I'm generally positive about the idea of an except expression (and > favor A except E pass B over any of the alternatives), I think it's worth > looking in a different direction. > > One of the motivations of this idea is that there are some use cases that > are particularly inconvenient. Another is the proliferation of APIs > accepting default values. > > So instead of a general solution, let me suggest an alternative that > addresses specific use cases. I'll write these in terms of an except > expression so that they can be directly compared head to head. > > a[?b] <=> a[b] except (IndexError, TypeError)pass None > a.?b <=> a.b except AttributeError pass None > a(?b) <=> a(b) except Exception pass None > a or? b <=> (a except Exception pass None) or b > a and? b <=> (a except Exception pass None) and b > > del? a.b <=> try: > del a.b > except AtributeError: > pass > > Of course you can chain these: > > a[?b][?c] > a.?b.?c > > a or? b or? c > > > Pros/Cons: > - no option to specify replacement value on an exception > - no option to specify exceptions to catch > - doesn't specifically solve the api default value issue -- and the option > for that case a(?b) catches all exceptions rather than specific ones > > > --- Bruce > Learn how hackers think: http://j.mp/gruyere-security > > Note: the syntax could just as easily be a?[b], a?.b, etc. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Sun Feb 16 00:17:47 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 16 Feb 2014 12:17:47 +1300 Subject: [Python-ideas] except expression In-Reply-To: <20140215181136.GH4281@ando> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> Message-ID: <52FFF59B.4090301@canterbury.ac.nz> Steven D'Aprano wrote: > This suggests that perhaps the parentheses aren't needed unless you have > multiple except parts: > > things[i] (except IndexError, KeyError as exc: exc.args) > things[i] except IndexError, KeyError as exc: exc.args Possibly not even necessary when there are multiple except clauses, if you don't require a comma between them: things[i] except IndexError: 42 except KeyError: 17 Parsing may be easier without the parens as well, since otherwise it looks like a function call when you hit the '(' until you see the 'except' after it. (I *think* that Python's parser could be made to handle that, but I'd have to experiment to find out for sure.) > Here is a torture-test for the syntax: can we combine it with an > if-expression? I think we can, although you may need parentheses to > disambiguate the expression: > > something(x) (except TypeError: a if condition else b) The paren-less version of this would be something(x) except TypeError: a if condition else b the interpretation of which would depend on what relative precedence we decide on between 'except' and 'if'. Parens can still be used to clarify, though: (something(x) except TypeError: a) if condition else b something(x) except TypeError: (a if condition else b) > This does look a tiny bit like a function call, especially if you delete > the space between the leading expression and the opening bracket: > > # ugly, don't do this > things[i](except IndexError: 42) Yes, that's why I'm leaning towards the paren-less version. -- Greg From greg.ewing at canterbury.ac.nz Sun Feb 16 00:28:28 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 16 Feb 2014 12:28:28 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> Message-ID: <52FFF81C.6080100@canterbury.ac.nz> Philipp A. wrote: > hmm i still don?t like the colon. everywhere else in python, it means > ?new block follows? No, it doesn't. There are at least two existing exceptions, lambdas and slice expressions. > and i?m sure nobody here wants to allow the following? > > |value = hovercraft() except EelsException: > remove_eels() > no_eels #this is assigned to ?value? if we have too many eels > > neither should we introduce a colon that /can?t/ be followed by a block. I don't think anyone is suggesting that. The colons in lambdas and slice expressions can't be followed by blocks either, and nobody seems to have difficulty with that. -- Greg From greg.ewing at canterbury.ac.nz Sun Feb 16 00:37:05 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 16 Feb 2014 12:37:05 +1300 Subject: [Python-ideas] except expression In-Reply-To: <20140215181136.GH4281@ando> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> Message-ID: <52FFFA21.3020401@canterbury.ac.nz> Steven D'Aprano wrote: > On Sat, Feb 15, 2014 at 11:20:13AM +1300, Greg Ewing wrote: >>Also it might be useful to be able to say >> >> things.remove(i) (except ValueError: pass) >> >>which would be equivalent to >> >> things.remove(i) (except ValueError: None) > > Certainly not! pass implies that *no return result is generated at all*, > which is not possible in Python. Well, it is, kind of -- an implicit None is produced when you don't specify a return value for a function; this is a similar thing. Would it help if it were *only* allow it in a context where the value of the expression is going to be ignored? -- Greg From rosuav at gmail.com Sun Feb 16 00:46:15 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 16 Feb 2014 10:46:15 +1100 Subject: [Python-ideas] except expression In-Reply-To: <52FFF59B.4090301@canterbury.ac.nz> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> Message-ID: On Sun, Feb 16, 2014 at 10:17 AM, Greg Ewing wrote: >> This does look a tiny bit like a function call, especially if you delete >> the space between the leading expression and the opening bracket: >> >> # ugly, don't do this >> things[i](except IndexError: 42) > > > Yes, that's why I'm leaning towards the paren-less version. I'm not liking the parenthesized version here, because the 'except' expression has to know how much of the preceding expression to modify. With a function call: expression(args) the expression is fully evaluated first, and then whatever it returns gets called; with the except expression, a try block has to be set up somewhere. This is more similar to if-else than to a function call, in that the expression before the 'if' isn't evaluated until the condition has been tested. Parens could go around the whole thing: (thing[i] except IndexError: 42) (1/x if x else "Div by 0") but not around the except clause: thing[i] (except IndexError: 42) # Nope 1/x (if x else "Div by 0") # Nope Incidentally, I'm looking at this being able to chain quite nicely: ((expr except Exception1: default1) except Exception2: default2) Ideally the peephole optimizer could set up a single try/except structure for both, but syntactically, I'm seeing this as the way the operator associates. I've mailed the first-draft PEP to peps@; is it appropriate to post it here as well, or should I wait to hear back from peps@? ChrisA From rosuav at gmail.com Sun Feb 16 01:06:50 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 16 Feb 2014 11:06:50 +1100 Subject: [Python-ideas] except expression In-Reply-To: <52FFFA21.3020401@canterbury.ac.nz> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFFA21.3020401@canterbury.ac.nz> Message-ID: On Sun, Feb 16, 2014 at 10:37 AM, Greg Ewing wrote: > Steven D'Aprano wrote: >> >> On Sat, Feb 15, 2014 at 11:20:13AM +1300, Greg Ewing wrote: > > >>> Also it might be useful to be able to say >>> >>> things.remove(i) (except ValueError: pass) >>> >>> which would be equivalent to >>> >>> things.remove(i) (except ValueError: None) >> >> >> Certainly not! pass implies that *no return result is generated at all*, >> which is not possible in Python. > > > Well, it is, kind of -- an implicit None is produced > when you don't specify a return value for a function; > this is a similar thing. > > Would it help if it were *only* allow it in a context > where the value of the expression is going to be ignored? That's nothing to do with the 'pass' keyword. def foo(): print("Hello!") This will return None, and it doesn't have 'pass'. def foo(x): if x: pass return 42 This will not, regardless of the value of x. Python's 'pass' means 'do nothing', in a place where you contextually need to do something. ChrisA From python at mrabarnett.plus.com Sun Feb 16 01:35:59 2014 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 16 Feb 2014 00:35:59 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> Message-ID: <530007EF.6030007@mrabarnett.plus.com> On 2014-02-15 23:46, Chris Angelico wrote: > On Sun, Feb 16, 2014 at 10:17 AM, Greg Ewing > wrote: >>> This does look a tiny bit like a function call, especially if you delete >>> the space between the leading expression and the opening bracket: >>> >>> # ugly, don't do this >>> things[i](except IndexError: 42) >> >> >> Yes, that's why I'm leaning towards the paren-less version. > > I'm not liking the parenthesized version here, because the 'except' > expression has to know how much of the preceding expression to modify. > With a function call: > > expression(args) > > the expression is fully evaluated first, and then whatever it returns > gets called; with the except expression, a try block has to be set up > somewhere. This is more similar to if-else than to a function call, in > that the expression before the 'if' isn't evaluated until the > condition has been tested. > > Parens could go around the whole thing: > > (thing[i] except IndexError: 42) > (1/x if x else "Div by 0") > > but not around the except clause: > > thing[i] (except IndexError: 42) # Nope > 1/x (if x else "Div by 0") # Nope > > Incidentally, I'm looking at this being able to chain quite nicely: > > ((expr except Exception1: default1) except Exception2: default2) > You'll also need to note that: ((expr except Exception1: default1) except Exception2: default2) is not the same as: (expr except Exception1: default1 except Exception2: default2) The first will also catch Exception2 if default1 raises it, whereas the second will catch Exception2 only if expr raises it and it hasn't been caught by the preceding 'except'. > Ideally the peephole optimizer could set up a single try/except > structure for both, but syntactically, I'm seeing this as the way the > operator associates. > > I've mailed the first-draft PEP to peps@; is it appropriate to post it > here as well, or should I wait to hear back from peps@? > From rosuav at gmail.com Sun Feb 16 01:57:03 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 16 Feb 2014 11:57:03 +1100 Subject: [Python-ideas] except expression In-Reply-To: <530007EF.6030007@mrabarnett.plus.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> Message-ID: On Sun, Feb 16, 2014 at 11:35 AM, MRAB wrote: > You'll also need to note that: > > > ((expr except Exception1: default1) except Exception2: default2) > > is not the same as: > > (expr except Exception1: default1 except Exception2: default2) > > The first will also catch Exception2 if default1 raises it, whereas the > second will catch Exception2 only if expr raises it and it hasn't been > caught by the preceding 'except'. Ooh. Good catch. This is not just an optimization, it's a specific piece of syntax: chaining 'except' blocks MUST catch only from the original. I'm also switching around my Proposal and Alternative Proposals a bit, as I'm currently leaning towards the colon rather than a keyword. Lots of edits to the PEP draft, but so far just kept local to my own computer. At what point (and in what way) should I start sharing those edits? ChrisA From ncoghlan at gmail.com Sun Feb 16 02:09:02 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 16 Feb 2014 11:09:02 +1000 Subject: [Python-ideas] Enhance exceptions by attaching some more information to them In-Reply-To: References: Message-ID: On 16 February 2014 01:40, Sebastian Kreft wrote: > More than once I've been in a situation where I wish that some of the stdlib > exceptions had a better message or some more information to help me diagnose > the problem. > > For example: > a = [1, 2, 3, 4, 5] > a[5] > IndexError: list index out of range > > In this case there's no reference to neither the length of the array nor to > the offending index. > > I'm of the idea that we could extend the exceptions by adding some more > information to them, so 3rd party libraries could use them for > debugging/logging. Yes, having particular exceptions expose more structured data would be a good thing, and some improvements have already been made in that direction (ImportError now has name and path attributes, for example) The barriers are largely practical ones rather than philosophical: 1. Each updated exception needs to have the new optional attributes and keyword-only arguments defined (in C for the building exceptions, although Argument Clinic makes the argument parsing aspects easier now), along with appropriate docstring and documentation updates, as well as changes to the default representation and/or implicit calculation of an args[0]. (They typically have to be optional keyword only parameters for backwards compatibility reasons) 2. To actually benefit end users, the new exception capabilities need to be used appropriately in the core interpreter and standard library. It's easy to look at the scope of work involved in step 2, and back away from working on step 1. However, it's important to realise that step 2 can be done incrementally - step 1 is about making it *possible* to provide more structured information in the exceptions, and then step 2 can happen more opportunistically by tackling a particular part of the interpreter or standard library at a time. (for example, that is how the improved introspection for builtins and extension modules is being rolled out). Step 1 could likely benefit from being elaborated in a PEP, as that then provides a useful starting point for other implementations when they go to update their own builtin exception implementations accordingly. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Sun Feb 16 02:15:23 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 16 Feb 2014 11:15:23 +1000 Subject: [Python-ideas] A suggested feature for difflib In-Reply-To: References: Message-ID: The general idea sounds reasonable to me, but a more appropriate signature addition may be "line_matcher=SequenceMatcher, char_matcher=SequenceMatcher". As the Differ docs point out: "Differ uses SequenceMatcher both to compare sequences of lines, and to compare sequences of characters within similar (near-matching) lines." Cheers, Nick. From steve at pearwood.info Sun Feb 16 04:35:02 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 16 Feb 2014 14:35:02 +1100 Subject: [Python-ideas] except expression In-Reply-To: <530007EF.6030007@mrabarnett.plus.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> Message-ID: <20140216033502.GA4519@ando> On Sun, Feb 16, 2014 at 12:35:59AM +0000, MRAB wrote: > On 2014-02-15 23:46, Chris Angelico wrote: > >On Sun, Feb 16, 2014 at 10:17 AM, Greg Ewing > > wrote: > >>>This does look a tiny bit like a function call, especially if you delete > >>>the space between the leading expression and the opening bracket: > >>> > >>> # ugly, don't do this > >>> things[i](except IndexError: 42) > >> > >> > >>Yes, that's why I'm leaning towards the paren-less version. > > > >I'm not liking the parenthesized version here, because the 'except' > >expression has to know how much of the preceding expression to modify. I don't think this is terribly different from the if-expression. The parser sees an expression: 1/x ... and doesn't know it is part of an if-expression until it has read on and seen the "if": 1/x if x != 0 ... so I don't expect this to be a problem. [...] > >Parens could go around the whole thing: Of course they can, that's just normal parentheses-as-grouping :-) > >(thing[i] except IndexError: 42) > >(1/x if x else "Div by 0") > > > >but not around the except clause: > > > >thing[i] (except IndexError: 42) # Nope > >1/x (if x else "Div by 0") # Nope I disagree about prohibiting this. I think it actually makes it easier to read in the case of multiple except terms. Since they ought to be separated by commas, and stylistically they ought to be split one-per-line, I prefer the bracket-less version when there is only a single except: thing(a, b) except ThisError, ThatError: default and brackets when there are more than one: thing(a, b) (except ThisError, ThatError: default, except SpamError, EggsError: something, except OutOfCheeseError: different_thing, ) although I'm happy for the parens to be optional in the second case if the parser can manage it, assuming that the parser can disambigulate all the various uses of commas. I can't think of any places where parens are explicitly prohibited. They used to be prohibited in imports, but that's no longer the case. One should always be allowed to use parens for grouping and implicit line continuations, nearly everywhere. How does this suggested syntax look with Raymond's recent example? Raymond regretted that max and min now take a "default" argument, as of Python 3.4. Let's compare: result = max(some_numbers, key=foo, default=float('nan')) result = max(some_numbers, key=foo) except ValueError: float('nan') Looks damn good to me! > >Incidentally, I'm looking at this being able to chain quite nicely: > > > >((expr except Exception1: default1) except Exception2: default2) This is equivalent to: try: try: result = expr except Exception1: result = default1 except Exception2: result = default2 > You'll also need to note that: > > ((expr except Exception1: default1) except Exception2: default2) > > is not the same as: > > (expr except Exception1: default1 except Exception2: default2) I disklike the lack of comma between except clauses. I think that ought to be written as: (expr except Exception1: default1, except Exception2: default2) modulo the above argument about parentheses. > The first will also catch Exception2 if default1 raises it, whereas the > second will catch Exception2 only if expr raises it and it hasn't been > caught by the preceding 'except'. Correct. The two are quite different. > >Ideally the peephole optimizer could set up a single try/except > >structure for both, but syntactically, I'm seeing this as the way the > >operator associates. > > > >I've mailed the first-draft PEP to peps@; is it appropriate to post it > >here as well, or should I wait to hear back from peps@? I think it is perfectly appropriate to post a draft PEP here. -- Steven From rosuav at gmail.com Sun Feb 16 04:57:12 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 16 Feb 2014 14:57:12 +1100 Subject: [Python-ideas] except expression In-Reply-To: <20140216033502.GA4519@ando> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> Message-ID: On Sun, Feb 16, 2014 at 2:35 PM, Steven D'Aprano wrote: > On Sun, Feb 16, 2014 at 12:35:59AM +0000, MRAB wrote: >> On 2014-02-15 23:46, Chris Angelico wrote: >> >>> things[i](except IndexError: 42) >> >> >> >I'm not liking the parenthesized version here, because the 'except' >> >expression has to know how much of the preceding expression to modify. > > I don't think this is terribly different from the if-expression. The > parser sees an expression: > > 1/x ... > > and doesn't know it is part of an if-expression until it has read on and > seen the "if": > > 1/x if x != 0 ... > > so I don't expect this to be a problem. >> >but not around the except clause: >> > >> >thing[i] (except IndexError: 42) # Nope >> >1/x (if x else "Div by 0") # Nope > > I disagree about prohibiting this. I think it actually makes it easier > to read in the case of multiple except terms. The if/else version is currently an error, because the parens are breaking the sequence up: >>> 1/x (if x else "Div by 0") SyntaxError: invalid syntax (highlighting the "if" as the invalid part) The way I understand it, the entire expression from 1/x to "Div by 0" has to be parsed as a unit. Putting parens around part of it breaks that. The same goes for the exception-catching version, unless the parens are specifically a part of the syntax. Since you can just put parens around the whole expression (to help with wrapping etc), I don't see a compelling reason to put them around just the except bit. > Since they ought to be > separated by commas, and stylistically they ought to be split > one-per-line, I prefer the bracket-less version when there is only a > single except: > > thing(a, b) except ThisError, ThatError: default > > and brackets when there are more than one: > > thing(a, b) (except ThisError, ThatError: default, > except SpamError, EggsError: something, > except OutOfCheeseError: different_thing, > ) Move the open parenthesis to before thing(a, b) and you have what I would be recommending. I'm +0 on the explicit comma separation between except clauses. Will add that to the proposal, complete with a caution about it being a potential bug magnet. > I can't think of any places where parens are explicitly prohibited. They > used to be prohibited in imports, but that's no longer the case. One > should always be allowed to use parens for grouping and implicit line > continuations, nearly everywhere. They're prohibited in the middles of things. You can't, for instance, put parens around a few of a function's arguments (if they're pure positional then it's legal but has other meaning, and other notations are syntax errors). The inside of ( ) generally has to be a valid, parseable expression. > How does this suggested syntax look with Raymond's recent example? > Raymond regretted that max and min now take a "default" argument, as of > Python 3.4. Let's compare: > > result = max(some_numbers, key=foo, default=float('nan')) > > result = max(some_numbers, key=foo) except ValueError: float('nan') > > Looks damn good to me! Yeah. I especially like how this means the author of max() doesn't need to think about this possibility. (I'm still trying to come up with, in my head, an external means of chaining method calls - one that makes sense and looks clean. Same reason; the method author shouldn't have to think in terms of chaining.) >> >Incidentally, I'm looking at this being able to chain quite nicely: >> > >> >((expr except Exception1: default1) except Exception2: default2) > > This is equivalent to: > > > try: > try: > result = expr > except Exception1: > result = default1 > except Exception2: > result = default2 Yes, as MRAB pointed out. I've updated the PEP to note this. The syntactic form has to itself understand chaining. >> >I've mailed the first-draft PEP to peps@; is it appropriate to post it >> >here as well, or should I wait to hear back from peps@? > > I think it is perfectly appropriate to post a draft PEP here. Okay. Will post that separately to this. ChrisA From rosuav at gmail.com Sun Feb 16 05:04:03 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 16 Feb 2014 15:04:03 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> Message-ID: PEP: XXX Title: Exception-catching expressions Version: $Revision$ Last-Modified: $Date$ Author: Chris Angelico Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 15-Feb-2014 Python-Version: 3.5 Post-History: 16-Feb-2014 Abstract ======== Just as PEP 308 introduced a means of value-based conditions in an expression, this system allows exception-based conditions to be used as part of an expression. Motivation ========== A number of functions and methods have parameters which will cause them to return a specified value instead of raising an exception. The current system is ad-hoc and inconsistent, and requires that each function be individually written to have this functionality; not all support this. * dict.get(key, default) - second positional argument in place of KeyError * next(iter, default) - second positional argument in place of StopIteration * list.pop() - no way to return a default (TODO: Get more examples. I know there are some but I can't think of any now.) Rationale ========= The current system requires that a function author predict the need for a default, and implement support for it. If this is not done, a full try/except block is needed. Note that the specific syntax is open to three metric tons of bike-shedding. The proposal may well be rejected, but even then is not useless; it can be maintained as a collection of failed syntaxes for expression exceptions. Proposal ======== Just as the 'or' operator and the three part 'if-else' expression give short circuiting methods of catching a falsy value and replacing it, this syntax gives a short-circuiting method of catching an exception and replacing it. This currently works:: lst = [1, 2, None, 3] value = lst[2] or "No value" The proposal adds this:: lst = [1, 2] value = lst[2] except IndexError: "No value" The exception object can be captured just as in a normal try/except block:: # Return the next yielded or returned value from a generator value = next(it) except StopIteration as e: e.args[0] This is effectively equivalent to:: try: _ = next(it) except StopIteration as e: _ = e.args[0] value = _ This ternary operator would be between lambda and if/else in precedence. Chaining -------- Multiple 'except' keywords can be used, and they will all catch exceptions raised in the original expression (only):: value = (expr except Exception1 [as e]: default1 except Exception2 [as e]: default2 # ... except ExceptionN [as e]: defaultN ) This is not the same as either parenthesized form:: value = (("Yield: "+next(it) except StopIteration as e: "End: "+e.args[0]) except TypeError: "Error: Non-string returned or raised") value = (next(it) except StopIteration as e: (e.args[0] except IndexError: None)) The first form will catch an exception raised in either the original expression or in the default expression; the second form will catch ONLY one raised by the default expression. All three effects have their uses. Alternative Proposals ===================== Discussion on python-ideas brought up the following syntax suggestions:: value = expr except default if Exception [as e] value = expr except default for Exception [as e] value = expr except default from Exception [as e] value = expr except Exception [as e] return default value = expr except (Exception [as e]: default) value = expr except Exception [as e] try default value = expr except Exception [as e] continue with default value = default except Exception [as e] else expr value = try expr except Exception [as e]: default value = expr except Exception [as e] pass default In all cases, default is an expression which will not be evaluated unless an exception is raised; if 'as' is used, this expression may refer to the exception object. It has also been suggested that a new keyword be created, rather than reusing an existing one. Such proposals fall into the same structure as the last form, but with a different keyword in place of 'pass'. Suggestions include 'then', 'when', and 'use'. Open Issues =========== finally clause -------------- Should the except expression be able to have a finally clause? No form of the proposal so far has included finally. Commas between multiple except clauses -------------------------------------- Where there are multiple except clauses, should they be separated by commas? It may be easier for the parser, that way:: value = (expr except Exception1 [as e]: default1, except Exception2 [as e]: default2, # ... except ExceptionN [as e]: defaultN, ) with an optional comma after the last, as per tuple rules. Downside: Omitting the comma would be syntactically valid, and would have almost identical semantics, but would nest the entire preceding expression in its exception catching rig - a matching exception raised in the default clause would be caught by the subsequent except clause. As this difference is so subtle, it runs the risk of being a major bug magnet. Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 From ron3200 at gmail.com Sun Feb 16 08:22:25 2014 From: ron3200 at gmail.com (Ron Adam) Date: Sun, 16 Feb 2014 01:22:25 -0600 Subject: [Python-ideas] @partials = decorators In-Reply-To: References: Message-ID: On 02/10/2014 01:44 PM, Alex Rodrigues wrote: > This is pretty cumbersome to write out every time so I wanted a function that could just be called on arbitrary histograms like side_by_side(data1.hist(bottom=0.1, log=True), data2.hist(bins=10)). The problem is that I do not have the axes available when I call the function and once I pass the plots to the function they have already been evaluated and there is no way to add arguments retroactively. With the continuation syntax it would be amazingly easier: > > def side_by_side(plot1, plot2): > fig = plt.figure() > ax1 = plt.subplot(121) > ax2 = plt.subplot(122) > plot1(ax=ax1) > plot2(ax=ax2) > > side_by_side(@data.hist(), @data.hist(log=True)) #Technically the first > one doesn't need to be a continuation, but whatever. > > tldr; I like it and see a decent number of applications. Also I'm > wondering if you'd have to chain continuations > @@a('arg1')('arg2')('arg3'). These functions are more like partials, rather than continuations, which pause in the middle of the function. No, you wouldn't need to chain them since each incomplete call returns a incomplete callable object. So this works... _()(f)(a1)(a2)(a3) --> result The latest version I have, ... it's evolving as I go... has two parts. 1. A caller function. Which attempts to call the function. 2. A callable holder object. Which holds the incomplete call state. Both of these take any number of arguments. The only requirement is that the first item in the sequence of arguments be a callable. You can be initiate the arguments with either the caller, or the holder, depending on weather or not you want to delay the initial call attempt. The caller will try to call the function with the initially given arguments and if that fails, it instead returns a callable holder object. When the holder is called, it will collect more arguments and pass those to the caller. Which may return a final result or another holder. If you apply a non callable object to these, it just gets returned. That allows you to catch deferred results, and pass other things through. A sort of explicit lazy evaluation. So far, I've found these are pretty good at doing lambda calculus, flattening decorators, and can be useful in deferring a call in some situations where normally you would use a lambda function or a nested function definition. And they may also be good at collecting non localised, or disjointed data. It's also nice that they are higher order functions that accept most any callable. The main issue is there is not a reliable way to get the argument count of builtin callable function or class. Python functions have an arg_count attribute in the code object. But builtin functions lack that attribute. (hint, hint...) So for now, these are more conceptual than useful, as they will break under certain situations... like giving too many arguments, or if a TypeError exception comes from within the function instead of during the call process. Cheers, Ron From zuo at chopin.edu.pl Sun Feb 16 13:01:00 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sun, 16 Feb 2014 13:01:00 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> Message-ID: <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> 16.02.2014 00:46, Chris Angelico wrote: > On Sun, Feb 16, 2014 at 10:17 AM, Greg Ewing > wrote: >>> This does look a tiny bit like a function call, especially if you >>> delete >>> the space between the leading expression and the opening bracket: >>> >>> # ugly, don't do this >>> things[i](except IndexError: 42) >> >> >> Yes, that's why I'm leaning towards the paren-less version. [snip] That's why what a proposed was: things[i] except (IndexError: 42) -- rather than: things[i] (except IndexError: 42). Also, see below... > Parens could go around the whole thing: > > (thing[i] except IndexError: 42) > (1/x if x else "Div by 0") > > but not around the except clause: > > thing[i] (except IndexError: 42) # Nope > 1/x (if x else "Div by 0") # Nope > > Incidentally, I'm looking at this being able to chain quite nicely: > > ((expr except Exception1: default1) except Exception2: default2) [snip] In terms of my proposal it would clearly be just: expr except (Exception1: default1) except (Exception2: default2) Of course grouping parens could be added without changing semantics: (expr except (Exception1: default1)) except (Exception2: default2) Also, see below... 15.02.2014 19:11, Steven D'Aprano wrote: > On Sat, Feb 15, 2014 at 11:20:13AM +1300, Greg Ewing wrote: >> Here's another one: >> >> things[i] (except IndexError: 42) > > I believe Jan Kaliszewski independently came up with the same syntax > earlier, which is a good sign. Two people suggesting the same thing > is > promising. Indeed, though please note that my proposal is a bit different: things[i] except (IndexError: 42) possibly extendable to: # with 'as' clause' some_io() except (OSError as exc: exc.errno) ...and/or: # with multiple exception catches some_io() except (FileNotFoundError: 42, OSError as exc: exc.errno) Before I posted the proposal I did think about the "things[i] (except ..." variant also but I don't like that the opening parenthesis character suggest to a human reader that it is a call... On the other hand, when the parenthesis is *after* the 'except' keyword it is clear for human readers that it is a dedicated syntax having nothing to do with a call. Also, for a multiple-exception-catches-variant I don't like repeating the 'except' keyword for each catch, as well as the ambiguity whether the consecutive 'except...' concerns only the initial expression or also the preceding 'except...'). In my proposal there is no such ambiguity. I also believe that making the parens *obligatory* is a good thing as then: * things are much more clear when several expressions are combined (e.g. except-expression + conditional expression; except-expression + another except-expression, except-expression + generator-expression...); * it is clear that the colon has nothing to do with opening a code block; * it just reads better than without parens. More complex (though probably not very realistic) example: msg = (cache[k] except ( LookupError: backend.read() except ( OSError: 'resource not available')) if check_permission(k) else 'access not allowed' ) except (Exception: 'internal error occurred') Cheers. *j From flying-sheep at web.de Sun Feb 16 13:16:38 2014 From: flying-sheep at web.de (Philipp A.) Date: Sun, 16 Feb 2014 13:16:38 +0100 Subject: [Python-ideas] except expression In-Reply-To: <20140215195731.GI4281@ando> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <20140215195731.GI4281@ando> Message-ID: 2014-02-15 20:57 GMT+01:00 Steven D?Aprano : That is incorrect. Did you read my full post? I specifically mentioned both dicts and lambda, both of which use colons in expressions, and don?t start a new block. oh, sorry, totally overlooked that. i wouldn?t count dicts (and slices), as they are literals of sorts, but lambdas are exactly what we?re sidcussing about: a expression, instisting of sub-expression, one of which is after a colon. forget my post. i now like the colon variant, as well as the ?pass? and ?try? variants, with no clear preference. ?pass? is a placeholder null statement, it has no relevance to try?except. You might just as well sensibly say stomach_content = eat() except ThinMintError import explode() stomach_content = eat() except ThinMintError del explode() stomach_content = eat() except ThinMintError class explode() stomach_content = eat() except ThinMintError def explode() stomach_content = eat() except ThinMintError pass explode() In all of these cases, I have picked a random keyword and just tossed it into the expression. None of them make any sense. hah! now it?s my turn to smugly say ?did you even read my post?!? ;) 2014-02-13 14:19 GMT+01:00 Philipp A. : pass is only used to mean ?do nothing? right now, and indeed that?s fitting: it?s just a border between EXC_TYPE [as EXC_NAME] and ALT_EXPR, *and its other meaning makes sense in english grammar*. Maybe even try? Like ?use this, except if that doesn?t work, then try using the other thing?. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Sun Feb 16 14:01:19 2014 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 17 Feb 2014 00:01:19 +1100 Subject: [Python-ideas] except expression In-Reply-To: <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> Message-ID: On Sun, Feb 16, 2014 at 11:01 PM, Jan Kaliszewski wrote: > In terms of my proposal it would clearly be just: > > expr except (Exception1: default1) except (Exception2: default2) > > Of course grouping parens could be added without changing semantics: > > (expr except (Exception1: default1)) except (Exception2: default2) > > Also, see below... Actually that _does_ change semantics, but in an extremely subtle way. In both versions, expr raising Exception2 will result in default2; but the first version is effectively this: try: _ = expr except Exception1: _ = default1 except Exception2: _ = default2 value_of_expression = _ If the evaluation of default1 raises Exception2, then this form will propagate that exception up. The second form is actually like this: try: try: _ = expr except Exception1: _ = default1 except Exception2: _ = default2 value_of_expression = _ In this, the evaluation of default1 is inside the outer try block, so if it raises Exception2, the expression result will be default2. It's an extremely subtle difference (since, in most normal cases, the default expressions will be very simple), but it is a difference. Also, the interpreter's forced to construct and process two try/except blocks, for what that's worth. I expect this won't ever be significant in a real-world case, but there is a difference: >>> timeit.repeat("try: 1/0\nexcept NameError: pass\nexcept ZeroDivisionError: pass") [2.3123623253793433, 2.240881173445228, 2.2435943674405543] >>> timeit.repeat("try:\n try: 1/0\n except NameError: pass\nexcept ZeroDivisionError: pass") [2.9560086547312565, 2.958433264562956, 2.9855417378465674] That's of the order of five microseconds per iteration, if I'm reading my figures correctly, so that's not exactly significant... but I still see no reason to go to the extra work unnecessarily :) > possibly extendable to: > > # with 'as' clause' > some_io() except (OSError as exc: exc.errno) Definitely want to keep the 'as' clause in there, although it does raise some issues of its own as regards name binding. (responding out of order as your code example nicely illustrates your next point) > Before I posted the proposal I did think about the "things[i] (except > ..." variant also but I don't like that the opening parenthesis > character suggest to a human reader that it is a call... On the other > hand, when the parenthesis is *after* the 'except' keyword it is clear > for human readers that it is a dedicated syntax having nothing to do > with a call. I don't see it as particularly significant either way. Opening parens and immediately having a keyword like "except" is strongly indicative of something different going on; same goes for having that keyword just _before_ the parens. > ...and/or: > > # with multiple exception catches > some_io() except (FileNotFoundError: 42, > OSError as exc: exc.errno) > > Also, for a multiple-exception-catches-variant I don't like repeating > the 'except' keyword for each catch, as well as the ambiguity whether > the consecutive 'except...' concerns only the initial expression or > also the preceding 'except...'). In my proposal there is no such > ambiguity. There's an ambiguity in this version, though. After the 'except' keyword, you can have a tuple of exceptions: try: 1/x except (NameError, ZeroDivisionError): print("Can't divide by nothing!") If the previous value is followed by a comma and the next thing could be a tuple, the parsing is going to get a bit hairy. Either the previous value or the exception_list could be a tuple. I'd rather repeat the word 'except'. > I also believe that making the parens *obligatory* is a good thing I'm happy with them being optional in cases where it's unambiguous, but I'd be okay with mandating them to disambiguate. After all, if the language says they're optional, style guides have the option of mandating; but if the language mandates, the freedom is gone. > More complex (though probably not very realistic) example: > > msg = (cache[k] except ( > LookupError: backend.read() except ( > OSError: 'resource not available')) > if check_permission(k) > else 'access not allowed' > ) except (Exception: 'internal error occurred') Fairly unrealistic, as the backend.read() call won't get put into the cache (and you can't put the final msg into the cache, as 'resource not available' sounds temporary and thus non-cached), not to mention that swallowing Exception to just return a constant string seems like a really REALLY bad idea! :) But this highlights another big issue: How should a complex except expression be laid out? With the statement form, it's pretty clear: first you do the line(s) that you're trying, then you have your except clauses, no more than one on any given line, and then you have your else, and your finally, if you have either. Everything's on separate lines. What about here? If it all fits on one line, great! But it often won't (the name "ZeroDivisionError" is over a fifth of your typical line width, all on its own), and then where do you break it? Not technically part of the proposal, but it's going to be an issue. ChrisA From rosuav at gmail.com Sun Feb 16 14:12:30 2014 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 17 Feb 2014 00:12:30 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <20140215195731.GI4281@ando> Message-ID: On Sun, Feb 16, 2014 at 11:16 PM, Philipp A. wrote: > i wouldn?t count dicts (and slices), as they are literals of sorts, but > lambdas are exactly what we?re sidcussing about: a expression, instisting of > sub-expression, one of which is after a colon. Braces syntax for dicts is somewhere between an expression and a literal. Syntactically it's not a literal, as can be shown with dis: >>> def f(): return {1:2, 3:4} >>> dis.dis(f) 2 0 BUILD_MAP 2 3 LOAD_CONST 1 (2) 6 LOAD_CONST 2 (1) 9 STORE_MAP 10 LOAD_CONST 3 (4) 13 LOAD_CONST 4 (3) 16 STORE_MAP 17 RETURN_VALUE But to the human, it's convenient to think of it as a "dict literal" in the same way that you can have a "string literal" or "float literal". You can compare something against a "literal", for instance: >>> f() == {3:4, 1:2} True However, it's not a literal, as it can have arbitrary expressions for both keys and values: >>> {int("123"):float("234")} {123: 234.0} > [ Steven said: ] > ?pass? is a placeholder null statement, it has no relevance to > try?except. You might just as well sensibly say > > stomach_content = eat() except ThinMintError import explode() > stomach_content = eat() except ThinMintError del explode() > stomach_content = eat() except ThinMintError class explode() > stomach_content = eat() except ThinMintError def explode() > stomach_content = eat() except ThinMintError pass explode() > > In all of these cases, I have picked a random keyword and just tossed it > into the expression. None of them make any sense. > [ Philipp responded: ] > hah! now it?s my turn to smugly say ?did you even read my post?!? ;) It's worth noting that the person who originally suggested "pass" here was me, and I came up with it by the exact method Steven described: pulled up a list of keywords and picked one that seemed plausible enough to post... *as a joke*. It wasn't till it was picked up on as a plausible suggestion that I considered it at all seriously. It was my preferred form in the first draft; now, my preferred form is with the colon. The two-keyword notation lent itself well to parallels with if/else, but since chaining needs to be special anyway, this isn't going to be a simple operator. (By the way, Steven, what on earth is a Thin Mint that could cause your eating to be so exceptional that your stomach explodes? The mind boggles...) ChrisA From flying-sheep at web.de Sun Feb 16 14:44:17 2014 From: flying-sheep at web.de (Philipp A.) Date: Sun, 16 Feb 2014 14:44:17 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <20140215195731.GI4281@ando> Message-ID: 2014-02-16 14:12 GMT+01:00 Chris Angelico : > (By the way, Steven, what on earth is a Thin Mint that could cause > your eating to be so exceptional that your stomach explodes? The mind > boggles...) > Did you ever wonder why the language is called ?Python? and why we use ?spam and eggs? instead of ?foo and bar?? Let?s just say that if you eat as much as Mr Creosote, even a wafer thin mint might be the straw that breaks the camel?s back? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sun Feb 16 15:37:59 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 17 Feb 2014 00:37:59 +1000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> Message-ID: On 16 February 2014 14:04, Chris Angelico wrote: > PEP: XXX > Title: Exception-catching expressions > Version: $Revision$ > Last-Modified: $Date$ > Author: Chris Angelico > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 15-Feb-2014 > Python-Version: 3.5 > Post-History: 16-Feb-2014 > > > Abstract > ======== > > Just as PEP 308 introduced a means of value-based conditions in an > expression, this system allows exception-based conditions to be used as part > of an expression. Thanks for putting this together, Chris! :) > Motivation > ========== > > A number of functions and methods have parameters which will cause them to > return a specified value instead of raising an exception. The current system > is ad-hoc and inconsistent, and requires that each function be individually > written to have this functionality; not all support this. > > * dict.get(key, default) - second positional argument in place of KeyError > > * next(iter, default) - second positional argument in place of StopIteration > > * list.pop() - no way to return a default > > (TODO: Get more examples. I know there are some but I can't think of any now.) * seq[index] - no way to return a default * str.find vs str.index (hardcoded -1 result vs exception) * min, max - keyword-only default in place of ValueError for empty iterable There's also a use case at the interactive prompt: >>> e = 1/0 except ZeroDivisionError as e: e (Currently, capturing an exception at the interactive prompt to poke around at the details is annoyingly painful, because "_" doesn't capture exceptions by detauls) > Proposal > ======== > > Just as the 'or' operator and the three part 'if-else' expression give short > circuiting methods of catching a falsy value and replacing it, this syntax > gives a short-circuiting method of catching an exception and replacing it. This section (or a separate "Detailed Semantics" one) is missing a detailed specification of the semantics, in particular: - what scope are subexpressions evaluated in? - if the "as" clause is permitted, what is the scope of that name binding? - if the current scope, then how does that square with the changes to list comprehensions in 3.0 to always use a new scope for the iteration variables (the same as generator expressions). Keep in mind that the except expression will need to preserve the Python 3 except clause behaviour of unbinding the exception name after evaluating the subexpression - if a new scope, what impact does that have on what subexpressions are valid, especially at class scope? - what happens if an except expression is used as part of the except clause in an ordinary try statement, or another except expression? (I suspect defining these except expressions as creating a new transient scope, akin to comprehensions and generator expressions, will actually avoid a variety of issues, in particular, preventing leaking the name of the bound exception into the containing scope. I also suspect a transient scope would make the PEP's currently preferred colon based notation more palatable to some readers) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From steve at pearwood.info Sun Feb 16 18:37:48 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 17 Feb 2014 04:37:48 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <20140215195731.GI4281@ando> Message-ID: <20140216173748.GE4519@ando> On Mon, Feb 17, 2014 at 12:12:30AM +1100, Chris Angelico wrote: > It's worth noting that the person who originally suggested "pass" here > was me, and I came up with it by the exact method Steven described: > pulled up a list of keywords and picked one that seemed plausible > enough to post... *as a joke*. You should say so in the PEP :-) > (By the way, Steven, what on earth is a Thin Mint that could cause > your eating to be so exceptional that your stomach explodes? The mind > boggles...) It's a Monty Python reference. It's actually a wafer-thin afterdinner mint. -- Steven From zuo at chopin.edu.pl Sun Feb 16 21:39:04 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sun, 16 Feb 2014 21:39:04 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> Message-ID: <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> 16.02.2014 14:01, Chris Angelico wrote: > On Sun, Feb 16, 2014 at 11:01 PM, Jan Kaliszewski > wrote: >> In terms of my proposal it would clearly be just: >> >> expr except (Exception1: default1) except (Exception2: default2) >> >> Of course grouping parens could be added without changing semantics: >> >> (expr except (Exception1: default1)) except (Exception2: >> default2) >> >> Also, see below... > > Actually that _does_ change semantics, but in an extremely subtle > way. As log as we are talking about my proposal -- it does not. The proposed syntax is roughly: except ( [as ]: , ...) ...and all these elements make up a compound expression (which obviously can be used as or in another except-compound-expr). > In both versions, expr raising Exception2 will result in default2; > but > the first version is effectively this: > > try: > _ = expr > except Exception1: > _ = default1 > except Exception2: > _ = default2 > value_of_expression = _ > > If the evaluation of default1 raises Exception2, then this form will > propagate that exception up. The second form is actually like this: > > try: > try: > _ = expr > except Exception1: > _ = default1 > except Exception2: > _ = default2 > value_of_expression = _ No, both expression forms I mentioned are equivalent to the *latter* (the nested one) of the above snippets. Equivalent to the *former* (the "flat" one) of the above snippets would be just the expression: expr except (Exception1: default1, Exception2: default2) [snip] >> Before I posted the proposal I did think about the "things[i] >> (except >> ..." variant also but I don't like that the opening parenthesis >> character suggest to a human reader that it is a call... On the >> other >> hand, when the parenthesis is *after* the 'except' keyword it is >> clear >> for human readers that it is a dedicated syntax having nothing to do >> with a call. > > I don't see it as particularly significant either way. Opening parens > and immediately having a keyword like "except" is strongly indicative > of something different going on; same goes for having that keyword > just _before_ the parens. Apparently it's a subjective matter as I feel that "expr except (...)" is less misleading for human eyes... >> ...and/or: >> >> # with multiple exception catches >> some_io() except (FileNotFoundError: 42, >> OSError as exc: exc.errno) >> >> Also, for a multiple-exception-catches-variant I don't like >> repeating >> the 'except' keyword for each catch, as well as the ambiguity >> whether >> the consecutive 'except...' concerns only the initial expression or >> also the preceding 'except...'). In my proposal there is no such >> ambiguity. > > There's an ambiguity in this version, though. After the 'except' > keyword, you can have a tuple of exceptions: > > try: > 1/x > except (NameError, ZeroDivisionError): > print("Can't divide by nothing!") > > If the previous value is followed by a comma and the next thing could > be a tuple, the parsing is going to get a bit hairy. Either the > previous value or the exception_list could be a tuple. I'd rather > repeat the word 'except'. Sorry, I don't catch the point. If I needed to use a complex exception spec (a tuple-like) and/or a tuple as the "default" expression -- I'd just do it: some_io() except (FileNotFoundError: (1, 2, 3), (ValueError, TypeError): 'spam') I see no ambiguity here. > I'm happy with them being optional in cases where it's unambiguous, > but I'd be okay with mandating them to disambiguate. After all, if > the > language says they're optional, style guides have the option of > mandating; but if the language mandates, the freedom is gone. Sometimes -- in syntax -- freedom is a bad thing. There are good reasons that 1-argument function calls have the form: "func(arg)" and not "func arg" -- though the latter could still be unambiguous. >> More complex (though probably not very realistic) example: >> >> msg = (cache[k] except ( >> LookupError: backend.read() except ( >> OSError: 'resource not available')) >> if check_permission(k) >> else 'access not allowed' >> ) except (Exception: 'internal error occurred') > > Fairly unrealistic, as the backend.read() call won't get put into the The example was about syntax of nested expressions, not about this or that use case. > cache (and you can't put the final msg into the cache, as 'resource > not available' sounds temporary and thus non-cached), not to mention > that swallowing Exception to just return a constant string seems like > a really REALLY bad idea! :) But this highlights another big issue: > How should a complex except expression be laid out? With the > statement > form, it's pretty clear: first you do the line(s) that you're trying, > then you have your except clauses, no more than one on any given > line, > and then you have your else, and your finally, if you have either. > Everything's on separate lines. What about here? If it all fits on > one > line, great! But it often won't (the name "ZeroDivisionError" is over > a fifth of your typical line width, all on its own), and then where > do > you break it? Not technically part of the proposal, but it's going to > be an issue. If we talk about *one* complex except expression, in terms of the syntax variant I proposed, the natural layout form seems to be: except ( [as VAR #1]: , [as VAR #3]: , [as VAR #3]: ) or except ( [as VAR #1]: , [as VAR #3]: , [as VAR #3]: ) Cheers. *j From rosuav at gmail.com Sun Feb 16 22:52:41 2014 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 17 Feb 2014 08:52:41 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> Message-ID: On Mon, Feb 17, 2014 at 1:37 AM, Nick Coghlan wrote: > Thanks for putting this together, Chris! :) Thanks for the edit advice! > This section (or a separate "Detailed Semantics" one) is missing a > detailed specification of the semantics, in particular: > - what happens if an except expression is used as part of the except > clause in an ordinary try statement, or another except expression? That should be fairly obvious - it's like nesting try/except. Does it need to be said? > - what scope are subexpressions evaluated in? > - if the "as" clause is permitted, what is the scope of that name binding? > - if the current scope, then how does that square with the changes to > list comprehensions in 3.0 to always use a new scope for the iteration > variables (the same as generator expressions). Keep in mind that the > except expression will need to preserve the Python 3 except clause > behaviour of unbinding the exception name after evaluating the > subexpression > - if a new scope, what impact does that have on what subexpressions > are valid, especially at class scope? > > (I suspect defining these except expressions as creating a new > transient scope, akin to comprehensions and generator expressions, > will actually avoid a variety of issues, in particular, preventing > leaking the name of the bound exception into the containing scope. I > also suspect a transient scope would make the PEP's currently > preferred colon based notation more palatable to some readers) Yeah, now, this part is a problem. Subsequently to the version posted, I've added an Open Issues subsection regarding scope. """ Scope of default expressions and 'as' ------------------------------------- In a try/except block, the use of 'as' to capture the exception object creates a local name binding, and implicitly deletes that binding in a finally clause. As 'finally' is not a part of this proposal (see below), this makes it tricky to describe; also, this use of 'as' gives a way to create a name binding in an expression context. Should the default clause have an inner scope in which the name exists, shadowing anything of the same name elsewhere? Should it behave the same way the statement try/except does, and unbind the name? Should it bind the name and leave it bound? (Almost certainly not; this behaviour was changed in Python 3 for good reason.) """ It's a knotty problem. I'm currently inclined to the inner scope idea (like a comprehension), but that's inconsistent with the statement form of try/except. Is that difference going to confuse people? >>> e = 1234 >>> [e for e in (1,)] [1] >>> e 1234 >>> try: 1/0 except ZeroDivisionError as e: pass >>> e Traceback (most recent call last): File "", line 1, in e NameError: name 'e' is not defined Also, what are the implications on the inside? Disassembling a function with a list comp shows that it calls an inner function, which I presume is the only way CPython can create an inner scope. Does the default-expression then have to be a separate function, and if so, what does that mean? Alternatively, what would it be like if an expression could do a name binding and unbinding (consistently with the statement form)? Would that confuse people no end? ChrisA From greg.ewing at canterbury.ac.nz Sun Feb 16 23:13:14 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 17 Feb 2014 11:13:14 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> Message-ID: <530137FA.10704@canterbury.ac.nz> Chris Angelico wrote: > I'm happy with them being optional in cases where it's unambiguous, > but I'd be okay with mandating them to disambiguate. I'm not sure I like the idea of parens being optional in an "odd" place, e.g. just before 'except'. I feel that such departures from the usual conventions should be a rigid feature of the syntax -- either mandatory or forbidden. -- Greg From greg.ewing at canterbury.ac.nz Sun Feb 16 23:15:13 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 17 Feb 2014 11:15:13 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <20140215195731.GI4281@ando> Message-ID: <53013871.4070602@canterbury.ac.nz> Chris Angelico wrote: > But to the human, it's convenient to think of it as a "dict literal" > in the same way that you can have a "string literal" or "float > literal". Sometimes the word "display" is used for this kind of construct. -- Greg From rosuav at gmail.com Sun Feb 16 23:16:35 2014 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 17 Feb 2014 09:16:35 +1100 Subject: [Python-ideas] except expression In-Reply-To: <20140216173748.GE4519@ando> References: <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <20140215195731.GI4281@ando> <20140216173748.GE4519@ando> Message-ID: On Mon, Feb 17, 2014 at 4:37 AM, Steven D'Aprano wrote: >> (By the way, Steven, what on earth is a Thin Mint that could cause >> your eating to be so exceptional that your stomach explodes? The mind >> boggles...) > > It's a Monty Python reference. It's actually a wafer-thin afterdinner > mint. Huh. I thought I was at least broadly familiar with MP, but that one's slipped me. Time to dig around on Youtube, I think! In some contexts, the unfamiliar will send people to the dictionary [1]. Around the Python lists, the unfamiliar can send people to Youtube. I think that's doing fairly well. :) ChrisA [1] See eg the Dec 18th comment here: https://www.kickstarter.com/projects/lukewilson/500-old-christian-books-republished/comments From rosuav at gmail.com Sun Feb 16 23:19:22 2014 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 17 Feb 2014 09:19:22 +1100 Subject: [Python-ideas] except expression In-Reply-To: <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> Message-ID: On Mon, Feb 17, 2014 at 7:39 AM, Jan Kaliszewski wrote: > Sorry, I don't catch the point. If I needed to use a complex > exception spec (a tuple-like) and/or a tuple as the "default" > expression -- I'd just do it: > > some_io() except (FileNotFoundError: (1, 2, 3), > (ValueError, TypeError): 'spam') > > I see no ambiguity here. Maybe not, but the only thing preventing that from parsing as a tuple containing two tuples is the colon a bit further along. That might be sufficient for the lexer (in the same way that, for instance, function arguments can contain tuples), but I suspect it may be confusing for humans. ChrisA From greg.ewing at canterbury.ac.nz Sun Feb 16 23:33:55 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 17 Feb 2014 11:33:55 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> Message-ID: <53013CD3.8030809@canterbury.ac.nz> > On Mon, Feb 17, 2014 at 7:39 AM, Jan Kaliszewski wrote: > >>Sorry, I don't catch the point. If I needed to use a complex >>exception spec (a tuple-like) and/or a tuple as the "default" >>expression -- I'd just do it: >> >> some_io() except (FileNotFoundError: (1, 2, 3), >> (ValueError, TypeError): 'spam') >> >>I see no ambiguity here. To make it parseable, it would probably be necessary to disallow commas in the expression following the colon (i.e. it would be a 'test' rather than a 'testlist' in terms of the grammar). -- Greg From rosuav at gmail.com Sun Feb 16 23:45:09 2014 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 17 Feb 2014 09:45:09 +1100 Subject: [Python-ideas] except expression In-Reply-To: <53013CD3.8030809@canterbury.ac.nz> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> <53013CD3.8030809@canterbury.ac.nz> Message-ID: On Mon, Feb 17, 2014 at 9:33 AM, Greg Ewing wrote: >> On Mon, Feb 17, 2014 at 7:39 AM, Jan Kaliszewski >> wrote: >> >>> Sorry, I don't catch the point. If I needed to use a complex >>> exception spec (a tuple-like) and/or a tuple as the "default" >>> expression -- I'd just do it: >>> >>> some_io() except (FileNotFoundError: (1, 2, 3), >>> (ValueError, TypeError): 'spam') >>> >>> I see no ambiguity here. > > > To make it parseable, it would probably be necessary to disallow > commas in the expression following the colon (i.e. it would > be a 'test' rather than a 'testlist' in terms of the grammar). Do you mean that there may not be a comma, or that commas must be surrounded by parens? I wouldn't mind *too* much if it's the latter, but I still think this could be confusing. Imagine if the exception list isn't literals, but comes from somewhere else (which is perfectly legal). How do you eyeball it and see that this is now a new except clause? ChrisA From zuo at chopin.edu.pl Sun Feb 16 23:45:12 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sun, 16 Feb 2014 23:45:12 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> Message-ID: 16.02.2014 23:19, Chris Angelico wrote: > On Mon, Feb 17, 2014 at 7:39 AM, Jan Kaliszewski > wrote: >> Sorry, I don't catch the point. If I needed to use a complex >> exception spec (a tuple-like) and/or a tuple as the "default" >> expression -- I'd just do it: >> >> some_io() except (FileNotFoundError: (1, 2, 3), >> (ValueError, TypeError): 'spam') >> >> I see no ambiguity here. > > Maybe not, but the only thing preventing that from parsing as a tuple > containing two tuples is the colon a bit further along. That might be > sufficient for the lexer (in the same way that, for instance, > function > arguments can contain tuples), but I suspect it may be confusing for > humans. IMHO, not more confusing than a dict literal containing tuple literals as keys or values... Cheers. *j From zuo at chopin.edu.pl Sun Feb 16 23:55:46 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sun, 16 Feb 2014 23:55:46 +0100 Subject: [Python-ideas] except expression In-Reply-To: <53013CD3.8030809@canterbury.ac.nz> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> <53013CD3.8030809@canterbury.ac.nz> Message-ID: <191e44b6245713fff08c09a6dc9f6b8a@chopin.edu.pl> 16.02.2014 23:33, Greg Ewing wrote: >> On Mon, Feb 17, 2014 at 7:39 AM, Jan Kaliszewski >> wrote: >> >>>Sorry, I don't catch the point. If I needed to use a complex >>>exception spec (a tuple-like) and/or a tuple as the "default" >>>expression -- I'd just do it: >>> >>> some_io() except (FileNotFoundError: (1, 2, 3), >>> (ValueError, TypeError): 'spam') >>> >>>I see no ambiguity here. > > To make it parseable, it would probably be necessary to disallow > commas in the expression following the colon (i.e. it would > be a 'test' rather than a 'testlist' in terms of the grammar). I believe that it would be somehow similar to what is allowed to be keys and values in dict literals (of course modulo the fact that here only except-like exception specification is allowed as a "key" [exception class or tuple of exception classes], and that an additional optional 'as' clause can be used...). In a way, it can be seen as a *kind of* mapping: expression specs to "default" values... So *some* similarity to a dict literal is not a completely random accident. Cheers. *j From jsbueno at python.org.br Mon Feb 17 03:06:24 2014 From: jsbueno at python.org.br (Joao S. O. Bueno) Date: Sun, 16 Feb 2014 23:06:24 -0300 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: <20140214204820.GE4281@ando> Message-ID: I am stepping late on the thread - but the wish for having a short cut for: if expensive_computation_0(): x = expensive_computation_0() # Do something with x... And havign dealt before with languages where one explicitly deal with the operanbd stack (like postscript), lead me to deploy this very simple module I called "stackfull" (in a wordplay with "stackless") - The functions in there simply annotate a list in the locals() dict of the current running frame (which cannot be accessed as a variable), and work as stack operators on that list - so the situation above become: if push(expensive_computation_0()): x = pop() # Do something with x... (all functions return the original operand, besides side effects on the "stack") It was devised most as a toy, but my wish was to fill the gap Python has for this pattern - so, if anyone is interested in invest in this idea so that we could have a nice, real useful thing to be used in the near future, be my guest. https://pypi.python.org/pypi/stackfull https://github.com/jsbueno/stackfull On 14 February 2014 23:43, Chris Angelico wrote: > On Sat, Feb 15, 2014 at 12:40 PM, Nick Coghlan wrote: >> If the handling is identical, there's no need for an if/elif chain at >> all, otherwise the revised if/elif chain is based on the attributes of >> the match object. > > If the handling's identical, short-circuiting 'or' will do the job. > Assume it's not. > >>> Removing one is just deleting its elif and corresponding block of >>> code. They are peers. The fact that Python doesn't currently have >>> syntax that allows this *in an elif chain* doesn't change this; if you >>> were to write it as pseudo-code, you would write them as peers. That's >>> why the function-with-return or loop-with-break still looks better, >>> because it structures the code the way that logically makes sense - >>> flat, not nested. >> >> That doesn't mean embedded assignments are the answer though - they're >> a sledgehammer solution that is tempting as a workaround for larger >> structural issues. > > I agree that embedded assignment isn't necessarily the answer. I just > think that a solution that has them all at the same indentation level > makes more sense than one that nests them inside each other, for > exactly the same reason that we have 'elif' in the first place. > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From ethan at stoneleaf.us Mon Feb 17 04:27:04 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 16 Feb 2014 19:27:04 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> Message-ID: <53018188.7020700@stoneleaf.us> On 02/16/2014 01:52 PM, Chris Angelico wrote: > On Mon, Feb 17, 2014 at 1:37 AM, Nick Coghlan wrote: >> >> - what happens if an except expression is used as part of the except >> clause in an ordinary try statement, or another except expression? > > That should be fairly obvious - it's like nesting try/except. Does it > need to be said? When writing a PEP about something new, *nothing* is obvious. -- ~Ethan~ From ethan at stoneleaf.us Mon Feb 17 03:47:20 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 16 Feb 2014 18:47:20 -0800 Subject: [Python-ideas] except expression In-Reply-To: <20140215181136.GH4281@ando> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> Message-ID: <53017838.9050706@stoneleaf.us> On 02/15/2014 10:11 AM, Steven D'Aprano wrote: > > Certainly not! pass implies that *no return result is generated at all*, > [...] Don't be silly. def have_a_mint(some, args, here): # flesh this out later pass Does anybody really think that that function will not return None? -- ~Ethan~ From ethan at stoneleaf.us Mon Feb 17 04:31:17 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 16 Feb 2014 19:31:17 -0800 Subject: [Python-ideas] except expression In-Reply-To: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> Message-ID: <53018285.1070003@stoneleaf.us> I see two motivating factors for an except expression: - one line try/except handling for simple cases - easier default value handling when function/method does not support a default value Given that, I don't think we need to support multiple exception types or chained exceptions. If it's that complicated, use an actual try block -- it'll be much more readable. -- ~Ethan~ From rosuav at gmail.com Mon Feb 17 05:04:30 2014 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 17 Feb 2014 15:04:30 +1100 Subject: [Python-ideas] except expression In-Reply-To: <53017838.9050706@stoneleaf.us> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <53017838.9050706@stoneleaf.us> Message-ID: On Mon, Feb 17, 2014 at 1:47 PM, Ethan Furman wrote: > On 02/15/2014 10:11 AM, Steven D'Aprano wrote: >> >> >> Certainly not! pass implies that *no return result is generated at all*, >> [...] > > > Don't be silly. > > def have_a_mint(some, args, here): > # flesh this out later > pass > > Does anybody really think that that function will not return None? Of course it'll return None, but that's nothing to do with the 'pass'. The keyword 'pass' doesn't generate any return result at all. ChrisA From steve at pearwood.info Mon Feb 17 05:46:02 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 17 Feb 2014 15:46:02 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> Message-ID: <20140217044602.GF4519@ando> On Mon, Feb 17, 2014 at 09:19:22AM +1100, Chris Angelico wrote: > On Mon, Feb 17, 2014 at 7:39 AM, Jan Kaliszewski wrote: > > Sorry, I don't catch the point. If I needed to use a complex > > exception spec (a tuple-like) and/or a tuple as the "default" > > expression -- I'd just do it: > > > > some_io() except (FileNotFoundError: (1, 2, 3), > > (ValueError, TypeError): 'spam') > > > > I see no ambiguity here. > > Maybe not, but the only thing preventing that from parsing as a tuple > containing two tuples is the colon a bit further along. That might be > sufficient for the lexer (in the same way that, for instance, function > arguments can contain tuples), but I suspect it may be confusing for > humans. It's certainly confusing for me! I'm now strongly leaning towards following the lead of generator expressions, and requiring parens around an except-expression. Like gen expressions, you can leave the parens out if they are already there: it = (calc(x) for x in iterable) # parentheses are mandatory result = sum(calc(x) for x in iterable) # but this is okay So if your except-expression stands alone, or is inside some other unbracketed expression, you need to bracket it: value = (expr except Error: default) mytuple = (1, 2, (expr except Error: default), 3) But if it's already directly surrounded by parentheses, say inside a function call with no other arguments, there is no need to double-up: result = function(expr except Error: default) I think that this will ensure that there is no visual ambiguity when you chain except-expressions. Even if the parser/lexer can disambiguate the expression, it risks being completely impenetrable to the human reader. Take this case, where I'm labelling parts for ease of discussion: # What does this do? expr except SpamError: spam except EggsError: eggs #^^^^^FIRST^^^^^^^^^^^^^^^^ ^^^^^^SECOND^^^^^^^^^^ Does the SECOND except capture exceptions in the evaluation of "spam" only, or in the entire FIRST except? If this question doesn't make sense to you, then please ponder these two examples: ((expr except SpamError: spam) except EggsError: eggs) (expr except SpamError: (spam except EggsError: eggs)) Requiring parens should also avoid the subtle error of leaving out a comma when you have multiple except-clauses: expr except SpamError: spam, except EggsError: eggs #..........................^ is a single expression that catches two exceptions, equivalent to: try: _ = expr except SpamError: _ = spam except EggsError: _ = eggs whereas leaving out the comma: expr except SpamError: spam except EggsError: eggs is *two* except-expressions, equivalent to either this: try: try: _ = expr except SpamError: _ = spam except EggsError: _ = eggs or this: try: _ = expr except SpamError: try: _ = spam except EggsError: _ = eggs depending on how strongly the except binds. The with- and without-comma versions are very subtly different, and consequently a potential bug magnet, from a very subtle and easy-to-make typo. By requiring parens, you can avoid that. The first version becomes: (expr except SpamError: spam, except EggsError: eggs) and dropping the comma is a SyntaxError, while the second version becomes either: ((expr except SpamError: spam) except EggsError: eggs) (expr except SpamError: (spam except EggsError: eggs)) depending on which semantics you want. -- Steven From steve at pearwood.info Mon Feb 17 06:07:27 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 17 Feb 2014 16:07:27 +1100 Subject: [Python-ideas] except expression In-Reply-To: <53017838.9050706@stoneleaf.us> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <53017838.9050706@stoneleaf.us> Message-ID: <20140217050727.GG4519@ando> On Sun, Feb 16, 2014 at 06:47:20PM -0800, Ethan Furman wrote: > On 02/15/2014 10:11 AM, Steven D'Aprano wrote: > > > >Certainly not! pass implies that *no return result is generated at all*, > >[...] > > Don't be silly. > > def have_a_mint(some, args, here): > # flesh this out later > pass > > Does anybody really think that that function will not return None? But it isn't the *pass* that generates the "return None". It's falling out the bottom of the function that generates the "return None". Do you intend for this function to return None? def trouble_down_mine(args): pass return "NOBODY expects the Spanish Inquisition!!!" "pass" does not mean "return from this function", as I'm sure you know. That's why I object to using "pass" in a form that implies "return None". It is a placeholder, used to satisfy the lexer/parser, and gets compiled out. As far as the compiler is concerned, "pass" simply disappears when compiled: py> from dis import dis py> dis("pass; spam") 1 0 LOAD_NAME 0 (spam) 3 POP_TOP 4 LOAD_CONST 0 (None) 7 RETURN_VALUE You may notice that the compiler currently adds a (redundant?) "return None" to everything(?). That includes "pass", and nothing at all: py> dis("pass") 1 0 LOAD_CONST 0 (None) 3 RETURN_VALUE py> dis("") 1 0 LOAD_CONST 0 (None) 3 RETURN_VALUE -- Steven From steve at pearwood.info Mon Feb 17 06:12:34 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 17 Feb 2014 16:12:34 +1100 Subject: [Python-ideas] except expression In-Reply-To: <53018285.1070003@stoneleaf.us> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <53018285.1070003@stoneleaf.us> Message-ID: <20140217051233.GH4519@ando> On Sun, Feb 16, 2014 at 07:31:17PM -0800, Ethan Furman wrote: > I see two motivating factors for an except expression: > > - one line try/except handling for simple cases > - easier default value handling when function/method does not support a > default value > > Given that, I don't think we need to support multiple exception types or > chained exceptions. If it's that complicated, use an actual try block -- > it'll be much more readable. I'm sympathetic to this view, but I think that by using appropriate parentheses, multiple exception types and multiple exceptions are very readable. Of course, like deeply nested list comps or generator expressions, or too many ternary-if expressions, you can obfuscate your code by abusing this proposed syntax. The answer to that is, Don't Do That. -- Steven From ethan at stoneleaf.us Mon Feb 17 06:05:01 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 16 Feb 2014 21:05:01 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <53017838.9050706@stoneleaf.us> Message-ID: <5301987D.4020103@stoneleaf.us> On 02/16/2014 08:04 PM, Chris Angelico wrote: > On Mon, Feb 17, 2014 at 1:47 PM, Ethan Furman wrote: >> On 02/15/2014 10:11 AM, Steven D'Aprano wrote: >>> >>> >>> Certainly not! pass implies that *no return result is generated at all*, >>> [...] >> >> >> Don't be silly. >> >> def have_a_mint(some, args, here): >> # flesh this out later >> pass >> >> Does anybody really think that that function will not return None? > > Of course it'll return None, but that's nothing to do with the 'pass'. > The keyword 'pass' doesn't generate any return result at all. Precisely. -- ~Ethan~ From steve at pearwood.info Mon Feb 17 06:29:12 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 17 Feb 2014 16:29:12 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> Message-ID: <20140217052912.GJ4519@ando> On Mon, Feb 17, 2014 at 08:52:41AM +1100, Chris Angelico wrote: > Scope of default expressions and 'as' > ------------------------------------- > > In a try/except block, the use of 'as' to capture the exception object creates > a local name binding, and implicitly deletes that binding in a finally clause. > As 'finally' is not a part of this proposal (see below), this makes it tricky > to describe; also, this use of 'as' gives a way to create a name binding in an > expression context. Should the default clause have an inner scope in which the > name exists, shadowing anything of the same name elsewhere? Should it behave > the same way the statement try/except does, and unbind the name? I consider that a wart, and would not like to repeat that unless absolutely necessary. > Should it > bind the name and leave it bound? (Almost certainly not; this behaviour was > changed in Python 3 for good reason.) I think the first option is best: avoid letting the "as" name leak out into the local scope, just like gen expressions and list comps: py> i = "spam" py> x = [i+1 for i in range(10)] py> i 'spam' > It's a knotty problem. I'm currently inclined to the inner scope idea > (like a comprehension), but that's inconsistent with the statement > form of try/except. Is that difference going to confuse people? I didn't think so. I didn't even realise that exceptions unbind the "as" name, instead of using a separate scope. I don't expect anyone is *relying* on that unbind-the-name behaviour, but even if they are, they can just keep using the try...except statement form. > Alternatively, what would it be like if an expression could do a name > binding and unbinding (consistently with the statement form)? Would > that confuse people no end? It would confuse, surprise and annoy me. I presume there are good reasons why the try...except statement does an unbind instead of using a new scope, but I really don't want to repeat that behaviour. -- Steven From rob.cliffe at btinternet.com Mon Feb 17 07:24:53 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Mon, 17 Feb 2014 06:24:53 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> Message-ID: <5301AB35.4020503@btinternet.com> On 16/02/2014 04:04, Chris Angelico wrote: > PEP: XXX > Title: Exception-catching expressions > Version: $Revision$ > Last-Modified: $Date$ > Author: Chris Angelico > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 15-Feb-2014 > Python-Version: 3.5 > Post-History: 16-Feb-2014 > > > Abstract > ======== > > Just as PEP 308 introduced a means of value-based conditions in an > expression, this system allows exception-based conditions to be used as part > of an expression. > > > Motivation > ========== > > A number of functions and methods have parameters which will cause them to > return a specified value instead of raising an exception. The current system > is ad-hoc and inconsistent, and requires that each function be individually > written to have this functionality; not all support this. > > * dict.get(key, default) - second positional argument in place of KeyError > > * next(iter, default) - second positional argument in place of StopIteration > > * list.pop() - no way to return a default > > (TODO: Get more examples. I know there are some but I can't think of any now.) > > > Rationale > ========= > > The current system requires that a function author predict the need for a > default, and implement support for it. If this is not done, a full try/except > block is needed. > > Note that the specific syntax is open to three metric tons of bike-shedding. > > The proposal may well be rejected, but even then is not useless; it can be > maintained as a collection of failed syntaxes for expression exceptions. > > > Proposal > ======== > > Just as the 'or' operator and the three part 'if-else' expression give short > circuiting methods of catching a falsy value and replacing it, this syntax > gives a short-circuiting method of catching an exception and replacing it. > > This currently works:: > lst = [1, 2, None, 3] > value = lst[2] or "No value" > > The proposal adds this:: > lst = [1, 2] > value = lst[2] except IndexError: "No value" > > The exception object can be captured just as in a normal try/except block:: > # Return the next yielded or returned value from a generator > value = next(it) except StopIteration as e: e.args[0] > > This is effectively equivalent to:: > try: > _ = next(it) > except StopIteration as e: > _ = e.args[0] > value = _ > > This ternary operator would be between lambda and if/else in precedence. Oops, I almost missed this (and was going to point out the omission, sorry!). Yes. So in 1 / x except ZeroDivisionError: 42 an error will be trapped evaluating "1/x" (not just "x") because "/" has higher precedence than "except", as we would all expect. > > > Chaining > -------- > > Multiple 'except' keywords can be used, and they will all catch exceptions > raised in the original expression (only):: > value = (expr > except Exception1 [as e]: default1 > except Exception2 [as e]: default2 > # ... except ExceptionN [as e]: defaultN > ) > > This is not the same as either parenthesized form:: > value = (("Yield: "+next(it) except StopIteration as e: "End: "+e.args[0]) > except TypeError: "Error: Non-string returned or raised") > value = (next(it) except StopIteration as e: > (e.args[0] except IndexError: None)) > > The first form will catch an exception raised in either the original > expression or in the default expression; the second form will catch ONLY one > raised by the default expression. All three effects have their uses. > > > Alternative Proposals > ===================== > > Discussion on python-ideas brought up the following syntax suggestions:: > value = expr except default if Exception [as e] > value = expr except default for Exception [as e] > value = expr except default from Exception [as e] > value = expr except Exception [as e] return default > value = expr except (Exception [as e]: default) > value = expr except Exception [as e] try default > value = expr except Exception [as e] continue with default > value = default except Exception [as e] else expr > value = try expr except Exception [as e]: default > value = expr except Exception [as e] pass default > > In all cases, default is an expression which will not be evaluated unless > an exception is raised; if 'as' is used, this expression may refer to the > exception object. > > It has also been suggested that a new keyword be created, rather than reusing > an existing one. Such proposals fall into the same structure as the last > form, but with a different keyword in place of 'pass'. Suggestions include > 'then', 'when', and 'use'. > > > Open Issues > =========== > > finally clause > -------------- > > Should the except expression be able to have a finally clause? No form of the > proposal so far has included finally. I don't see (though I could be missing something) that "finally" makes any sense inside an expression. "finally" introduces an *action* to be performed regardless of whether a preceding operation raised an error. Here we are evaluating an *expression*, and to tack on "I want to perform this action" on the end of an expression seems weird (and better done by the existing try ... finally mechanism). It did occur to me briefly that it might be used to provide a final default: x = eval(UserExpression) except NameError: "You forgot to define something" except ZeroDivisionError: "Division by zero" finally: "Some error or other" but this would be pretty well equivalent to using a bare "except:" clause, or "except BaseException". Oh yes, by the way, I think that a bare "except:" should be legal (for consistency), albeit discouraged. > > > Commas between multiple except clauses > -------------------------------------- > Where there are multiple except clauses, should they be separated by commas? > It may be easier for the parser, that way:: > value = (expr > except Exception1 [as e]: default1, > except Exception2 [as e]: default2, > # ... except ExceptionN [as e]: defaultN, > ) > with an optional comma after the last, as per tuple rules. Downside: Omitting > the comma would be syntactically valid, and would have almost identical > semantics, but would nest the entire preceding expression in its exception > catching rig - a matching exception raised in the default clause would be > caught by the subsequent except clause. As this difference is so subtle, it > runs the risk of being a major bug magnet. I would argue against comma separators being allowed: 1) They add extra grit to the syntax. 2) When the parser sees a comma, it doesn't know if this is terminating the except clause, or part of a tuple default value: value = expr except Exception1: x,y, except Exception2: z (The ambiguity is resolved when it sees another "except". The intervening comma just adds confusion and makes the parser's job *harder*.) And for what it's worth, if you did want a 1-element tuple as your default value you might write the ugly value = expr except Exception1: x,, except Exception2: z # Yuk! Please put "x," inside parentheses. 3) With "no commas", the subtly different semantics you mention would be distinguished transparently by value = expr except Exception1: default1 except Exception2: default2 value = (expr except Exception1: default1) except Exception2: default2 In the first form, Exception2 will not be caught if it is raised evaluating default1. In the second it will be. Rob Cliffe > > > Copyright > ========= > > This document has been placed in the public domain. > > > > > .. > Local Variables: > mode: indented-text > indent-tabs-mode: nil > sentence-end-double-space: t > fill-column: 70 > coding: utf-8 > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > ----- > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6597 - Release Date: 02/16/14 > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Mon Feb 17 07:28:47 2014 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 17 Feb 2014 17:28:47 +1100 Subject: [Python-ideas] except expression In-Reply-To: <20140217052912.GJ4519@ando> References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> Message-ID: On Mon, Feb 17, 2014 at 4:29 PM, Steven D'Aprano wrote: > On Mon, Feb 17, 2014 at 08:52:41AM +1100, Chris Angelico wrote: > >> Scope of default expressions and 'as' >> ------------------------------------- >> >> In a try/except block, the use of 'as' to capture the exception object creates >> a local name binding, and implicitly deletes that binding in a finally clause. >> As 'finally' is not a part of this proposal (see below), this makes it tricky >> to describe; also, this use of 'as' gives a way to create a name binding in an >> expression context. Should the default clause have an inner scope in which the >> name exists, shadowing anything of the same name elsewhere? Should it behave >> the same way the statement try/except does, and unbind the name? > > I consider that a wart, and would not like to repeat that unless > absolutely necessary. Yeah, the unbinding is really weird. I didn't know about it until I started experimenting, too... and then went digging in the docs and found out that it implicitly creates a 'finally' that dels the name. >> It's a knotty problem. I'm currently inclined to the inner scope idea >> (like a comprehension), but that's inconsistent with the statement >> form of try/except. Is that difference going to confuse people? > > I didn't think so. I didn't even realise that exceptions unbind the "as" > name, instead of using a separate scope. I don't expect anyone is > *relying* on that unbind-the-name behaviour, but even if they are, > they can just keep using the try...except statement form. The inner scope does seem to demand a function call, though. Maybe this is a good opportunity to introduce the concept of "sub-scopes" for specific constructs. Comprehensions and 'as' clauses (I doubt anyone will object to the statement try/except being brought in line with this) could then use a sub-scope inside the existing function; it'd affect only the LOAD_FAST opcode, I think - assignment could delete it from the sub-scope and put it into the main scope. This would mean that: try: f() except Exception as e: e = e print(e) would print out the exception. But that would be a completely separate PEP. I'd like it to happen, but it's not part of exception expressions. ChrisA From rosuav at gmail.com Mon Feb 17 07:40:11 2014 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 17 Feb 2014 17:40:11 +1100 Subject: [Python-ideas] except expression In-Reply-To: <5301AB35.4020503@btinternet.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <5301AB35.4020503@btinternet.com> Message-ID: On Mon, Feb 17, 2014 at 5:24 PM, Rob Cliffe wrote: > So in > 1 / x except ZeroDivisionError: 42 > an error will be trapped evaluating "1/x" (not just "x") because "/" has > higher precedence than "except", as we would all expect. Yes, definitely. Operator precedence can (and should) be used to make this do exactly what everyone expects. Though there's a proposal on the board to demand parens: (1 / x except ZeroDivisionError: 42) which would settle the deal. > I don't see (though I could be missing something) that "finally" makes any > sense inside an expression. "finally" introduces an action to be performed > regardless of whether a preceding operation raised an error. Here we are > evaluating an expression, and to tack on "I want to perform this action" on > the end of an expression seems weird (and better done by the existing try > ... finally mechanism). It did occur to me briefly that it might be used to > provide a final default: > x = eval(UserExpression) except NameError: "You forgot to define > something" except ZeroDivisionError: "Division by zero" finally: "Some error > or other" > but this would be pretty well equivalent to using a bare "except:" clause, > or "except BaseException". Agreed, finally makes no use. That section has been reworded, but is worth keeping. > Oh yes, by the way, I think that a bare "except:" should be legal (for > consistency), albeit discouraged. Agreed. Never even thought about it, but yes, that's legal. No reason to forbid it. Added a sentence to the PEP to say so. > I would argue against comma separators being allowed: > 1) They add extra grit to the syntax. > 2) When the parser sees a comma, it doesn't know if this is terminating > the except clause, or part of a tuple default value: > value = expr except Exception1: x,y, except Exception2: z > (The ambiguity is resolved when it sees another "except". The > intervening comma just adds confusion and makes the parser's job harder.) > And for what it's worth, if you did want a 1-element tuple as your > default value you might write the ugly > value = expr except Exception1: x,, except Exception2: z # Yuk! > Please put "x," inside parentheses. > 3) With "no commas", the subtly different semantics you mention would be > distinguished transparently by > > value = expr except Exception1: default1 except Exception2: default2 > value = (expr except Exception1: default1) except Exception2: > default2 > In the first form, Exception2 will not be caught if it is raised evaluating > default1. In the second it will be. I'm not convinced either way about the commas, at the moment. At some point, I'm probably going to have to come up with a bunch of actual examples (real-world or synthesized) and then show how each would be written as a statement, and then as an expression, and show the latter with the full cross-multiplication of proposals: commas or not, parens or not, etc, etc, etc. In any case, I'm pretty happy with the level of interest this is generating. A number of people have strong opinions on the matter. That's a good sign! I'm happy to re-post the current version of the PEP any time people like; or, if it makes sense, I could shove the repository onto github so people can see the changes live. At some point, of course, the authoritative text will be in the official PEP repository, but until then, I'm open to suggestions. ChrisA From rosuav at gmail.com Mon Feb 17 08:00:56 2014 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 17 Feb 2014 18:00:56 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <5301AB35.4020503@btinternet.com> Message-ID: On Mon, Feb 17, 2014 at 5:40 PM, Chris Angelico wrote: > I'm happy to re-post the current version of the PEP any time people > like; or, if it makes sense, I could shove the repository onto github > so people can see the changes live. At some point, of course, the > authoritative text will be in the official PEP repository, but until > then, I'm open to suggestions. Since github's easy for me to do, I've posted it there. Here's the current version of the PEP; this URL will point to the latest, until it gets accepted into the main PEPs repo. https://raw2.github.com/Rosuav/ExceptExpr/master/pep.txt Let me know if I've missed anything. I've been reading every message that goes through, but sometimes I don't properly reflect comments into the draft PEP. ChrisA From p.f.moore at gmail.com Mon Feb 17 08:43:08 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 17 Feb 2014 07:43:08 +0000 Subject: [Python-ideas] except expression In-Reply-To: <20140217044602.GF4519@ando> References: <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> <20140217044602.GF4519@ando> Message-ID: On 17 February 2014 04:46, Steven D'Aprano wrote: >> > some_io() except (FileNotFoundError: (1, 2, 3), >> > (ValueError, TypeError): 'spam') >> > >> > I see no ambiguity here. >> >> Maybe not, but the only thing preventing that from parsing as a tuple >> containing two tuples is the colon a bit further along. That might be >> sufficient for the lexer (in the same way that, for instance, function >> arguments can contain tuples), but I suspect it may be confusing for >> humans. > > It's certainly confusing for me! Just as a data point - with my eyesight and the monitor resolution/font I use, I can't even *see* the colon without squinting. There's an old principle "new syntax should not look like grit on Tim's monitor" that probably applies here :-) Paul From steve at pearwood.info Mon Feb 17 09:40:05 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 17 Feb 2014 19:40:05 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> <20140217044602.GF4519@ando> Message-ID: <20140217084005.GK4519@ando> On Mon, Feb 17, 2014 at 07:43:08AM +0000, Paul Moore wrote: > Just as a data point - with my eyesight and the monitor > resolution/font I use, I can't even *see* the colon without squinting. > There's an old principle "new syntax should not look like grit on > Tim's monitor" that probably applies here :-) I sympathise, but you already have to deal with dicts, slices, and lambda, and with the exception (pun intended) of lambda, they're likely to be much more common than except-expression. -- Steven From p.f.moore at gmail.com Mon Feb 17 10:56:21 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 17 Feb 2014 09:56:21 +0000 Subject: [Python-ideas] except expression In-Reply-To: <20140217084005.GK4519@ando> References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> <20140217044602.GF4519@ando> <20140217084005.GK4519@ando> Message-ID: On 17 February 2014 08:40, Steven D'Aprano wrote: > I sympathise, but you already have to deal with dicts, slices, and > lambda, and with the exception (pun intended) of lambda, they're likely > to be much more common than except-expression. I should have expanded :-) (See? I also have to deal with smilies!) With dicts slices and lambdas, the colon is essentially not so important for the *human* reader, although it is needed for the parser. If dicts were simply {key, value, key, value, ...} they would still make sense to the human reader, just by pairing things up and by the fact that anyone writing something complicate would lay it out readably. When discussing new syntax, it becomes very easy to focus on the corner cases and how to make them unambiguous and/or clearer than the alternatives. In my view, that's not the point - what matters is whether the syntax cleanly expresses the simpler cases that are the ones motivating the change in the first place. I'm more than happy to relegate the corner cases to a status of "well, it works, but you should be using the statement form when it's this complex anyway". My point with the example I quoted (reproduced here for reference) >> > some_io() except (FileNotFoundError: (1, 2, 3), >> > (ValueError, TypeError): 'spam') is precisely that - the subtle change in meaning based on a relatively hard to spot colon is *not* something that matters when debating the relative merits of syntax proposals, because no-one should be writing code like this in the first place. The parallel with dicts is misleading, as you can't write my_dict = {1,2:3,4:5,6,7:8} - first of all, you have to parenthesize the relevant tuples, and secondly nobody sane would write it like that, they'd use layout to make the intent clear to the human reader). Anyway, I don't want to beat up on one example. I'm just trying to make the points 1. Let's worry about making the basic case as readable as possible, rather than the complex cases 2. Punctuation for disambiguation can be a problem - I'd prefer it if the keywords stood by themselves in this case 3. Needing extra parentheses to disambiguate (or even requiring them - consider yield expressions or generator expressions) is not a bad thing, nor is needing to make complex cases multi-line - human readability is more important than using the minimum syntax that the grammar requires. Paul From mal at egenix.com Mon Feb 17 11:00:38 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 17 Feb 2014 11:00:38 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> Message-ID: <5301DDC6.6020505@egenix.com> On 16.02.2014 05:04, Chris Angelico wrote: > PEP: XXX > Title: Exception-catching expressions > Version: $Revision$ > Last-Modified: $Date$ > Author: Chris Angelico > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 15-Feb-2014 > Python-Version: 3.5 > Post-History: 16-Feb-2014 > > > Abstract > ======== > > Just as PEP 308 introduced a means of value-based conditions in an > expression, this system allows exception-based conditions to be used as part > of an expression. Thanks for writing up this PEP. > Motivation > ========== > > A number of functions and methods have parameters which will cause them to > return a specified value instead of raising an exception. The current system > is ad-hoc and inconsistent, and requires that each function be individually > written to have this functionality; not all support this. > > * dict.get(key, default) - second positional argument in place of KeyError > > * next(iter, default) - second positional argument in place of StopIteration > > * list.pop() - no way to return a default > > (TODO: Get more examples. I know there are some but I can't think of any now.) > > > Rationale > ========= > > The current system requires that a function author predict the need for a > default, and implement support for it. If this is not done, a full try/except > block is needed. > > Note that the specific syntax is open to three metric tons of bike-shedding. > > The proposal may well be rejected, but even then is not useless; it can be > maintained as a collection of failed syntaxes for expression exceptions. > > > Proposal > ======== > > Just as the 'or' operator and the three part 'if-else' expression give short > circuiting methods of catching a falsy value and replacing it, this syntax > gives a short-circuiting method of catching an exception and replacing it. > > This currently works:: > lst = [1, 2, None, 3] > value = lst[2] or "No value" > > The proposal adds this:: > lst = [1, 2] > value = lst[2] except IndexError: "No value" The colon in there breaks the basic Python concept of having colons end headers which start a new block of statements: http://docs.python.org/reference/compound_stmts.html I think it would be better to have the PEP focus on a solution that doesn't introduce more lambda like syntax to the language :-) Please also add examples where the expression notation is used as part of a larger expression, e.g. value = lst[2] except IndexError: 0, 1 value = lst[2] except IndexError: 0 if x else -1 value = lst[2] except IndexError: 0 if lst[1] except IndexError: False else -1 value = lst[2] except IndexError: 0 + lst[0] d = {'a': lst[2] except IndexError: 0, 'b': lst[1] except IndexError: 0 if lst[0] else 1, 'c': lst[0], } Or code like this: try: x = lst[0] except IndexError: x = lst[2] except Exception: lst[1] l.sort(key = lambda x: x[1] except IndexError: x[0]) IMO, the colon in there makes the code less readable than the variants which try to reuse a keyword. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 17 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From denis.spir at gmail.com Mon Feb 17 11:22:56 2014 From: denis.spir at gmail.com (spir) Date: Mon, 17 Feb 2014 11:22:56 +0100 Subject: [Python-ideas] Enhance exceptions (and improve tracebakcs) In-Reply-To: References: Message-ID: <5301E300.2070202@gmail.com> On 02/15/2014 04:40 PM, Sebastian Kreft wrote: > More than once I've been in a situation where I wish that some of the > stdlib exceptions had a better message or some more information to help me > diagnose the problem. > > For example: > a = [1, 2, 3, 4, 5] > a[5] > IndexError: list index out of range > > In this case there's no reference to neither the length of the array nor to > the offending index. > > I'm of the idea that we could extend the exceptions by adding some more > information to them, so 3rd party libraries could use them for > debugging/logging. > > For example, one such use case would be to have a modified test runner, > that in case of exceptions automatically prints some debug information. > Another would be a logger system that in case of an exception also logs > some debug info that could be relevant to understand and solve the issue. > > I propose extending (at least) the following exceptions with the following > attributes: > KeyError: key, object > IndexError: index, object > AttributeError: attribute, object > NameError: name > > Of course that populating these attributes could be controlled by a flag. > > I know that some of this information is already present in some exceptions, > depending on the context. However, I propose adding these attributes, as in > this way a tool could easily and reliably extract the information and work > with it, as opposed to have to parse the huge variety of different messages > there are. > > For the first use case mentioned above I have a working prototype, although > it does not use this proposal, but a series of hacks (I'm modifying the > bytecode to save a reference to the key and object :() and parsing of the > exception messages. But I want to share what the output of such a tool > could be. > > ====================================================================== > ERROR: test_attribute (example.ExampleTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/home/skreft/test/debug_exception/example.py.py", line 18, in > test_attribute > AttributeError: 'str' object has no attribute 'Lower'. Did you mean > 'islower', 'lower'? > Debug info: > Object: '' > Type: > > ====================================================================== > ERROR: test_index (example.ExampleTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/home/skreft/test/debug_exception/example.py.py", line 6, in > test_index > IndexError: list index out of range > Debug info: > Object: [1, 2] > Object len: 2 > Index: 2 > > ====================================================================== > ERROR: test_key (example.ExampleTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/home/skreft/test/debug_exception/example.py.py", line 10, in > test_key > KeyError_: 'fooo', did you mean 'foo'? > Debug info: > Object: {'foo': 1} > Key: 'fooo' > > ====================================================================== > ERROR: test_name (example.ExampleTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/home/skreft/test/debug_exception/example.py.py", line 14, in > test_name > NameError: global name 'fooo' is not defined. Did you mean 'foo'? > > ---------------------------------------------------------------------- > Ran 4 tests in 0.005s I very much approve this proposal (also the comments by Nick). Actually, I had a similar idea of systematically adding _relevant data_ to error reports; and also proposing it as standard practice in guidelines, not only when using standard error types, but for custom ones too. That these data are set as attributes on error objects, making them reusable in custom error report formats, is also an excellent point. I would however take the opporunity to improve a bit the traceback part of error reports by (1) renaming it (2) setting it apart (with a simple line). (As they are now, tracebacks are real enigmas to novice (python) programmers, and they mess up the rest of error messages.) For instance: ====================================================================== ERROR: test_key (example.ExampleTest) ---------------------------------------------------------------------- KeyError_: 'fooo', did you mean 'foo'? Debug info: Object: {'foo': 1} Key: 'fooo' ---------------------------------------------------------------------- Function call chain chronology: ---------------------------------------------------------------------- File "_.py", line 9, in main() File "_.py", line 7, in main test() File "_.py", line 5, in test test_key() File "_.py", line 3, in test_key print(x['fooo']) ====================================================================== Maybe "Function call chain" in not the best term --propose your own-- but it still helps and understand what it's all about. "Chronology" seems self-explaining to me and avoids "(most recent call last)". Placing the tracback after the message just reflects the fact that users (especially novice ones) read top down. I do agree that experienced programmers often read python error messages backwards --starting at the bottom-- but it's because what's usually the most relevant info is placed there, at the end of the message, in standard python error format: Traceback (most recent call last): File "_.py", line 10, in main() File "_.py", line 8, in main test() File "_.py", line 6, in test test_key() File "_.py", line 4, in test_key print(x['fooo']) KeyError: 'fooo' But I would not fight for this. d From p.f.moore at gmail.com Mon Feb 17 11:45:52 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 17 Feb 2014 10:45:52 +0000 Subject: [Python-ideas] except expression In-Reply-To: <5301DDC6.6020505@egenix.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <5301DDC6.6020505@egenix.com> Message-ID: On 17 February 2014 10:00, M.-A. Lemburg wrote: > Please also add examples where the expression notation is > used as part of a larger expression +1. When I see isolated exception-catching expressions as examples, I think in terms of assigning them to something, and that is often better done with a try/except statement. So it doesn't show the benefits of the expression form. Paul From markus at unterwaditzer.net Mon Feb 17 14:12:44 2014 From: markus at unterwaditzer.net (Markus Unterwaditzer) Date: Mon, 17 Feb 2014 14:12:44 +0100 Subject: [Python-ideas] docs.python.org: Short URLs Message-ID: <20140217131244.GA7967@untibox.unti> http://cryto.net/~joepie91/blog/2013/02/19/the-python-documentation-is-bad-and-you-should-feel-bad/ A few of you might have read that infamous rant about Python's community and documentation and Brian Curtins response: http://blog.briancurtin.com/posts/why-i-dont-feel-so-bad.html Like Brian, i think that the unconstructive tone ruined the points the author was trying to make, a part of which i actually agree with. So, here is an attempt at a positive consequence of this rant. The author talks about the inaccessibility of Python's documentation via Google compared to PHP's. One can easily verify that by themselves: Just enter the name of any builtin into the search engine at docs.python.org, such as str, float, or print. In none of these cases, the appropriate documentation is the first result. The search engine at php.net is, on the other hand, a breeze to use, probably because the search engine doesn't have to take namespaces into account when searching. The suggestion i want to make isn't just "make the search engine better". I know that such things are better filed against Sphinx. What i actually wanted to suggest is a convenience feature that is available at php.net: http://php.net/str_replace directly shows the documentation of PHP's str_replace. Concretely, I propose that, for example, http://docs.python.org/2/str redirects to the documentation for `str` in Python 2 , while http://docs.python.org/3/str, http://docs.python.org/str and/or http://python.org/str redirect to the documentation for `str` in Python 3. Here is a simple script that roughly shows how i would expect this redirect to resolve the given object names: import sphinx.ext.intersphinx as intersphinx class FakeSphinxApp(object): def warn(self, x): print(x) baseurl = 'http://docs.python.org/3/' inv = intersphinx.fetch_inventory(FakeSphinxApp(), baseurl, baseurl + 'objects.inv') index = {} for value in inv.values(): index.update(value) def url_for_object(s): return index[s][2] # >>> url_for_object('str') # 'http://docs.python.org/3/library/stdtypes.html#str' # >>> url_for_object('datetime.datetime') # 'http://docs.python.org/3/library/datetime.html#datetime.datetime' I am aware that the pydoc command-line utility ships with the default distribution of Python. However, i don't think any beginner wants to get in touch with the command line more than what is absolutely neccessary, and i also doubt that people are aware of this utility. I personally also haven't used pydoc and Python's `help` builtin mostly because of this: $ pydoc datetime.datetime Help on class datetime in datetime: datetime.datetime = class datetime(date) | datetime(year, month, day[, hour[, minute[, second[, microsecond[,tzinfo]]]]]) | | The year, month and day arguments are required. tzinfo may be None, or an | instance of a tzinfo subclass. The remaining arguments may be ints. | | Method resolution order: | datetime | date | builtins.object | | Methods defined here: | | __add__(...) | x.__add__(y) <==> x+y | | __eq__(...) | x.__eq__(y) <==> x==y | | __ge__(...) | x.__ge__(y) <==> x>=y | | __getattribute__(...) | x.__getattribute__('name') <==> x.name ... What do you think? -- Markus From rob.cliffe at btinternet.com Mon Feb 17 14:38:13 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Mon, 17 Feb 2014 13:38:13 +0000 Subject: [Python-ideas] Enhance exceptions by attaching some more information to them In-Reply-To: References: Message-ID: <530210C5.8000406@btinternet.com> +1 on the idea (I can't comment on the implementation). One of Python's great strengths is its excellent error messages. But if they can be made still better, that would be fantastic. Rob Cliffe On 15/02/2014 15:40, Sebastian Kreft wrote: > More than once I've been in a situation where I wish that some of the > stdlib exceptions had a better message or some more information to > help me diagnose the problem. > > For example: > a = [1, 2, 3, 4, 5] > a[5] > IndexError: list index out of range > > In this case there's no reference to neither the length of the array > nor to the offending index. > > I'm of the idea that we could extend the exceptions by adding some > more information to them, so 3rd party libraries could use them for > debugging/logging. > > For example, one such use case would be to have a modified test > runner, that in case of exceptions automatically prints some debug > information. > Another would be a logger system that in case of an exception also > logs some debug info that could be relevant to understand and solve > the issue. > > I propose extending (at least) the following exceptions with the > following attributes: > KeyError: key, object > IndexError: index, object > AttributeError: attribute, object > NameError: name > > Of course that populating these attributes could be controlled by a flag. > > I know that some of this information is already present in some > exceptions, depending on the context. However, I propose adding these > attributes, as in this way a tool could easily and reliably extract > the information and work with it, as opposed to have to parse the huge > variety of different messages there are. > > For the first use case mentioned above I have a working prototype, > although it does not use this proposal, but a series of hacks (I'm > modifying the bytecode to save a reference to the key and object :() > and parsing of the exception messages. But I want to share what the > output of such a tool could be. > > ====================================================================== > ERROR: test_attribute (example.ExampleTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/home/skreft/test/debug_exception/example.py.py > ", line 18, in test_attribute > AttributeError: 'str' object has no attribute 'Lower'. Did you mean > 'islower', 'lower'? > Debug info: > Object: '' > Type: > > ====================================================================== > ERROR: test_index (example.ExampleTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/home/skreft/test/debug_exception/example.py.py > ", line 6, in test_index > IndexError: list index out of range > Debug info: > Object: [1, 2] > Object len: 2 > Index: 2 > > ====================================================================== > ERROR: test_key (example.ExampleTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/home/skreft/test/debug_exception/example.py.py > ", line 10, in test_key > KeyError_: 'fooo', did you mean 'foo'? > Debug info: > Object: {'foo': 1} > Key: 'fooo' > > ====================================================================== > ERROR: test_name (example.ExampleTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/home/skreft/test/debug_exception/example.py.py > ", line 14, in test_name > NameError: global name 'fooo' is not defined. Did you mean 'foo'? > > ---------------------------------------------------------------------- > Ran 4 tests in 0.005s > > -- > Sebastian Kreft > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6597 - Release Date: 02/16/14 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zuo at chopin.edu.pl Mon Feb 17 15:10:07 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Mon, 17 Feb 2014 15:10:07 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> <20140217044602.GF4519@ando> <20140217084005.GK4519@ando> Message-ID: <5ee68e9147b0b401597c6eb6d9c2f15d@chopin.edu.pl> W dniu 17.02.2014 10:56, Paul Moore napisa?(a): [...] >>> > some_io() except (FileNotFoundError: (1, 2, 3), >>> > (ValueError, TypeError): 'spam') > > is precisely that - the subtle change in meaning based on a > relatively > hard to spot colon [...] Please note that in case of the cited example (of the syntax variant proposed by me, i.e. the parens-after-except one) colon does not provide a subtle change but constitutes the whole expression syntax -- without the colon(s) it would just cause SytaxError. Cheers. *j From zuo at chopin.edu.pl Mon Feb 17 15:25:26 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Mon, 17 Feb 2014 15:25:26 +0100 Subject: [Python-ideas] except expression In-Reply-To: <20140217044602.GF4519@ando> References: <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> <20140217044602.GF4519@ando> Message-ID: <79548e2ac23cfad6d9849385445db01b@chopin.edu.pl> 17.02.2014 05:46, Steven D'Aprano wrote: > On Mon, Feb 17, 2014 at 09:19:22AM +1100, Chris Angelico wrote: >> On Mon, Feb 17, 2014 at 7:39 AM, Jan Kaliszewski >> wrote: >> > Sorry, I don't catch the point. If I needed to use a complex >> > exception spec (a tuple-like) and/or a tuple as the "default" >> > expression -- I'd just do it: >> > >> > some_io() except (FileNotFoundError: (1, 2, 3), >> > (ValueError, TypeError): 'spam') >> > >> > I see no ambiguity here. >> >> Maybe not, but the only thing preventing that from parsing as a >> tuple >> containing two tuples is the colon a bit further along. That might >> be >> sufficient for the lexer (in the same way that, for instance, >> function >> arguments can contain tuples), but I suspect it may be confusing for >> humans. > > It's certainly confusing for me! > > I'm now strongly leaning towards following the lead of generator > expressions, and requiring parens around an except-expression. Like > gen > expressions, you can leave the parens out if they are already there: > > it = (calc(x) for x in iterable) # parentheses are mandatory > result = sum(calc(x) for x in iterable) # but this is okay > > So if your except-expression stands alone, or is inside some other > unbracketed expression, you need to bracket it: > > value = (expr except Error: default) > mytuple = (1, 2, (expr except Error: default), 3) > > > But if it's already directly surrounded by parentheses, say inside a > function call with no other arguments, there is no need to double-up: > > result = function(expr except Error: default) Now, I am confused a bit. First you cite a discussion about the paren-after-except syntax variant (the one I proposed) but then you discuss the syntax variant from the PEP draft by Chris. Please note that these variants are different. See below... > I think that this will ensure that there is no visual ambiguity when > you > chain except-expressions. Even if the parser/lexer can disambiguate > the > expression, it risks being completely impenetrable to the human > reader. > Take this case, where I'm labelling parts for ease of discussion: > > # What does this do? > expr except SpamError: spam except EggsError: eggs > #^^^^^FIRST^^^^^^^^^^^^^^^^ ^^^^^^SECOND^^^^^^^^^^ > > Does the SECOND except capture exceptions in the evaluation of "spam" > only, or in the entire FIRST except? If this question doesn't make > sense > to you, then please ponder these two examples: > > ((expr except SpamError: spam) except EggsError: eggs) > (expr except SpamError: (spam except EggsError: eggs)) > > > Requiring parens should also avoid the subtle error of leaving out a > comma when you have multiple except-clauses: > > expr except SpamError: spam, except EggsError: eggs > #..........................^ > > is a single expression that catches two exceptions, equivalent to: > > try: > _ = expr > except SpamError: > _ = spam > except EggsError: > _ = eggs > > > whereas leaving out the comma: > > expr except SpamError: spam except EggsError: eggs > > is *two* except-expressions, equivalent to either this: > > try: > try: > _ = expr > except SpamError: > _ = spam > except EggsError: > _ = eggs > > > or this: > > try: > _ = expr > except SpamError: > try: > _ = spam > except EggsError: > _ = eggs > > > depending on how strongly the except binds. > > The with- and without-comma versions are very subtly different, and > consequently a potential bug magnet, from a very subtle and > easy-to-make > typo. By requiring parens, you can avoid that. The first version > becomes: > > (expr except SpamError: spam, except EggsError: eggs) > > and dropping the comma is a SyntaxError, while the second version > becomes either: > > ((expr except SpamError: spam) except EggsError: eggs) > (expr except SpamError: (spam except EggsError: eggs)) > > > depending on which semantics you want. On the other hand the paren-after-except variant makes it unambiguous: expr except (SpamError: spam, EggsError: eggs) is clearly different from: expr except (SpamError: spam) except (EggsError: eggs) ...which is the same as: (expr except (SpamError: spam)) except (EggsError: eggs) ...as neither "(SpamError: spam)" nor "except (SpamError: spam)" are valid expressions themselves. But only "EXPRESSION except (SpamError: spam)" taken as a whole makes up a valid expression. Cheers. *j From zuo at chopin.edu.pl Mon Feb 17 15:27:44 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Mon, 17 Feb 2014 15:27:44 +0100 Subject: [Python-ideas] except expression In-Reply-To: <5ee68e9147b0b401597c6eb6d9c2f15d@chopin.edu.pl> References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> <20140217044602.GF4519@ando> <20140217084005.GK4519@ando> <5ee68e9147b0b401597c6eb6d9c2f15d@chopin.edu.pl> Message-ID: 17.02.2014 15:10, Jan Kaliszewski wrote: [...] > Please note that in case of the cited example (of the syntax variant > proposed by me, i.e. the parens-after-except one) colon does not > provide a subtle change but constitutes the whole expression syntax [...] Together with the 'except' keyword of course. *j From rob.cliffe at btinternet.com Mon Feb 17 15:33:34 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Mon, 17 Feb 2014 14:33:34 +0000 Subject: [Python-ideas] except expression In-Reply-To: <5301DDC6.6020505@egenix.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <5301DDC6.6020505@egenix.com> Message-ID: <53021DBE.5030006@btinternet.com> On 17/02/2014 10:00, M.-A. Lemburg wrote: > On 16.02.2014 05:04, Chris Angelico wrote: > The colon in there breaks the basic Python concept of having > colons end headers which start a new block of statements: > > http://docs.python.org/reference/compound_stmts.html > > I think it would be better to have the PEP focus on a solution > that doesn't introduce more lambda like syntax to the language :-) > > Please also add examples where the expression notation is > used as part of a larger expression, e.g. > > value = lst[2] except IndexError: 0, 1 > value = lst[2] except IndexError: 0 if x else -1 > value = lst[2] except IndexError: 0 if lst[1] except IndexError: False else -1 > value = lst[2] except IndexError: 0 + lst[0] > d = {'a': lst[2] except IndexError: 0, > 'b': lst[1] except IndexError: 0 if lst[0] else 1, > 'c': lst[0], > } > > Just wanted to point out the ambiguity in some of these, at least to a human reader. If I am not mistaken, the proposed precedence for "except" (higher than "lambda", lower than everything else including "if - else") means that: The first one means value = lst[2] except IndexError: (0,1) not value = (lst[2] except IndexError: 0), 1 The second means value = lst[2] except IndexError: (0 if x else -1) not value = (lst[2] except IndexError: 0) if x else -1 The third means value = lst[2] except IndexError: (0 if (lst[1] except IndexError: False) else -1) rather than anything else (this is getting a bit mind-boggling). The fourth means value = lst[2] except IndexError: (0 + lst[0]) rather than value = (lst[2] except IndexError: 0) + lst[0] I wonder if the actual meaning coincides in all cases with your intention when you wrote them? No offence intended, MAL, I'm not trying to slag you off, just pointing out some of the issues that arise. :-) And wondering if there is a case for "except" having a higher precedence than "if - else". I don't have an opinion yet, I'm just thinking aloud. The last one is I believe unambiguous except for the value associated with the key 'b', but to my mind is another reason for not allowing commas in the proposed "except expression" syntax - confusion with commas as separators between dictionary literals (sorry I keep plugging this!). There is already ISTM a clash between comma as separator between dictionary items and between tuple elements: >>> { 'a' : (1,3), 'b' : 2 } { 'a': (1, 3), 'b': 2 } >>> { 'a' : 1,3, 'b' : 2 } # Python 2.7.3 SyntaxError: invalid syntax so I think it would be a mistake to overload the comma further. Best wishes, Rob Cliffe From rosuav at gmail.com Mon Feb 17 15:34:25 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 18 Feb 2014 01:34:25 +1100 Subject: [Python-ideas] docs.python.org: Short URLs In-Reply-To: <20140217131244.GA7967@untibox.unti> References: <20140217131244.GA7967@untibox.unti> Message-ID: On Tue, Feb 18, 2014 at 12:12 AM, Markus Unterwaditzer wrote: > What i actually wanted to suggest is a convenience feature that is available at > php.net: http://php.net/str_replace directly shows the documentation of PHP's > str_replace. Concretely, I propose that, for example, > http://docs.python.org/2/str redirects to the documentation for `str` in Python > 2 , while http://docs.python.org/3/str, http://docs.python.org/str and/or > http://python.org/str redirect to the documentation for `str` in Python 3. Here > is a simple script that roughly shows how i would expect this redirect to > resolve the given object names: I like the idea, but your URLs might conflict with other things. Would it be too awkward if namespaced? http://docs.python.org/[ver]/kw/str and then have a keyword index (hence "kw", which is nice and short) which could have function/class names, common search keywords, a few aliases (regex -> re), etc, etc. Pushing those sorts of URLs would help with the ease of finding docs, though. Usually when I want to look up something specific, it takes me two or three clicks _after_ a Google search. (Mind you, a lot of the linked-to rant about listing error conditions won't ever happen in Python, because Python is not PHP. You can list every error return from a PHP function *because every one of them had to be coded in explicitly*. How can you list every exception a Python function might throw back at you? You'd have to recurse into everything that function calls, and see what they might raise. And that requires knowing the types of every object used, including those passed in as parameters... so, pretty much impossible.) ChrisA From markus at unterwaditzer.net Mon Feb 17 15:51:58 2014 From: markus at unterwaditzer.net (Markus Unterwaditzer) Date: Mon, 17 Feb 2014 15:51:58 +0100 Subject: [Python-ideas] docs.python.org: Short URLs In-Reply-To: References: <20140217131244.GA7967@untibox.unti> <20140217144225.GA28593@untibox.unti> Message-ID: <321523c2-268f-43fe-bdbd-0dc203b85575@email.android.com> I did mean to take it to the list. Just replying so it is there content-wise. On 17 February 2014 15:49:16 CET, Chris Angelico wrote: >Not sure if you meant to mail me this privately rather than post >on-list. I'll keep this private for now, but feel free to take it to >the list. > >On Tue, Feb 18, 2014 at 1:42 AM, Markus Unterwaditzer > wrote: >> On Tue, Feb 18, 2014 at 01:34:25AM +1100, Chris Angelico wrote: >>> I like the idea, but your URLs might conflict with other things. >> >> I'd just display the other thing instead then. Actually i think this >should be >> implemented in the 404 error handler or something like that. > >That could be done, but I was thinking more of a safe way to type in a >URL - maybe with a browser plugin, even - that can't possibly bring up >the "wrong thing". > >>> Would it be too awkward if namespaced? >>> >>> http://docs.python.org/[ver]/kw/str >>> >>> and then have a keyword index (hence "kw", which is nice and short) >>> which could have function/class names, common search keywords, a few >>> aliases (regex -> re), etc, etc. >> >> We can have that too, additionally. /name/ would strike me as more >obvious, >> though this is just a matter of preference. > >Either way. "kw" was the best thing I could come up with at short >notice. My point is to namespace it; what the name actually is can >always be bikeshedded :) > >ChrisA From phd at phdru.name Mon Feb 17 15:55:00 2014 From: phd at phdru.name (Oleg Broytman) Date: Mon, 17 Feb 2014 15:55:00 +0100 Subject: [Python-ideas] docs.python.org: Short URLs In-Reply-To: <321523c2-268f-43fe-bdbd-0dc203b85575@email.android.com> References: <20140217131244.GA7967@untibox.unti> <20140217144225.GA28593@untibox.unti> <321523c2-268f-43fe-bdbd-0dc203b85575@email.android.com> Message-ID: <20140217145500.GA28299@phdru.name> On Mon, Feb 17, 2014 at 03:51:58PM +0100, Markus Unterwaditzer wrote: > >>> http://docs.python.org/[ver]/kw/str > > > >Either way. "kw" was the best thing I could come up with at short > >notice. My point is to namespace it; what the name actually is can > >always be bikeshedded :) Painting it 'term'. Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From mal at egenix.com Mon Feb 17 16:00:28 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 17 Feb 2014 16:00:28 +0100 Subject: [Python-ideas] except expression In-Reply-To: <53021DBE.5030006@btinternet.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <5301DDC6.6020505@egenix.com> <53021DBE.5030006@btinternet.com> Message-ID: <5302240C.8030105@egenix.com> On 17.02.2014 15:33, Rob Cliffe wrote: > > On 17/02/2014 10:00, M.-A. Lemburg wrote: >> On 16.02.2014 05:04, Chris Angelico wrote: >> The colon in there breaks the basic Python concept of having >> colons end headers which start a new block of statements: >> >> http://docs.python.org/reference/compound_stmts.html >> >> I think it would be better to have the PEP focus on a solution >> that doesn't introduce more lambda like syntax to the language :-) >> >> Please also add examples where the expression notation is >> used as part of a larger expression, e.g. >> >> value = lst[2] except IndexError: 0, 1 >> value = lst[2] except IndexError: 0 if x else -1 >> value = lst[2] except IndexError: 0 if lst[1] except IndexError: False else -1 >> value = lst[2] except IndexError: 0 + lst[0] >> d = {'a': lst[2] except IndexError: 0, >> 'b': lst[1] except IndexError: 0 if lst[0] else 1, >> 'c': lst[0], >> } >> >> > Just wanted to point out the ambiguity in some of these, at least to a human reader. This ambiguity is the reason why I pointed at these examples :-) > If I am not mistaken, the proposed precedence for "except" (higher than "lambda", lower than > everything else including "if - else") means that: > > The first one means > value = lst[2] except IndexError: (0,1) > not > value = (lst[2] except IndexError: 0), 1 The second is what Python would do. You can use if-else for testing this: >>> 1 if 2 else 3, 4 (1, 4) > The second means > value = lst[2] except IndexError: (0 if x else -1) > not > value = (lst[2] except IndexError: 0) if x else -1 Yes. > The third means > value = lst[2] except IndexError: (0 if (lst[1] except IndexError: False) else -1) > rather than anything else (this is getting a bit mind-boggling). Yes; but I'm not entirely sure if the Python parser could even handle this case. It also fails on this example: >>> 1 if 2 if 3 else 4 else 5 File "", line 1 1 if 2 if 3 else 4 else 5 ^ SyntaxError: invalid syntax > The fourth means > value = lst[2] except IndexError: (0 + lst[0]) > rather than > value = (lst[2] except IndexError: 0) + lst[0] Yes. > I wonder if the actual meaning coincides in all cases with your intention when you wrote them? > No offence intended, MAL, I'm not trying to slag you off, just pointing out some of the issues that > arise. :-) Looking at such issues was the intention of posting the list :-) > And wondering if there is a case for "except" having a higher precedence than "if - else". I don't > have an opinion yet, I'm just thinking aloud. > > The last one is I believe unambiguous except for the value associated with the key 'b', but to my > mind is another reason for not allowing commas in the proposed "except expression" syntax - > confusion with commas as separators between dictionary literals (sorry I keep plugging this!). > There is already ISTM a clash between comma as separator between dictionary items and between tuple > elements: > >>>> { 'a' : (1,3), 'b' : 2 } > { 'a': (1, 3), 'b': 2 } >>>> { 'a' : 1,3, 'b' : 2 } # Python 2.7.3 > SyntaxError: invalid syntax > > so I think it would be a mistake to overload the comma further. Commas are indeed too overloaded already. Guess what these evaluate to :-) x = lambda y: y, 2 x = 2, x = 1, 2, x = {1:2,} -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 17 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From rosuav at gmail.com Mon Feb 17 16:06:48 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 18 Feb 2014 02:06:48 +1100 Subject: [Python-ideas] except expression In-Reply-To: <5302240C.8030105@egenix.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <5301DDC6.6020505@egenix.com> <53021DBE.5030006@btinternet.com> <5302240C.8030105@egenix.com> Message-ID: On Tue, Feb 18, 2014 at 2:00 AM, M.-A. Lemburg wrote: > Guess what these evaluate to :-) > > x = lambda y: y, 2 > x = 2, > x = 1, 2, > x = {1:2,} The first one is the only one that might be at all in question. The others are straight-forward: a one-element tuple, a two-element tuple, and a one-element dict. The last two are simply making use of the fact that trailing commas are permitted, which is *extremely* handy in avoiding code churn. ChrisA From rosuav at gmail.com Mon Feb 17 16:08:53 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 18 Feb 2014 02:08:53 +1100 Subject: [Python-ideas] except expression In-Reply-To: <5302240C.8030105@egenix.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <5301DDC6.6020505@egenix.com> <53021DBE.5030006@btinternet.com> <5302240C.8030105@egenix.com> Message-ID: On Tue, Feb 18, 2014 at 2:00 AM, M.-A. Lemburg wrote: > x = 1, 2, > x = {1:2,} Actually, a more confusing pair than those is this: x = {1,2,} x = {1:2,} In terms of the "grit on the screen syntax" problem, this one is at some risk. My currently-preferred proposal for exceptions makes good use of the 'except' keyword, so it should be fairly clear what's going on. ChrisA From ryan at ryanhiebert.com Mon Feb 17 16:24:46 2014 From: ryan at ryanhiebert.com (Ryan Hiebert) Date: Mon, 17 Feb 2014 09:24:46 -0600 Subject: [Python-ideas] docs.python.org: Short URLs In-Reply-To: <20140217145500.GA28299@phdru.name> References: <20140217131244.GA7967@untibox.unti> <20140217144225.GA28593@untibox.unti> <321523c2-268f-43fe-bdbd-0dc203b85575@email.android.com> <20140217145500.GA28299@phdru.name> Message-ID: I'm not sure that just these URLs are going to be any more helpful than what we currently have: ``http://docs.python.org/3.2/library/functions.html#str`` Are we looking at adding separate pages for each function, to give more in-depth details to function, or is it really just a redirect? Ryan On Mon, Feb 17, 2014 at 8:55 AM, Oleg Broytman wrote: > On Mon, Feb 17, 2014 at 03:51:58PM +0100, Markus Unterwaditzer < > markus at unterwaditzer.net> wrote: > > >>> http://docs.python.org/[ver]/kw/str > > > > > >Either way. "kw" was the best thing I could come up with at short > > >notice. My point is to namespace it; what the name actually is can > > >always be bikeshedded :) > > Painting it 'term'. > > Oleg. > -- > Oleg Broytman http://phdru.name/ phd at phdru.name > Programmers don't die, they just GOSUB without RETURN. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From markus at unterwaditzer.net Mon Feb 17 16:50:50 2014 From: markus at unterwaditzer.net (Markus Unterwaditzer) Date: Mon, 17 Feb 2014 16:50:50 +0100 Subject: [Python-ideas] docs.python.org: Short URLs Message-ID: And again i forgot to CC the list, my apologies. -------- Original Message -------- Subject: Re: [Python-ideas] docs.python.org: Short URLs Date: 2014-02-17 16:49 From: Markus Unterwaditzer To: Ryan Hiebert On 2014-02-17 16:24, Ryan Hiebert wrote: > I'm not sure that just these URLs are going to be any more helpful > than what we currently have: > ``http://docs.python.org/3.2/library/functions.html#str`` [5] > Are we looking at adding separate pages for each function, to give > more in-depth details to function, or is it really just a redirect? > Ryan A redirect. Content-wise i don't propose any changes. From random832 at fastmail.us Mon Feb 17 17:17:16 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Mon, 17 Feb 2014 11:17:16 -0500 Subject: [Python-ideas] docs.python.org: Short URLs In-Reply-To: References: <20140217131244.GA7967@untibox.unti> <20140217144225.GA28593@untibox.unti> <321523c2-268f-43fe-bdbd-0dc203b85575@email.android.com> <20140217145500.GA28299@phdru.name> Message-ID: <1392653836.2709.84406433.15FD1DB0@webmail.messagingengine.com> On Mon, Feb 17, 2014, at 10:24, Ryan Hiebert wrote: > I'm not sure that just these URLs are going to be any more helpful than > what we currently have: > > ``http://docs.python.org/3.2/library/functions.html#str`` > > Are we looking at adding separate pages for each function, to give more > in-depth details to function, or is it really just a redirect? [bikeshedding intensifies] I, personally, if given a URL consisting of only "str", would expect it to lead to the documentation of the str _class_ (currently mostly at http://docs.python.org/3.2/library/stdtypes.html#string-methods), more or less From ryan at ryanhiebert.com Mon Feb 17 17:40:04 2014 From: ryan at ryanhiebert.com (Ryan Hiebert) Date: Mon, 17 Feb 2014 10:40:04 -0600 Subject: [Python-ideas] docs.python.org: Short URLs In-Reply-To: <1392653836.2709.84406433.15FD1DB0@webmail.messagingengine.com> References: <20140217131244.GA7967@untibox.unti> <20140217144225.GA28593@untibox.unti> <321523c2-268f-43fe-bdbd-0dc203b85575@email.android.com> <20140217145500.GA28299@phdru.name> <1392653836.2709.84406433.15FD1DB0@webmail.messagingengine.com> Message-ID: On Mon, Feb 17, 2014 at 10:17 AM, wrote: > > On Mon, Feb 17, 2014, at 10:24, Ryan Hiebert wrote: > > I'm not sure that just these URLs are going to be any more helpful than > > what we currently have: > > > > ``http://docs.python.org/3.2/library/functions.html#str`` > > > I, personally, if given a URL consisting of only "str", would expect it > to lead to the documentation of the str _class_ (currently mostly at > http://docs.python.org/3.2/library/stdtypes.html#string-methods), more > or less Yeah, that would be a better place to redirect for 'str'. I was just pointing that I'm not sure what these URLs really gain us, if they aren't meant to be used as the canonical URLs, since the #identified URLs are perfectly useful as permalinks. Does having these redirects make it easier to find these things using a search engine? If not, we gain some neat URLs that are still only useful to those of us that actually already know the documentation anyway. -------------- next part -------------- An HTML attachment was scrubbed... URL: From random832 at fastmail.us Mon Feb 17 18:25:03 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Mon, 17 Feb 2014 12:25:03 -0500 Subject: [Python-ideas] docs.python.org: Short URLs In-Reply-To: References: <20140217131244.GA7967@untibox.unti> <20140217144225.GA28593@untibox.unti> <321523c2-268f-43fe-bdbd-0dc203b85575@email.android.com> <20140217145500.GA28299@phdru.name> <1392653836.2709.84406433.15FD1DB0@webmail.messagingengine.com> Message-ID: <1392657903.23260.84442985.1976EDE5@webmail.messagingengine.com> On Mon, Feb 17, 2014, at 11:40, Ryan Hiebert wrote: > Yeah, that would be a better place to redirect for 'str'. > > I was just pointing that I'm not sure what these URLs really gain us, > if they aren't meant to be used as the canonical URLs, since the > #identified > URLs are perfectly useful as permalinks. Does having these redirects make > it easier to find these things using a search engine? If not, we gain > some > neat URLs that are still only useful to those of us that actually already > know > the documentation anyway. I don't know why they should be redirects, no. I think what's being proposed in the article _was_ a separate page for every class and every function. PHP does that. .NET does that. Why shouldn't Python? Is it possible that the Python documentation suffers from excessive conciseness _because_ everything's on one page? From ryan at ryanhiebert.com Mon Feb 17 18:32:16 2014 From: ryan at ryanhiebert.com (Ryan Hiebert) Date: Mon, 17 Feb 2014 11:32:16 -0600 Subject: [Python-ideas] docs.python.org: Short URLs In-Reply-To: <1392657903.23260.84442985.1976EDE5@webmail.messagingengine.com> References: <20140217131244.GA7967@untibox.unti> <20140217144225.GA28593@untibox.unti> <321523c2-268f-43fe-bdbd-0dc203b85575@email.android.com> <20140217145500.GA28299@phdru.name> <1392653836.2709.84406433.15FD1DB0@webmail.messagingengine.com> <1392657903.23260.84442985.1976EDE5@webmail.messagingengine.com> Message-ID: On Mon, Feb 17, 2014 at 11:25 AM, wrote: > > > I don't know why they should be redirects, no. I think what's being > proposed in the article _was_ a separate page for every class and every > function. PHP does that. .NET does that. Why shouldn't Python? Is it > possible that the Python documentation suffers from excessive > conciseness _because_ everything's on one page? > > I don't know that it's necessarily because everything is on one page, but I could imagine that it's a contributing factor to why things are concise, and I could see that splitting things up into more pages would encourage longer prose and more examples. Longer isn't always better, but sometimes saying things multiple ways can help the message get across. -------------- next part -------------- An HTML attachment was scrubbed... URL: From phd at phdru.name Mon Feb 17 18:36:47 2014 From: phd at phdru.name (Oleg Broytman) Date: Mon, 17 Feb 2014 18:36:47 +0100 Subject: [Python-ideas] docs.python.org: Short URLs In-Reply-To: <1392657903.23260.84442985.1976EDE5@webmail.messagingengine.com> References: <20140217131244.GA7967@untibox.unti> <20140217144225.GA28593@untibox.unti> <321523c2-268f-43fe-bdbd-0dc203b85575@email.android.com> <20140217145500.GA28299@phdru.name> <1392653836.2709.84406433.15FD1DB0@webmail.messagingengine.com> <1392657903.23260.84442985.1976EDE5@webmail.messagingengine.com> Message-ID: <20140217173647.GA30745@phdru.name> On Mon, Feb 17, 2014 at 12:25:03PM -0500, random832 at fastmail.us wrote: > I don't know why they should be redirects, no. I think what's being > proposed in the article _was_ a separate page for every class and every > function. PHP does that. .NET does that. Why shouldn't Python? Because someone has to write all those pages. Do you want to lead the effort? Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From abarnert at yahoo.com Mon Feb 17 18:51:56 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Mon, 17 Feb 2014 09:51:56 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <5301DDC6.6020505@egenix.com> <53021DBE.5030006@btinternet.com> <5302240C.8030105@egenix.com> Message-ID: <425ECECE-6A83-4CBA-84C5-CA278138855E@yahoo.com> On Feb 17, 2014, at 7:08, Chris Angelico wrote: > On Tue, Feb 18, 2014 at 2:00 AM, M.-A. Lemburg wrote: >> x = 1, 2, >> x = {1:2,} > > Actually, a more confusing pair than those is this: > > x = {1,2,} > x = {1:2,} > > In terms of the "grit on the screen syntax" problem, this one is at > some risk. My currently-preferred proposal for exceptions makes good > use of the 'except' keyword, so it should be fairly clear what's going > on. That's just because set displays are somewhat degenerate. With 0 elements they are impossible to distinguish from dict displays are therefore illegal; with 1 or 2 elements they can be hard for humans over 30 to distinguish from dict displays (especially without PEP 8-style proper spacing, as in your examples) and are therefore a bit risky; it's just a lesser version of the same problem. From abarnert at yahoo.com Mon Feb 17 19:01:43 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Mon, 17 Feb 2014 10:01:43 -0800 Subject: [Python-ideas] docs.python.org: Short URLs In-Reply-To: References: <20140217131244.GA7967@untibox.unti> <20140217144225.GA28593@untibox.unti> <321523c2-268f-43fe-bdbd-0dc203b85575@email.android.com> <20140217145500.GA28299@phdru.name> <1392653836.2709.84406433.15FD1DB0@webmail.messagingengine.com> <1392657903.23260.84442985.1976EDE5@webmail.messagingengine.com> Message-ID: <836346CA-A62A-4A7D-8880-9F3C0E54FD39@yahoo.com> On Feb 17, 2014, at 9:32, Ryan Hiebert wrote: > On Mon, Feb 17, 2014 at 11:25 AM, wrote: >> >> >> I don't know why they should be redirects, no. I think what's being >> proposed in the article _was_ a separate page for every class and every >> function. PHP does that. .NET does that. Why shouldn't Python? Is it >> possible that the Python documentation suffers from excessive >> conciseness _because_ everything's on one page? > > I don't know that it's necessarily because everything is on one page, but I could imagine that it's a contributing factor to why things are concise, and I could see that splitting things up into more pages would encourage longer prose and more examples. Longer isn't always better, but sometimes saying things multiple ways can help the message get across. I think the conciseness and single-page-ness is often a virtue. It leads to people discovering other methods of the class whose method they're looking up. For example, last week there was a question on StackOverflow from a novice who discovered str.partition because he was trying to figure out how to make str.split always return 2 elements and saw it on the same page. (He wanted to know how to make partition split N times instead of 1, and got a fancy genexpr as an answer that he could never have written himself and probably didn't understand, but that's a deficiency of SO, not of the docs.) It's possible to design documentation that's the best of both worlds, using a hierarchical nav bar on the left, or even having the str.split page actually be a collapsible version of the str page with various sections pre-collapsed. And if someone is willing to do all the work for _that_, I have no issue. It's only if we're comparing current Python docs vs. PHP style "every function, even the most trivial, is entirely independent" docs. -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Mon Feb 17 19:35:13 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Mon, 17 Feb 2014 10:35:13 -0800 Subject: [Python-ideas] docs.python.org: Short URLs In-Reply-To: <836346CA-A62A-4A7D-8880-9F3C0E54FD39@yahoo.com> References: <20140217131244.GA7967@untibox.unti> <20140217144225.GA28593@untibox.unti> <321523c2-268f-43fe-bdbd-0dc203b85575@email.android.com> <20140217145500.GA28299@phdru.name> <1392653836.2709.84406433.15FD1DB0@webmail.messagingengine.com> <1392657903.23260.84442985.1976EDE5@webmail.messagingengine.com> <836346CA-A62A-4A7D-8880-9F3C0E54FD39@yahoo.com> Message-ID: <65A6AD34-BB86-41F8-9E8B-EE12C783D09D@yahoo.com> On Feb 17, 2014, at 10:01, Andrew Barnert wrote: > On Feb 17, 2014, at 9:32, Ryan Hiebert wrote: > >> On Mon, Feb 17, 2014 at 11:25 AM, wrote: >>> >>> >>> I don't know why they should be redirects, no. I think what's being >>> proposed in the article _was_ a separate page for every class and every >>> function. PHP does that. .NET does that. Why shouldn't Python? Is it >>> possible that the Python documentation suffers from excessive >>> conciseness _because_ everything's on one page? >> >> I don't know that it's necessarily because everything is on one page, but I could imagine that it's a contributing factor to why things are concise, and I could see that splitting things up into more pages would encourage longer prose and more examples. Longer isn't always better, but sometimes saying things multiple ways can help the message get across. > > I think the conciseness and single-page-ness is often a virtue. It leads to people discovering other methods of the class whose method they're looking up. > > For example, last week there was a question on StackOverflow from a novice who discovered str.partition because he was trying to figure out how to make str.split always return 2 elements and saw it on the same page. (He wanted to know how to make partition split N times instead of 1, and got a fancy genexpr as an answer that he could never have written himself and probably didn't understand, but that's a deficiency of SO, not of the docs.) > > It's possible to design documentation that's the best of both worlds, using a hierarchical nav bar on the left, or even having the str.split page actually be a collapsible version of the str page with various sections pre-collapsed. And if someone is willing to do all the work for _that_, I have no issue. It's only if we're comparing current Python docs vs. PHP style "every function, even the most trivial, is entirely independent" docs. That being said, there might be a good middle ground--a page that had just the docs on str, rather than on all builtin types, might be handy. But other than that one page on all of the builtin types, I can't think of any others that are overly large... -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Mon Feb 17 19:46:57 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Mon, 17 Feb 2014 10:46:57 -0800 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: <20140214204820.GE4281@ando> Message-ID: On Feb 16, 2014, at 18:06, "Joao S. O. Bueno" wrote: > I am stepping late on the thread - > but the wish for having a short cut for: > > if expensive_computation_0(): > x = expensive_computation_0() > # Do something with x... > > And havign dealt before with languages where one explicitly deal with > the operanbd stack (like postscript), lead me to deploy this very simple > module I called "stackfull" (in a wordplay with "stackless") - > > The functions in there simply annotate a list in the locals() dict of > the current > running frame (which cannot be accessed as a variable), and work as > stack operators > on that list - > so the situation above become: > > if push(expensive_computation_0()): > x = pop() > # Do something with x... > > (all functions return the original operand, besides side effects on the "stack") > It was devised most as a toy, but my wish was to fill the gap Python > has for this pattern - > so, if anyone is interested in invest in this idea so that we could > have a nice, real useful thing > to be used in the near future, be my guest. > > https://pypi.python.org/pypi/stackfull > https://github.com/jsbueno/stackfull Clever. What happens if expensive_computation() raises? Does the push get ignored? Capture the exception, push it, and re-raise, so the next pop will raise the same exception? Is there any reason this has to be local? If it were, say, a thread-local global, it could be used to wedge additional arguments through functions that weren't expecting them. (Together with a "mark" function of some kind you could even use this to experiment with alternate calling conventions, etc.) I still don't see how this fills a gap that needs to be filled; how is the first version any more readable, writable, concise, whatever than the second? if push(expensive()): x = pop() dostuff(x) x = expensive() if x: dostuff(x) But it's a clever idea even if it doesn't help there. > On 14 February 2014 23:43, Chris Angelico wrote: >> On Sat, Feb 15, 2014 at 12:40 PM, Nick Coghlan wrote: >>> If the handling is identical, there's no need for an if/elif chain at >>> all, otherwise the revised if/elif chain is based on the attributes of >>> the match object. >> >> If the handling's identical, short-circuiting 'or' will do the job. >> Assume it's not. >> >>>> Removing one is just deleting its elif and corresponding block of >>>> code. They are peers. The fact that Python doesn't currently have >>>> syntax that allows this *in an elif chain* doesn't change this; if you >>>> were to write it as pseudo-code, you would write them as peers. That's >>>> why the function-with-return or loop-with-break still looks better, >>>> because it structures the code the way that logically makes sense - >>>> flat, not nested. >>> >>> That doesn't mean embedded assignments are the answer though - they're >>> a sledgehammer solution that is tempting as a workaround for larger >>> structural issues. >> >> I agree that embedded assignment isn't necessarily the answer. I just >> think that a solution that has them all at the same indentation level >> makes more sense than one that nests them inside each other, for >> exactly the same reason that we have 'elif' in the first place. >> >> ChrisA >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From steve at pearwood.info Mon Feb 17 21:50:54 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 18 Feb 2014 07:50:54 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> <20140217044602.GF4519@ando> <20140217084005.GK4519@ando> Message-ID: <20140217205054.GM4519@ando> On Mon, Feb 17, 2014 at 09:56:21AM +0000, Paul Moore wrote: > On 17 February 2014 08:40, Steven D'Aprano wrote: > > I sympathise, but you already have to deal with dicts, slices, and > > lambda, and with the exception (pun intended) of lambda, they're likely > > to be much more common than except-expression. > > I should have expanded :-) (See? I also have to deal with smilies!) > > With dicts slices and lambdas, the colon is essentially not so > important for the *human* reader, although it is needed for the > parser. If dicts were simply {key, value, key, value, ...} they would > still make sense to the human reader, just by pairing things up Which can be quite tedious and error-prone, especially if you follow the PEP 8 recommendation to *not* line up columns in your data. Quickly now, is None a key or a value? d = {258, 261, 71, 201, 3, 204, 907, 210, 4, 513, 23, 8, 219, None, 31, 225, 528, 39, 234, 231, 1237, 47, 640, 243, 246, 55, 149, 255, 252, 13} I suppose Python might have used spaces to seperate slice fields: mylist[start end stride] although that does make it awkward to do slices using defaults: mylist[ -1] # default start=0 mylist[1 ] # default end=len of the sequence > and by > the fact that anyone writing something complicate would lay it out > readably. Oh my, have you lived a sheltered life! *wink* [...] > My point with the example I quoted (reproduced here for reference) > > >> > some_io() except (FileNotFoundError: (1, 2, 3), > >> > (ValueError, TypeError): 'spam') > > is precisely that - the subtle change in meaning based on a relatively > hard to spot colon is *not* something that matters when debating the > relative merits of syntax proposals, because no-one should be writing > code like this in the first place. That's not the syntax I'm pushing for. The form I want to see requires the `except` keyword to be repeated: (some_io() except FileNotFoundError: (1, 2, 3), except ValueError, TypeError: 'spam') Brackets around the ValueError, TypeError pair might be allowed: (some_io() except FileNotFoundError: (1, 2, 3), except (ValueError, TypeError): 'spam') > 1. Let's worry about making the basic case as readable as possible, > rather than the complex cases Agreed. But I think the basic case is very readable, and the complicated case less so only because it is complicated, not because of the syntax. > 2. Punctuation for disambiguation can be a problem - I'd prefer it if > the keywords stood by themselves in this case Yes but no but yes but no but yes but ... There are very few cases in Python of using space as mandatory separator. All the cases I can think apply only to keywords: class C ... def func ... import this, that, other del a, b, c but not import this that other del a b c The most relevant case to this is, I think, lambda, where we write: lambda parameter_list : expression rather than (say): lambda (parameter_list) expression with mandatory brackets around the parameter list to separate it from the expression. Using the existing syntax as a guideline, I get this: expression except exception_list : default_expression rather than this: expression except (exception_list) default_expression > 3. Needing extra parentheses to disambiguate (or even requiring them - > consider yield expressions or generator expressions) is not a bad > thing, nor is needing to make complex cases multi-line - human > readability is more important than using the minimum syntax that the > grammar requires. I agree with this. -- Steven From markus at unterwaditzer.net Mon Feb 17 21:54:09 2014 From: markus at unterwaditzer.net (Markus Unterwaditzer) Date: Mon, 17 Feb 2014 21:54:09 +0100 Subject: [Python-ideas] docs.python.org: Short URLs In-Reply-To: <1392657903.23260.84442985.1976EDE5@webmail.messagingengine.com> References: <20140217131244.GA7967@untibox.unti> <20140217144225.GA28593@untibox.unti> <321523c2-268f-43fe-bdbd-0dc203b85575@email.android.com> <20140217145500.GA28299@phdru.name> <1392653836.2709.84406433.15FD1DB0@webmail.messagingengine.com> <1392657903.23260.84442985.1976EDE5@webmail.messagingengine.com> Message-ID: <20140217205409.GA8891@untibox.unti> On Mon, Feb 17, 2014 at 12:25:03PM -0500, random832 at fastmail.us wrote: > On Mon, Feb 17, 2014, at 11:40, Ryan Hiebert wrote: > > Yeah, that would be a better place to redirect for 'str'. > > > > I was just pointing that I'm not sure what these URLs really gain us, > > if they aren't meant to be used as the canonical URLs, since the > > #identified > > URLs are perfectly useful as permalinks. Does having these redirects make > > it easier to find these things using a search engine? If not, we gain > > some > > neat URLs that are still only useful to those of us that actually already > > know > > the documentation anyway. > > I don't know why they should be redirects, no. I think what's being > proposed in the article _was_ a separate page for every class and every > function. PHP does that. .NET does that. Why shouldn't Python? Is it > possible that the Python documentation suffers from excessive > conciseness _because_ everything's on one page? > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ I agree, it would be a better idea to have short and meaningful URLs, but as far as i know, this would require major modifications to Sphinx (unless you're actually creating an RST page for every object) and would require significantly more effort than the change i am suggesting. In my original mail i didn't even consider the SEO-aspect of short URLs, i just remembered from my PHP days that typing php.net/function_name was very helpful once i've discovered i can do that. -- Markus From steve at pearwood.info Mon Feb 17 22:07:17 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 18 Feb 2014 08:07:17 +1100 Subject: [Python-ideas] Commas [was Re: except expression] In-Reply-To: <5302240C.8030105@egenix.com> References: <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <5301DDC6.6020505@egenix.com> <53021DBE.5030006@btinternet.com> <5302240C.8030105@egenix.com> Message-ID: <20140217210717.GN4519@ando> On Mon, Feb 17, 2014 at 04:00:28PM +0100, M.-A. Lemburg wrote: > Commas are indeed too overloaded already. > > Guess what these evaluate to :-) > > x = lambda y: y, 2 > x = 2, > x = 1, 2, > x = {1:2,} With the exception of the first one, where I'm not only 90% confident, I don't think there is any difficulty with any of those. The first makes x a tuple containing (function, 2). The second is a tuple of one element, (2,). (Remember that, with the exception of the empty tuple, tuples are created by commas, not round brackets.) The third is a tuple of two elements, (1, 2). The last is a dict with one key:item pair {1:2}. The only one I needed to check at the interactive interpreter was the one with the lambda. -- Steven From alexander.belopolsky at gmail.com Mon Feb 17 22:36:32 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 17 Feb 2014 16:36:32 -0500 Subject: [Python-ideas] Commas [was Re: except expression] In-Reply-To: <20140217210717.GN4519@ando> References: <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <5301DDC6.6020505@egenix.com> <53021DBE.5030006@btinternet.com> <5302240C.8030105@egenix.com> <20140217210717.GN4519@ando> Message-ID: On Mon, Feb 17, 2014 at 4:07 PM, Steven D'Aprano wrote: > > Guess what these evaluate to :-) > > > > x = lambda y: y, 2 > > x = 2, > > x = 1, 2, > > x = {1:2,} > > With the exception of the first one, where I'm not only 90% confident, I > don't think there is any difficulty with any of those. > Same here. > > The first makes x a tuple containing (function, 2). > Funny: less than an hour ago, I paused when writing a lambda to return a tuple. I conservatively put parentheses around the tuple and did not bother to check if they are required. Now I know that they are. -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Mon Feb 17 22:40:33 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 18 Feb 2014 10:40:33 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> <53013CD3.8030809@canterbury.ac.nz> Message-ID: <530281D1.1070808@canterbury.ac.nz> Chris Angelico wrote: > Do you mean that there may not be a comma, or that commas must be > surrounded by parens? Commas would need to be surroounded by parens. > Imagine if the exception > list isn't literals, but comes from somewhere else (which is perfectly > legal). How do you eyeball it and see that this is now a new except > clause? I'm not a big fan of this idea either. It seems to me that wanting more than one except clause is going to be *extremely* rare, so it's not worth going out of our way to make it super-concise. -- Greg From markus at unterwaditzer.net Mon Feb 17 22:51:22 2014 From: markus at unterwaditzer.net (Markus Unterwaditzer) Date: Mon, 17 Feb 2014 22:51:22 +0100 Subject: [Python-ideas] docs.python.org: Short URLs In-Reply-To: <1392653836.2709.84406433.15FD1DB0@webmail.messagingengine.com> References: <20140217131244.GA7967@untibox.unti> <20140217144225.GA28593@untibox.unti> <321523c2-268f-43fe-bdbd-0dc203b85575@email.android.com> <20140217145500.GA28299@phdru.name> <1392653836.2709.84406433.15FD1DB0@webmail.messagingengine.com> Message-ID: <20140217215122.GA6577@untibox.unti> On Mon, Feb 17, 2014 at 11:17:16AM -0500, random832 at fastmail.us wrote: > > > On Mon, Feb 17, 2014, at 10:24, Ryan Hiebert wrote: > > I'm not sure that just these URLs are going to be any more helpful than > > what we currently have: > > > > ``http://docs.python.org/3.2/library/functions.html#str`` > > > > Are we looking at adding separate pages for each function, to give more > > in-depth details to function, or is it really just a redirect? > > [bikeshedding intensifies] > > I, personally, if given a URL consisting of only "str", would expect it > to lead to the documentation of the str _class_ (currently mostly at > http://docs.python.org/3.2/library/stdtypes.html#string-methods), more > or less > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ Sure, i agree. The script i posted meshed all dictionaries (one for functions, objects, classes, modules each) together in arbitrary order, i know this can be improved. -- Markus From greg.ewing at canterbury.ac.nz Mon Feb 17 23:26:41 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 18 Feb 2014 11:26:41 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <53017838.9050706@stoneleaf.us> Message-ID: <53028CA1.5080502@canterbury.ac.nz> Chris Angelico wrote: > On Mon, Feb 17, 2014 at 1:47 PM, Ethan Furman wrote: > >>def have_a_mint(some, args, here): >> # flesh this out later >> pass >> >>Does anybody really think that that function will not return None? > > Of course it'll return None, but that's nothing to do with the 'pass'. > The keyword 'pass' doesn't generate any return result at all. Function return values are a red herring here. The point is to allow the programmer to directly express the intent of the code without having to introduce spurious values that will be ignored. There's no logical difference between not generating a result at all, and generating a result of None and then throwing it away. The same thing applies here: menu.remove(mint) except ValueError: pass This says exactly what the programmer means: "Remove mint from the menu if it's there, otherwise do nothing." An alternative would be to allow the exceptional value part to be omitted altogether: menu.remove(mint) except ValueError but that looks incomplete, making you wonder if the programmer forgot something. Either way, this would be allowed *only* for top level expressions whose value is ignored. In any context where the value of the expression is used for something, the exceptional value would have to be spelled out explicitly. -- Greg From greg.ewing at canterbury.ac.nz Mon Feb 17 23:38:01 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 18 Feb 2014 11:38:01 +1300 Subject: [Python-ideas] except expression In-Reply-To: <20140217044602.GF4519@ando> References: <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> <20140217044602.GF4519@ando> Message-ID: <53028F49.1060405@canterbury.ac.nz> Steven D'Aprano wrote: > I'm now strongly leaning towards following the lead of generator > expressions, and requiring parens around an except-expression. > > I think that this will ensure that there is no visual ambiguity when you > chain except-expressions. I think we should keep in mind that we're talking about what will be a somewhat rarely used construct, and that the overwhelming majority of those rare uses will be the simplest possible ones, with just a single 'except' clause and a very simple expression for the exceptional case, e.g. things[i] except IndexError: None We should concentrate on making that case as clear and uncluttered as possible, and not worry overmuch about how things look in more complicated cases that will hardly ever arise in practice. -- Greg From greg.ewing at canterbury.ac.nz Mon Feb 17 23:45:04 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 18 Feb 2014 11:45:04 +1300 Subject: [Python-ideas] except expression In-Reply-To: <20140217052912.GJ4519@ando> References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> Message-ID: <530290F0.7030506@canterbury.ac.nz> > On Mon, Feb 17, 2014 at 08:52:41AM +1100, Chris Angelico wrote: > >>Should the default clause have an inner scope in which the >>name exists, shadowing anything of the same name elsewhere? Should it behave >>the same way the statement try/except does, and unbind the name? We could sidestep the whole problem by not allowing an 'as' clause at all. I don't think it would be unreasonable to require the use of a try statement if you want to do anything that fancy. -- Greg From greg.ewing at canterbury.ac.nz Mon Feb 17 23:50:44 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 18 Feb 2014 11:50:44 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> Message-ID: <53029244.1090205@canterbury.ac.nz> Chris Angelico wrote: > The inner scope does seem to demand a function call, though. A function call isn't *required* to achieve the effect of an inner scope, as far as I know. It's just that implementing an inner scope for list comprehensions without it would have required extensive changes to the way the bytecode compiler currently works, and using a function was easier. -- Greg From ncoghlan at gmail.com Mon Feb 17 23:52:23 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 18 Feb 2014 08:52:23 +1000 Subject: [Python-ideas] docs.python.org: Short URLs In-Reply-To: <65A6AD34-BB86-41F8-9E8B-EE12C783D09D@yahoo.com> References: <20140217131244.GA7967@untibox.unti> <20140217144225.GA28593@untibox.unti> <321523c2-268f-43fe-bdbd-0dc203b85575@email.android.com> <20140217145500.GA28299@phdru.name> <1392653836.2709.84406433.15FD1DB0@webmail.messagingengine.com> <1392657903.23260.84442985.1976EDE5@webmail.messagingengine.com> <836346CA-A62A-4A7D-8880-9F3C0E54FD39@yahoo.com> <65A6AD34-BB86-41F8-9E8B-EE12C783D09D@yahoo.com> Message-ID: On 18 Feb 2014 04:39, "Andrew Barnert" wrote: > > > That being said, there might be a good middle ground--a page that had just the docs on str, rather than on all builtin types, might be handy. But other than that one page on all of the builtin types, I can't think of any others that are overly large... That's the main one, although some of the module docs jam a tutorial, reference and cookbook into one page, which can be a bit overwhelming. The trick lies in figuring out how to split it up without breaking any existing deep links. I actually wonder if there might be a benefit in having Sphinx generate a full "index" URL subtree, where each of those pages links back to the relevant anchors in the existing docs. That would be potentially useful in its own right (as a short hand for "here's all the places that provide info about str"), as well as potentially helping with SEO. Cheers, Nick. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Mon Feb 17 23:56:24 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 18 Feb 2014 11:56:24 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> Message-ID: <53029398.5030409@canterbury.ac.nz> Chris Angelico wrote: > Maybe this is a good opportunity to introduce the concept of > "sub-scopes" for specific constructs. Comprehensions and 'as' clauses > (I doubt anyone will object to the statement try/except being brought > in line with this) could then use a sub-scope inside the existing > function; it'd affect only the LOAD_FAST opcode, I think - assignment > could delete it from the sub-scope and put it into the main scope. It *should* be possible to handle this entirely at compile time. Conceptually, all you need to do is rename the names in the subscope so that they don't clash with anything in the outer scope, and then generate bytecode as usual. I haven't studied the compiler closely enough to tell how difficult this would be to achieve, though. Whoever implemented inner scopes for list comprehensions reportedly found it harder than he liked at the time. -- Greg From jsbueno at python.org.br Mon Feb 17 23:59:22 2014 From: jsbueno at python.org.br (Joao S. O. Bueno) Date: Mon, 17 Feb 2014 19:59:22 -0300 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: <20140214204820.GE4281@ando> Message-ID: On 17 February 2014 15:46, Andrew Barnert wrote: > On Feb 16, 2014, at 18:06, "Joao S. O. Bueno" wrote: > >> I am stepping late on the thread - >> but the wish for having a short cut for: >> >> if expensive_computation_0(): >> x = expensive_computation_0() >> # Do something with x... >> >> And havign dealt before with languages where one explicitly deal with >> the operanbd stack (like postscript), lead me to deploy this very simple >> module I called "stackfull" (in a wordplay with "stackless") - >> >> The functions in there simply annotate a list in the locals() dict of >> the current >> running frame (which cannot be accessed as a variable), and work as >> stack operators >> on that list - >> so the situation above become: >> >> if push(expensive_computation_0()): >> x = pop() >> # Do something with x... >> >> (all functions return the original operand, besides side effects on the "stack") >> It was devised most as a toy, but my wish was to fill the gap Python >> has for this pattern - >> so, if anyone is interested in invest in this idea so that we could >> have a nice, real useful thing >> to be used in the near future, be my guest. >> >> https://pypi.python.org/pypi/stackfull >> https://github.com/jsbueno/stackfull > > Clever. > > What happens if expensive_computation() raises? Does the push get ignored? Capture the exception, push it, and re-raise, so the next pop will raise the same exception? As it is today, the operators there are just normal functions (I had some distant plans of fiddling with Pymacro one of these days to see if they could be something different). So, if "expensive_computation" raises, push will never be called. > > Is there any reason this has to be local? If it were, say, a thread-local global, it could be used to wedge additional arguments through functions that weren't expecting them. (Together with a "mark" function of some kind you could even use this to experiment with alternate calling conventions, etc.) There is no special reason for it to be local, other than 'a simple implementation' that would allow me to have the a value of "expensive_computation()" I could re-use in the same expression. > > I still don't see how this fills a gap that needs to be filled; how is the first version any more readable, writable, concise, whatever than the second? > > if push(expensive()): > x = pop() > dostuff(x) > > x = expensive() > if x: > dostuff(x) When I wrote it, my need was actually for a pattern in the inline "if" operation: x = pop() if complex_comparison(push(expensive_operation())) else None clear() (again, "complex_comparison" need just not to be that comples, just something that has to check the value of "expensive()" for anything other than boolean true/falsiness) Though I recognize the pattern gets in the way of readability, since in this case the pop is placed before the "push" . But if this gets any traction (even in my personal projects), maybe writing: x = None if not complex_comparison(push(expensive_operation())) else pop() might be more recommended. As I said, it is more or less a toy project, although I had used it "for real" in some places - but it does fill a gap I perceive in Python syntax, which is the one discussed in this thread. And I am all open to expand "stackfull" into something else, like, for example, "even use this to experiment with alternate calling conventions, etc" :-) js -><- From greg.ewing at canterbury.ac.nz Tue Feb 18 00:12:38 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 18 Feb 2014 12:12:38 +1300 Subject: [Python-ideas] except expression In-Reply-To: <5301DDC6.6020505@egenix.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <5301DDC6.6020505@egenix.com> Message-ID: <53029766.4060809@canterbury.ac.nz> M.-A. Lemburg wrote: > The colon in there breaks the basic Python concept of having > colons end headers which start a new block of statements: > > http://docs.python.org/reference/compound_stmts.html That section is talking about statements. The construct we're discussing is an expression, so it wouldn't be bound by the same rules. And there are at least 3 existing uses of colons in expressions, so ending headers is not the *only* use of colons. -- Greg From tjreedy at udel.edu Tue Feb 18 00:35:14 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 17 Feb 2014 18:35:14 -0500 Subject: [Python-ideas] docs.python.org: Short URLs In-Reply-To: <20140217131244.GA7967@untibox.unti> References: <20140217131244.GA7967@untibox.unti> Message-ID: On 2/17/2014 8:12 AM, Markus Unterwaditzer wrote: > The author talks about the inaccessibility of Python's documentation via Google > compared to PHP's. One can easily verify that by themselves: Just enter the > name of any builtin into the search engine at docs.python.org, such as str, There is an issue on the tracker about the uselessness of the search box. I and others sugggested that it first see if the search term is in the index and return the indexed pages in the manual before doing whatever it does not. Those pages are nearly always what one wants, not what Google decides. I happen to use the index directly, but many do not. I don't know why that has not been done. If you want, find the issue and add your comments, or, if you can, a patch. -- Terry Jan Reedy From rosuav at gmail.com Tue Feb 18 00:49:32 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 18 Feb 2014 10:49:32 +1100 Subject: [Python-ideas] except expression In-Reply-To: <20140217205054.GM4519@ando> References: <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> <20140217044602.GF4519@ando> <20140217084005.GK4519@ando> <20140217205054.GM4519@ando> Message-ID: On Tue, Feb 18, 2014 at 7:50 AM, Steven D'Aprano wrote: > Brackets around the ValueError, TypeError pair might be allowed: > > (some_io() except FileNotFoundError: (1, 2, 3), > except (ValueError, TypeError): 'spam') I'd say that the parens are definitely allowed, and might be required. (They're currently required in the statement form, but I think that's just to distinguish from the old pre-'as' syntax.) ChrisA From rosuav at gmail.com Tue Feb 18 00:54:50 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 18 Feb 2014 10:54:50 +1100 Subject: [Python-ideas] except expression In-Reply-To: <53028CA1.5080502@canterbury.ac.nz> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <53017838.9050706@stoneleaf.us> <53028CA1.5080502@canterbury.ac.nz> Message-ID: On Tue, Feb 18, 2014 at 9:26 AM, Greg Ewing wrote: > There's no logical difference between not generating > a result at all, and generating a result of None and > then throwing it away. > > The same thing applies here: > > menu.remove(mint) except ValueError: pass > > This says exactly what the programmer means: "Remove > mint from the menu if it's there, otherwise do nothing." In a statement context, it's possible to say "otherwise do nothing". An if without an else does this, as does putting 'pass' in certain places. But in an expression context, there are 'two possibility'. Either some kind of value is returned, or an exception is raised. (At least, I don't think there are any other options. Harry Hoo, not the Spanish Inquisition.) What should happen here: func(menu.remove(mint) except ValueError: pass) If remove() raises ValueError, should func be called? If so, with what argument? (Note that it's because English is happy with "passing" arguments to functions that I was happy with the notation "except ValueError pass None", but that's using pass as a syntactic element, in place of the colon.) ChrisA From ncoghlan at gmail.com Tue Feb 18 00:54:57 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 18 Feb 2014 09:54:57 +1000 Subject: [Python-ideas] except expression In-Reply-To: <53029398.5030409@canterbury.ac.nz> References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <53029398.5030409@canterbury.ac.nz> Message-ID: On 18 Feb 2014 08:57, "Greg Ewing" wrote: > > Chris Angelico wrote: >> >> Maybe this is a good opportunity to introduce the concept of >> "sub-scopes" for specific constructs. Comprehensions and 'as' clauses >> (I doubt anyone will object to the statement try/except being brought >> in line with this) could then use a sub-scope inside the existing >> function; it'd affect only the LOAD_FAST opcode, I think - assignment >> could delete it from the sub-scope and put it into the main scope. > > > It *should* be possible to handle this entirely at > compile time. Conceptually, all you need to do is > rename the names in the subscope so that they > don't clash with anything in the outer scope, and > then generate bytecode as usual. > > I haven't studied the compiler closely enough to > tell how difficult this would be to achieve, though. > Whoever implemented inner scopes for list comprehensions > reportedly found it harder than he liked at the time. That was me, and it's handling nested closures correctly that gets excruciatingly difficult. In theory, the compiler has enough info to figure out the details and generate appropriate inline code, but in practice, reusing the existing closure support made the change so much simpler to implement that was the option I eventually chose. While that simplicity of implementation did come at the cost of making the behaviour at class scope a bit quirky, those quirks already existed for generator expressions, so they even had the virtue of consistency in their favour. Cheers, Nick. > > -- > Greg > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Tue Feb 18 01:01:22 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 18 Feb 2014 11:01:22 +1100 Subject: [Python-ideas] except expression In-Reply-To: <530290F0.7030506@canterbury.ac.nz> References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> Message-ID: On Tue, Feb 18, 2014 at 9:45 AM, Greg Ewing wrote: >> On Mon, Feb 17, 2014 at 08:52:41AM +1100, Chris Angelico wrote: >> >>> Should the default clause have an inner scope in which the >>> name exists, shadowing anything of the same name elsewhere? Should it >>> behave >>> the same way the statement try/except does, and unbind the name? > > > We could sidestep the whole problem by not allowing > an 'as' clause at all. I don't think it would be > unreasonable to require the use of a try statement > if you want to do anything that fancy. I'd accept that if and only if it makes a huge difference to the implementability of the proposal. Conceptually, it's easy to say "inner scope". If it turns out that it's really REALLY hard to do that, then rejecting 'as' could be a fall-back. But I'd really rather keep it. >> The inner scope does seem to demand a function call, though. > > A function call isn't *required* to achieve the effect > of an inner scope, as far as I know. It's just that > implementing an inner scope for list comprehensions > without it would have required extensive changes to > the way the bytecode compiler currently works, and > using a function was easier. Interesting. I'll reword that section a bit, then. It may be that adding another use-case for inner scopes will make them more worth implementing, and then everything (comprehensions, except statements, and except expressions) can use them. It might also be worth using an inner scope for a 'with' statement, too. Every block statement that uses 'as' could introduce a sub-scope. But that is *definitely* outside the, uh, scope of this PEP. ChrisA From rosuav at gmail.com Tue Feb 18 01:06:22 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 18 Feb 2014 11:06:22 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <53029398.5030409@canterbury.ac.nz> Message-ID: On Tue, Feb 18, 2014 at 10:54 AM, Nick Coghlan wrote: >> I haven't studied the compiler closely enough to >> tell how difficult this would be to achieve, though. >> Whoever implemented inner scopes for list comprehensions >> reportedly found it harder than he liked at the time. > > That was me, and it's handling nested closures correctly that gets > excruciatingly difficult. In theory, the compiler has enough info to figure > out the details and generate appropriate inline code, but in practice, > reusing the existing closure support made the change so much simpler to > implement that was the option I eventually chose. Thanks Nick. Useful explanation, and now linked to in the PEP draft. https://github.com/Rosuav/ExceptExpr/raw/master/pep.txt ChrisA From rosuav at gmail.com Tue Feb 18 01:11:09 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 18 Feb 2014 11:11:09 +1100 Subject: [Python-ideas] if expensive_computation() as x: In-Reply-To: References: <20140214204820.GE4281@ando> Message-ID: On Tue, Feb 18, 2014 at 9:59 AM, Joao S. O. Bueno > As it is today, the operators there are just normal functions (I had > some distant plans of fiddling with Pymacro one of these days to see > if they could be something different). > So, if "expensive_computation" raises, push will never be called. Which is also the semantics I would expect based on a purely conceptual reading of the theory. Keep it like that :) > x = pop() if complex_comparison(push(expensive_operation())) else None > clear() > > (again, "complex_comparison" need just not to be that comples, just something > that has to check the value of "expensive()" for anything other than > boolean true/falsiness) How about: x = complex_comparison(push(expensive_operation())) and pop() That'll leave you with some falsy value, rather than specifically None, but if you know expensive_operation will return something nonzero, that could work. ChrisA From ethan at stoneleaf.us Tue Feb 18 01:20:26 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 17 Feb 2014 16:20:26 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <53017838.9050706@stoneleaf.us> <53028CA1.5080502@canterbury.ac.nz> Message-ID: <5302A74A.9060809@stoneleaf.us> On 02/17/2014 03:54 PM, Chris Angelico wrote: > On Tue, Feb 18, 2014 at 9:26 AM, Greg Ewing wrote: >> There's no logical difference between not generating >> a result at all, and generating a result of None and >> then throwing it away. >> >> The same thing applies here: >> >> menu.remove(mint) except ValueError: pass >> >> This says exactly what the programmer means: "Remove >> mint from the menu if it's there, otherwise do nothing." > > In a statement context, it's possible to say "otherwise do nothing". > An if without an else does this, as does putting 'pass' in certain > places. > > But in an expression context, there are 'two possibility'. Either some > kind of value is returned, or an exception is raised. (At least, I > don't think there are any other options. Harry Hoo, not the Spanish > Inquisition.) What should happen here: > > func(menu.remove(mint) except ValueError: pass) > > If remove() raises ValueError, should func be called? If so, with what > argument? (Note that it's because English is happy with "passing" > arguments to functions that I was happy with the notation "except > ValueError pass None", but that's using pass as a syntactic element, > in place of the colon.) Yes, func should be called, and it should be called with no arguments. -- ~Ethan~ From rosuav at gmail.com Tue Feb 18 01:48:25 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 18 Feb 2014 11:48:25 +1100 Subject: [Python-ideas] except expression In-Reply-To: <5302A74A.9060809@stoneleaf.us> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <53017838.9050706@stoneleaf.us> <53028CA1.5080502@canterbury.ac.nz> <5302A74A.9060809@stoneleaf.us> Message-ID: On Tue, Feb 18, 2014 at 11:20 AM, Ethan Furman wrote: > On 02/17/2014 03:54 PM, Chris Angelico wrote: >> But in an expression context, there are 'two possibility'. Either some >> kind of value is returned, or an exception is raised. (At least, I >> don't think there are any other options. Harry Hoo, not the Spanish >> Inquisition.) What should happen here: >> >> func(menu.remove(mint) except ValueError: pass) >> >> If remove() raises ValueError, should func be called? If so, with what >> argument? (Note that it's because English is happy with "passing" >> arguments to functions that I was happy with the notation "except >> ValueError pass None", but that's using pass as a syntactic element, >> in place of the colon.) > > > Yes, func should be called, and it should be called with no arguments. Ooh. That's an interesting one. I'm not sure how that would go in terms of readability, but it is an interesting concept. Very interesting concept. Not sure that I like it, but ... the mind does like the idea ... in a weird and dangerous way. def throw(ex): raise ex some_func(1, 2, (x if x is not None else throw(SyntaxError)) except SyntaxError: pass) So if x is None, it doesn't get passed at all. Useful feature... wordy and clunky spelling... no, I'm now sure that I do not like this. But it's amusing! ChrisA From rob.cliffe at btinternet.com Tue Feb 18 01:50:49 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Tue, 18 Feb 2014 00:50:49 +0000 Subject: [Python-ideas] except expression In-Reply-To: <53028CA1.5080502@canterbury.ac.nz> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <53017838.9050706@stoneleaf.us> <53028CA1.5080502@canterbury.ac.nz> Message-ID: <5302AE69.4080701@btinternet.com> On 17/02/2014 22:26, Greg Ewing wrote: > Chris Angelico wrote: >> On Mon, Feb 17, 2014 at 1:47 PM, Ethan Furman >> wrote: >> >>> def have_a_mint(some, args, here): >>> # flesh this out later >>> pass >>> >>> Does anybody really think that that function will not return None? >> >> Of course it'll return None, but that's nothing to do with the 'pass'. >> The keyword 'pass' doesn't generate any return result at all. > > Function return values are a red herring here. The > point is to allow the programmer to directly express > the intent of the code without having to introduce > spurious values that will be ignored. > > There's no logical difference between not generating > a result at all, and generating a result of None and > then throwing it away. > > The same thing applies here: > > menu.remove(mint) except ValueError: pass Yes, this does read better than (say) "except ValueError: None" or even "except ValueError: doNothing()". I'm starting to like it, although it introduces an inconsistency between expressions whose values are used and those whose values are not used. > > > This says exactly what the programmer means: "Remove > mint from the menu if it's there, otherwise do nothing." > > An alternative would be to allow the exceptional > value part to be omitted altogether: > > menu.remove(mint) except ValueError > > but that looks incomplete, making you wonder if the > programmer forgot something. > > Either way, this would be allowed *only* for top > level expressions whose value is ignored. Yes of course, otherwise it would make no sense. Rob Cliffe > In any > context where the value of the expression is used for > something, the exceptional value would have to be > spelled out explicitly. > From python at mrabarnett.plus.com Tue Feb 18 01:57:28 2014 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 18 Feb 2014 00:57:28 +0000 Subject: [Python-ideas] except expression In-Reply-To: <5302A74A.9060809@stoneleaf.us> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <53017838.9050706@stoneleaf.us> <53028CA1.5080502@canterbury.ac.nz> <5302A74A.9060809@stoneleaf.us> Message-ID: <5302AFF8.3080608@mrabarnett.plus.com> On 2014-02-18 00:20, Ethan Furman wrote: > On 02/17/2014 03:54 PM, Chris Angelico wrote: >> On Tue, Feb 18, 2014 at 9:26 AM, Greg Ewing wrote: >>> There's no logical difference between not generating >>> a result at all, and generating a result of None and >>> then throwing it away. >>> >>> The same thing applies here: >>> >>> menu.remove(mint) except ValueError: pass >>> >>> This says exactly what the programmer means: "Remove >>> mint from the menu if it's there, otherwise do nothing." >> >> In a statement context, it's possible to say "otherwise do nothing". >> An if without an else does this, as does putting 'pass' in certain >> places. >> >> But in an expression context, there are 'two possibility'. Either some >> kind of value is returned, or an exception is raised. (At least, I >> don't think there are any other options. Harry Hoo, not the Spanish >> Inquisition.) What should happen here: >> >> func(menu.remove(mint) except ValueError: pass) >> >> If remove() raises ValueError, should func be called? If so, with what >> argument? (Note that it's because English is happy with "passing" >> arguments to functions that I was happy with the notation "except >> ValueError pass None", but that's using pass as a syntactic element, >> in place of the colon.) > > Yes, func should be called, and it should be called with no arguments. > In that case, is: func(foo if condition else pass) allowed (being like "func(foo) if condition else func()")? And, in fact, is: func(pass) also allowed (meaning "func()")? From rosuav at gmail.com Tue Feb 18 01:59:20 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 18 Feb 2014 11:59:20 +1100 Subject: [Python-ideas] except expression In-Reply-To: <5302AE69.4080701@btinternet.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <53017838.9050706@stoneleaf.us> <53028CA1.5080502@canterbury.ac.nz> <5302AE69.4080701@btinternet.com> Message-ID: On Tue, Feb 18, 2014 at 11:50 AM, Rob Cliffe wrote: > > On 17/02/2014 22:26, Greg Ewing wrote: >> There's no logical difference between not generating >> a result at all, and generating a result of None and >> then throwing it away. >> >> The same thing applies here: >> >> menu.remove(mint) except ValueError: pass > > Yes, this does read better than (say) "except ValueError: None" or even > "except ValueError: doNothing()". > I'm starting to like it, although it introduces an inconsistency between > expressions whose values are used and those whose values are not used. > >> This says exactly what the programmer means: "Remove >> mint from the menu if it's there, otherwise do nothing." >> >> Either way, this would be allowed *only* for top >> level expressions whose value is ignored. > > Yes of course, otherwise it would make no sense. The trouble with this is that it blurs the line between a statement and an expression. Normally, "menu.remove(mint)" is an expression, it has a value. In the interactive interpreter, any non-None value returned will be printed. Adding "except ValueError: pass" to the end... might make it not an expression any more. Or does it always make it not an expression? This can be spelled: try: f() except ValueError: pass which is only two lines (down from four for a "classic" try block). I'm not sure that going down to one is worth the confusion. You can keep it as an expression by just returning None, or you can make it a statement in two lines. But if you like, I can add another section to the PEP mentioning this. ChrisA From rosuav at gmail.com Tue Feb 18 02:00:46 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 18 Feb 2014 12:00:46 +1100 Subject: [Python-ideas] except expression In-Reply-To: <5302AFF8.3080608@mrabarnett.plus.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <53017838.9050706@stoneleaf.us> <53028CA1.5080502@canterbury.ac.nz> <5302A74A.9060809@stoneleaf.us> <5302AFF8.3080608@mrabarnett.plus.com> Message-ID: On Tue, Feb 18, 2014 at 11:57 AM, MRAB wrote: > In that case, is: > > func(foo if condition else pass) > > allowed (being like "func(foo) if condition else func()")? > > And, in fact, is: > > func(pass) > > also allowed (meaning "func()")? Oh, and: func(pass, 2, 3) should be the same as func(2, 3) right? ChrisA From ron3200 at gmail.com Tue Feb 18 02:31:10 2014 From: ron3200 at gmail.com (Ron Adam) Date: Mon, 17 Feb 2014 19:31:10 -0600 Subject: [Python-ideas] except expression In-Reply-To: <530281D1.1070808@canterbury.ac.nz> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> <53013CD3.8030809@canterbury.ac.nz> <530281D1.1070808@canterbury.ac.nz> Message-ID: On 02/17/2014 03:40 PM, Greg Ewing wrote: > Chris Angelico wrote: >> Do you mean that there may not be a comma, or that commas must be >> surrounded by parens? > > Commas would need to be surroounded by parens. > >> Imagine if the exception >> list isn't literals, but comes from somewhere else (which is perfectly >> legal). How do you eyeball it and see that this is now a new except >> clause? > > I'm not a big fan of this idea either. It seems > to me that wanting more than one except clause is > going to be *extremely* rare, so it's not worth > going out of our way to make it super-concise. I agree, and adding too much too soon, could limit options for later enhancement. In the recent thread on enhanced exceptions, it's suggested we add more information to exceptions. I have a different point of view that would have an effect on this idea. To me, the added information on exceptions, should also be usable, preferable to allow more control on catching exceptions. For example:. value = expr1 except IndexError from foo: expr2 Where the exception caught is one raised in "foo". While other IndexError's are allowed to bubble out. We can't use else... although that would be my first choice if else wasn't already used differently in except and for statements. To explain a bit further, when you get a trace back from an uncaught exception, the last line of the trace back contains the function it happened in. But that info is difficult to get from a caught exception. It's even nicer to be able to use it to specify more precisely what exceptions to catch. Cheers, Ron From ethan at stoneleaf.us Tue Feb 18 02:30:10 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 17 Feb 2014 17:30:10 -0800 Subject: [Python-ideas] except expression In-Reply-To: <5302AFF8.3080608@mrabarnett.plus.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <53017838.9050706@stoneleaf.us> <53028CA1.5080502@canterbury.ac.nz> <5302A74A.9060809@stoneleaf.us> <5302AFF8.3080608@mrabarnett.plus.com> Message-ID: <5302B7A2.1050008@stoneleaf.us> On 02/17/2014 04:57 PM, MRAB wrote: > On 2014-02-18 00:20, Ethan Furman wrote: >> On 02/17/2014 03:54 PM, Chris Angelico wrote: >>> On Tue, Feb 18, 2014 at 9:26 AM, Greg Ewing wrote: >>>> There's no logical difference between not generating >>>> a result at all, and generating a result of None and >>>> then throwing it away. >>>> >>>> The same thing applies here: >>>> >>>> menu.remove(mint) except ValueError: pass >>>> >>>> This says exactly what the programmer means: "Remove >>>> mint from the menu if it's there, otherwise do nothing." >>> >>> In a statement context, it's possible to say "otherwise do nothing". >>> An if without an else does this, as does putting 'pass' in certain >>> places. >>> >>> But in an expression context, there are 'two possibility'. Either some >>> kind of value is returned, or an exception is raised. (At least, I >>> don't think there are any other options. Harry Hoo, not the Spanish >>> Inquisition.) What should happen here: >>> >>> func(menu.remove(mint) except ValueError: pass) >>> >>> If remove() raises ValueError, should func be called? If so, with what >>> argument? (Note that it's because English is happy with "passing" >>> arguments to functions that I was happy with the notation "except >>> ValueError pass None", but that's using pass as a syntactic element, >>> in place of the colon.) >> >> Yes, func should be called, and it should be called with no arguments. >> > In that case, is: > > func(foo if condition else pass) > > allowed (being like "func(foo) if condition else func()")? > > And, in fact, is: > > func(pass) > > also allowed (meaning "func()")? I'm not saying it's a good idea, just that it's the natural conclusion from allowing pass. I don't care for it, myself. Just use None, since that's Python "No value" value. -- ~Ethan~ From rosuav at gmail.com Tue Feb 18 03:08:52 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 18 Feb 2014 13:08:52 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> <53013CD3.8030809@canterbury.ac.nz> <530281D1.1070808@canterbury.ac.nz> Message-ID: On Tue, Feb 18, 2014 at 12:31 PM, Ron Adam wrote: > To me, the added information on exceptions, should also be usable, > preferable to allow more control on catching exceptions. > > For example:. > > value = expr1 except IndexError from foo: expr2 > > Where the exception caught is one raised in "foo". While other IndexError's > are allowed to bubble out. > > We can't use else... although that would be my first choice if else wasn't > already used differently in except and for statements. > > To explain a bit further, when you get a trace back from an uncaught > exception, the last line of the trace back contains the function it happened > in. But that info is difficult to get from a caught exception. > It's even nicer to be able to use it to specify more precisely what > exceptions to catch. I'd say that that's tangential to this proposal. Let's look at that in terms of the statement try/except: # Catch everything and swallow it try: foo() except: pass # Catch everything and reraise it (pretty useless actually) try: foo() except: raise # Catch based on class: try: foo() except IndexError: pass except: pass # This bit is implicitly done, if you like # Catch based on something else: try: foo() except Exception as e: if e.foo: pass else: raise except: raise It does make sense to be able to, for instance: # Catch based on something else: try: foo() except IndexError as e if e.key<30: pass One advantage of the currently-favoured syntax for exception expressions is that it'd be easy to add extra syntax to both statement and expression forms, since they're identical (modulo the "try:" at the beginning). So if someone wants to propose any of these hypothetical features, they'd fit fine into the expression version too: try: foo() except as e: pass # Catch everything, and capture the exception try: foo() except OSError not FileNotFoundError: pass # Don't ignore FileNotFound except Exception as e: # log the exception, which might be several lines of code try: foo() except collections.Callable and collections.Iterable as e: # Deal with... exceptions that are both callable and iterable?!?? Okay, so that last one is a bit stupid, but you get the idea. Same thing would work in either context, so it can be raised (pun intended) as a separate proposal. ChrisA From python at mrabarnett.plus.com Tue Feb 18 03:09:05 2014 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 18 Feb 2014 02:09:05 +0000 Subject: [Python-ideas] except expression In-Reply-To: <5302B7A2.1050008@stoneleaf.us> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <53017838.9050706@stoneleaf.us> <53028CA1.5080502@canterbury.ac.nz> <5302A74A.9060809@stoneleaf.us> <5302AFF8.3080608@mrabarnett.plus.com> <5302B7A2.1050008@stoneleaf.us> Message-ID: <5302C0C1.8000204@mrabarnett.plus.com> On 2014-02-18 01:30, Ethan Furman wrote: > On 02/17/2014 04:57 PM, MRAB wrote: >> On 2014-02-18 00:20, Ethan Furman wrote: >>> On 02/17/2014 03:54 PM, Chris Angelico wrote: >>>> On Tue, Feb 18, 2014 at 9:26 AM, Greg Ewing wrote: >>>>> There's no logical difference between not generating >>>>> a result at all, and generating a result of None and >>>>> then throwing it away. >>>>> >>>>> The same thing applies here: >>>>> >>>>> menu.remove(mint) except ValueError: pass >>>>> >>>>> This says exactly what the programmer means: "Remove >>>>> mint from the menu if it's there, otherwise do nothing." >>>> >>>> In a statement context, it's possible to say "otherwise do nothing". >>>> An if without an else does this, as does putting 'pass' in certain >>>> places. >>>> >>>> But in an expression context, there are 'two possibility'. Either some >>>> kind of value is returned, or an exception is raised. (At least, I >>>> don't think there are any other options. Harry Hoo, not the Spanish >>>> Inquisition.) What should happen here: >>>> >>>> func(menu.remove(mint) except ValueError: pass) >>>> >>>> If remove() raises ValueError, should func be called? If so, with what >>>> argument? (Note that it's because English is happy with "passing" >>>> arguments to functions that I was happy with the notation "except >>>> ValueError pass None", but that's using pass as a syntactic element, >>>> in place of the colon.) >>> >>> Yes, func should be called, and it should be called with no arguments. >>> >> In that case, is: >> >> func(foo if condition else pass) >> >> allowed (being like "func(foo) if condition else func()")? >> >> And, in fact, is: >> >> func(pass) >> >> also allowed (meaning "func()")? > > I'm not saying it's a good idea, just that it's the natural conclusion from allowing pass. > > I don't care for it, myself. Just use None, since that's Python "No value" value. > IMHO, if "pass" were allowed, it should be as part of a statement, not an expression, so: menu.remove(mint) except ValueError: pass would be short for: try: menu.remove(mint) except ValueError: pass but: func(menu.remove(mint) except ValueError: pass) wouldn't be allowed. From python at mrabarnett.plus.com Tue Feb 18 03:11:47 2014 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 18 Feb 2014 02:11:47 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <53017838.9050706@stoneleaf.us> <53028CA1.5080502@canterbury.ac.nz> <5302A74A.9060809@stoneleaf.us> Message-ID: <5302C163.7040403@mrabarnett.plus.com> On 2014-02-18 00:48, Chris Angelico wrote: > On Tue, Feb 18, 2014 at 11:20 AM, Ethan Furman wrote: >> On 02/17/2014 03:54 PM, Chris Angelico wrote: >>> But in an expression context, there are 'two possibility'. Either some >>> kind of value is returned, or an exception is raised. (At least, I >>> don't think there are any other options. Harry Hoo, not the Spanish >>> Inquisition.) What should happen here: >>> >>> func(menu.remove(mint) except ValueError: pass) >>> >>> If remove() raises ValueError, should func be called? If so, with what >>> argument? (Note that it's because English is happy with "passing" >>> arguments to functions that I was happy with the notation "except >>> ValueError pass None", but that's using pass as a syntactic element, >>> in place of the colon.) >> >> >> Yes, func should be called, and it should be called with no arguments. > > Ooh. That's an interesting one. I'm not sure how that would go in > terms of readability, but it is an interesting concept. Very > interesting concept. Not sure that I like it, but ... the mind does > like the idea ... in a weird and dangerous way. > > def throw(ex): > raise ex > > some_func(1, 2, (x if x is not None else throw(SyntaxError)) except > SyntaxError: pass) > > So if x is None, it doesn't get passed at all. Useful feature... wordy > and clunky spelling... no, I'm now sure that I do not like this. But > it's amusing! > That would be clearer as: some_func(1, 2, (x if x is not None else pass)) for some value of "clearer". :-) From rosuav at gmail.com Tue Feb 18 03:19:44 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 18 Feb 2014 13:19:44 +1100 Subject: [Python-ideas] except expression In-Reply-To: <5302C163.7040403@mrabarnett.plus.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <53017838.9050706@stoneleaf.us> <53028CA1.5080502@canterbury.ac.nz> <5302A74A.9060809@stoneleaf.us> <5302C163.7040403@mrabarnett.plus.com> Message-ID: On Tue, Feb 18, 2014 at 1:11 PM, MRAB wrote: >> def throw(ex): >> raise ex >> >> some_func(1, 2, (x if x is not None else throw(SyntaxError)) except >> SyntaxError: pass) >> >> So if x is None, it doesn't get passed at all. Useful feature... wordy >> and clunky spelling... no, I'm now sure that I do not like this. But >> it's amusing! >> > That would be clearer as: > > some_func(1, 2, (x if x is not None else pass)) > > for some value of "clearer". :-) That depends on the "if/else pass" syntax also being added. The convolutions above mean that only "except: pass" is needed. :) Actually, all it requires is that there be some kind of magic object called "pass" which is whatever the function's default is. Then it would behave exactly as described, bar the bit about actually passing less args - it'd require some sort of default. I'm still not sure it's a good idea, though :) ChrisA From rob.cliffe at btinternet.com Tue Feb 18 03:32:44 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Tue, 18 Feb 2014 02:32:44 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> <53013CD3.8030809@canterbury.ac.nz> <530281D1.1070808@canterbury.ac.nz> Message-ID: <5302C64C.4040608@btinternet.com> On 18/02/2014 02:08, Chris Angelico wrote: > On Tue, Feb 18, 2014 at 12:31 PM, Ron Adam wrote: >> To me, the added information on exceptions, should also be usable, >> preferable to allow more control on catching exceptions. >> >> For example:. >> >> value = expr1 except IndexError from foo: expr2 >> >> Where the exception caught is one raised in "foo". While other IndexError's >> are allowed to bubble out. >> >> We can't use else... although that would be my first choice if else wasn't >> already used differently in except and for statements. >> >> To explain a bit further, when you get a trace back from an uncaught >> exception, the last line of the trace back contains the function it happened >> in. But that info is difficult to get from a caught exception. >> It's even nicer to be able to use it to specify more precisely what >> exceptions to catch. > I'd say that that's tangential to this proposal. Let's look at that in > terms of the statement try/except: > > # Catch everything and swallow it > try: foo() > except: pass > > # Catch everything and reraise it (pretty useless actually) > try: foo() > except: raise > > # Catch based on class: > try: foo() > except IndexError: pass > except: pass # This bit is implicitly done, if you like > > # Catch based on something else: > try: foo() > except Exception as e: > if e.foo: pass > else: raise > except: raise > > It does make sense to be able to, for instance: > > # Catch based on something else: > try: foo() > except IndexError as e if e.key<30: pass > > One advantage of the currently-favoured syntax for exception > expressions is that it'd be easy to add extra syntax to both statement > and expression forms, since they're identical (modulo the "try:" at > the beginning). So if someone wants to propose any of these > hypothetical features, they'd fit fine into the expression version > too: > > try: foo() > except as e: pass # Catch everything, and capture the exception > > try: foo() > except OSError not FileNotFoundError: pass # Don't ignore FileNotFound > except Exception as e: > # log the exception, which might be several lines of code > > try: foo() > except collections.Callable and collections.Iterable as e: > # Deal with... exceptions that are both callable and iterable?!?? > > Okay, so that last one is a bit stupid, but you get the idea. Same > thing would work in either context, so it can be raised (pun intended) > as a separate proposal. Yes! I love consistency. No new learning curve. Did I not imply in an earlier post that any future syntax added to "except" statements could also be incorporated in "except" expressions? (I did.) (Please be tolerant with me, I've had a few drinks. Thank you. :-) ) Rob Cliffe > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > ----- > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6600 - Release Date: 02/17/14 > > From rosuav at gmail.com Tue Feb 18 03:35:35 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 18 Feb 2014 13:35:35 +1100 Subject: [Python-ideas] except expression In-Reply-To: <5302C64C.4040608@btinternet.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> <53013CD3.8030809@canterbury.ac.nz> <530281D1.1070808@canterbury.ac.nz> <5302C64C.4040608@btinternet.com> Message-ID: On Tue, Feb 18, 2014 at 1:32 PM, Rob Cliffe wrote: > Yes! I love consistency. No new learning curve. Did I not imply in an > earlier post that any future syntax added to "except" statements could also > be incorporated in "except" expressions? (I did.) (Please be tolerant with > me, I've had a few drinks. Thank you. :-) ) Yeah. That point is a strong one in favour of the "except Exception [as e]: expr" notation. ChrisA From rob.cliffe at btinternet.com Tue Feb 18 03:37:13 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Tue, 18 Feb 2014 02:37:13 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <8fa6bbba11e96b623c2f92407c556141@chopin.edu.pl> <68242d0cc2aec250943f8639dfc952e1@chopin.edu.pl> <53013CD3.8030809@canterbury.ac.nz> <530281D1.1070808@canterbury.ac.nz> <5302C64C.4040608@btinternet.com> Message-ID: <5302C759.1070707@btinternet.com> On 18/02/2014 02:35, Chris Angelico wrote: > On Tue, Feb 18, 2014 at 1:32 PM, Rob Cliffe wrote: >> Yes! I love consistency. No new learning curve. Did I not imply in an >> earlier post that any future syntax added to "except" statements could also >> be incorporated in "except" expressions? (I did.) (Please be tolerant with >> me, I've had a few drinks. Thank you. :-) ) > Yeah. That point is a strong one in favour of the "except Exception > [as e]: expr" notation. I agree. (Of course.) RobC > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > ----- > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6600 - Release Date: 02/17/14 > > From rosuav at gmail.com Tue Feb 18 05:45:16 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 18 Feb 2014 15:45:16 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <53029398.5030409@canterbury.ac.nz> Message-ID: On Tue, Feb 18, 2014 at 11:06 AM, Chris Angelico wrote: > https://github.com/Rosuav/ExceptExpr/raw/master/pep.txt > > ChrisA If a core committer who's active in this discussion has the time, could one please assign a number and commit this into the PEPs repo? Thanks! ChrisA From greg.ewing at canterbury.ac.nz Tue Feb 18 06:19:28 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 18 Feb 2014 18:19:28 +1300 Subject: [Python-ideas] Commas [was Re: except expression] In-Reply-To: References: <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <5301DDC6.6020505@egenix.com> <53021DBE.5030006@btinternet.com> <5302240C.8030105@egenix.com> <20140217210717.GN4519@ando> Message-ID: <5302ED60.9010307@canterbury.ac.nz> Alexander Belopolsky wrote: > Funny: less than an hour ago, I paused when writing a lambda to return a > tuple. I conservatively put parentheses around the tuple and did not > bother to check if they are required. Now I know that they are. It's even more confusing when you consider that you can write lambda x, y: x + y, 2 and the first comma doesn't break up the lambda, but the second one does! With oddities like this already in the language, I don't think we need to worry too much about the corner cases. People are always free to insert parens to make things clearer. E.g. I would probably write the above as (lambda (x, y): x + y), 2 -- Greg From greg.ewing at canterbury.ac.nz Tue Feb 18 07:19:25 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 18 Feb 2014 19:19:25 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <53017838.9050706@stoneleaf.us> <53028CA1.5080502@canterbury.ac.nz> Message-ID: <5302FB6D.4030006@canterbury.ac.nz> Chris Angelico wrote: > func(menu.remove(mint) except ValueError: pass) That would be a syntax error, because the except expression is *not* in a value-ignoring context here. You would have to say func(menu.remove(mint) except ValueError: None) -- Greg From greg.ewing at canterbury.ac.nz Tue Feb 18 07:23:12 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 18 Feb 2014 19:23:12 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> Message-ID: <5302FC50.8060109@canterbury.ac.nz> Chris Angelico wrote: > Conceptually, it's easy to say > "inner scope". If it turns out that it's really REALLY hard to do > that, then rejecting 'as' could be a fall-back. But I'd really rather > keep it. Do you have any non-contrived use cases in mind for 'as' in an except-expression? I'm having trouble thinking of any. -- Greg From rosuav at gmail.com Tue Feb 18 07:27:09 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 18 Feb 2014 17:27:09 +1100 Subject: [Python-ideas] except expression In-Reply-To: <5302FC50.8060109@canterbury.ac.nz> References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> Message-ID: On Tue, Feb 18, 2014 at 5:23 PM, Greg Ewing wrote: > Chris Angelico wrote: >> >> Conceptually, it's easy to say >> "inner scope". If it turns out that it's really REALLY hard to do >> that, then rejecting 'as' could be a fall-back. But I'd really rather >> keep it. > > > Do you have any non-contrived use cases in mind for 'as' in an > except-expression? I'm having trouble thinking of any. There are a few in the current PEP draft now. Fetching info out of an exception isn't uncommon. ChrisA From rosuav at gmail.com Tue Feb 18 07:27:52 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 18 Feb 2014 17:27:52 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> Message-ID: On Tue, Feb 18, 2014 at 5:27 PM, Chris Angelico wrote: > There are a few in the current PEP draft now. Fetching info out of an > exception isn't uncommon. Oops, meant to include the link. https://raw2.github.com/Rosuav/ExceptExpr/master/pep.txt ChrisA From greg.ewing at canterbury.ac.nz Tue Feb 18 07:37:10 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 18 Feb 2014 19:37:10 +1300 Subject: [Python-ideas] except expression In-Reply-To: <5302AE69.4080701@btinternet.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <53017838.9050706@stoneleaf.us> <53028CA1.5080502@canterbury.ac.nz> <5302AE69.4080701@btinternet.com> Message-ID: <5302FF96.7050604@canterbury.ac.nz> Rob Cliffe wrote: > > On 17/02/2014 22:26, Greg Ewing wrote: > >> menu.remove(mint) except ValueError: pass > > I'm starting to like it, although it introduces an inconsistency > between expressions whose values are used and those whose values are not > used. If it helps, you could think of this form of the syntax as a statement rather than an expression. It's an abbreviation for try: menu.remove(mint) except ValueError: pass which you wouldn't otherwise be able to write on one line. -- Greg From henry.schafer.harrison at gmail.com Tue Feb 18 07:41:22 2014 From: henry.schafer.harrison at gmail.com (Henry Harrison) Date: Tue, 18 Feb 2014 01:41:22 -0500 Subject: [Python-ideas] pickling obscured function references Message-ID: Hello all, I can't be the first to have thought of this, so there must be a reason this isn't the case, but I have to ask. Why is __main__ the fallback when pickle can't find a function reference? Instead of something like: os.path.basename(inspect.getsourcefile(func))[:-3] Thanks for humoring my curiosity, - Henry -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Tue Feb 18 08:12:00 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 18 Feb 2014 20:12:00 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> Message-ID: <530307C0.7080409@canterbury.ac.nz> Chris Angelico wrote: > On Tue, Feb 18, 2014 at 5:23 PM, Greg Ewing wrote: > >>Do you have any non-contrived use cases in mind for 'as' in an >>except-expression? I'm having trouble thinking of any. > > There are a few in the current PEP draft now. Fetching info out of an > exception isn't uncommon. Not in general, but in situations where you need the conciseness of an except-expression? The examples in the PEP all seem to be variations on value = next(it) except StopIteration as e: e.args[0] which seems a bit contrived. It's hard to think of a situation where you'd want to treat the yielded values and the return value of a generator in the same way. -- Greg From rosuav at gmail.com Tue Feb 18 08:22:12 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 18 Feb 2014 18:22:12 +1100 Subject: [Python-ideas] except expression In-Reply-To: <530307C0.7080409@canterbury.ac.nz> References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> Message-ID: On Tue, Feb 18, 2014 at 6:12 PM, Greg Ewing wrote: > Not in general, but in situations where you need the > conciseness of an except-expression? > > The examples in the PEP all seem to be variations on > > > value = next(it) except StopIteration as e: e.args[0] > > which seems a bit contrived. It's hard to think of a > situation where you'd want to treat the yielded values > and the return value of a generator in the same way. There's another one based on retrieving a document or its error text, and showing that to a human. Beyond that, I really need other people to make suggestions; there are those here who spend 99% of their time writing Python code and know all the ins and outs of the libraries, but I am not them. ChrisA From abarnert at yahoo.com Tue Feb 18 08:33:34 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Mon, 17 Feb 2014 23:33:34 -0800 Subject: [Python-ideas] except expression In-Reply-To: <5302FF96.7050604@canterbury.ac.nz> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <53017838.9050706@stoneleaf.us> <53028CA1.5080502@canterbury.ac.nz> <5302AE69.4080701@btinternet.com> <5302FF96.7050604@canterbury.ac.nz> Message-ID: <0309175B-998F-4E9D-8687-F536120CD568@yahoo.com> On Feb 17, 2014, at 22:37, Greg Ewing wrote: > Rob Cliffe wrote: >> On 17/02/2014 22:26, Greg Ewing wrote: >>> menu.remove(mint) except ValueError: pass >> I'm starting to like it, although it introduces an inconsistency between expressions whose values are used and those whose values are not used. > > If it helps, you could think of this form of the syntax > as a statement rather than an expression. That helps a lot, in that it explains exactly what's wrong with the idea. The except expression is useful because it lets you handle exceptions in any expression context--a function argument, whatever. Code that would otherwise be clunky to write becomes simple. The fact that it also lets you avoid a newline or two is not the point, and is barely worth mentioning as an added bonus. Adding pass to the except expression does not allow any code to become simpler. The _only_ thing it does is allow you to avoid a newline. And it doesn't even do _that_: > It's an > abbreviation for > > try: > menu.remove(mint) > except ValueError: > pass > > which you wouldn't otherwise be able to write on one line. Sure you would: menu.remove(mint) except ValueError: None Exactly as short as the "pass" version, and without requiring the user or the compiler (or the linter, font-lock mode, etc.) to go through the hurdles of realizing that this is happening at the top level of an expression statement, rather than just in any expression, and therefore the value is guaranteed ignored (which isn't actually even true, given things like _ at the interactive interpreter, but we have to _pretend_ it is to even make sense of the idea), and therefore we use different syntax with a pass keyword in place of a value. OK, it does require one more keystroke to capitalize the N, but if that bothers you, write it this way: menu.remove(mint) except ValueError: 0 From p.f.moore at gmail.com Tue Feb 18 09:00:56 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 18 Feb 2014 08:00:56 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> Message-ID: On 18 February 2014 07:22, Chris Angelico wrote: > On Tue, Feb 18, 2014 at 6:12 PM, Greg Ewing wrote: >> Not in general, but in situations where you need the >> conciseness of an except-expression? >> >> The examples in the PEP all seem to be variations on >> >> >> value = next(it) except StopIteration as e: e.args[0] >> >> which seems a bit contrived. It's hard to think of a >> situation where you'd want to treat the yielded values >> and the return value of a generator in the same way. > > There's another one based on retrieving a document or its error text, > and showing that to a human. Beyond that, I really need other people > to make suggestions; there are those here who spend 99% of their time > writing Python code and know all the ins and outs of the libraries, > but I am not them. That was about the only even vaguely compelling one to me, and even then I'm not convinced. Reproducing it here (btw, spacing out the proposed and current syntax would be a huge help): Set a PyGTK label to a human-readable result from fetching a URL:: display.set_text( urllib.request.urlopen(url) except urllib.error.HTTPError as e: "Error %d: %s"%(x.getcode(), x.msg) except (ValueError, urllib.error.URLError) as e: "Invalid URL: "+str(e) ) try: display.set_text(urllib.request.urlopen(url)) except urllib.error.HTTPError as e: display.set_text("Error %d: %s"%(x.getcode(), x.msg)) except (ValueError, urllib.error.URLError) as e: display.set_text("Invalid URL: "+str(e)) However, your "current syntax" is not what I'd write at all - why not use a temporary variable, as in: try: text = urllib.request.urlopen(url) except urllib.error.HTTPError as e: text = "Error %d: %s"%(x.getcode(), x.msg) except (ValueError, urllib.error.URLError) as e: text = "Invalid URL: "+str(e) display.set_text(text) That seems much clearer and natural to me than either of your two examples. At this point the discussion starts to remind me of the "we need multiline lambda" discussions, which fall apart when trying to find compelling use cases where a simple def using a named function wouldn't be better. I really think that this proposal needs to focus on the short, simple use cases and *not* worry about too much generality. For example: sum(x[3] except 0 for x in list_of_tuples) Note that when your controlled expression is sufficiently small (an index lookup here) you don't even need an exception name, because you'll never *get* anything except IndexError. And yes, I know this is naive and I know that function calls are the obvious extension and you probably need the exception type there, but my point is that if you focus on the simplest case only, and reject any extension of scope that isn't backed by a genuine real-world use case that someone can show you in actual code, you'll end up with a much tighter proposal. I would actually be interested in being pointed at real-world code that wouldn't work fine with only a "catch everything" option. Note that the urllib.request example above doesn't bother me because (a) the multiline version with a named variable suits me fine, and (b) if I want something shorter all I need to do is use more generic error text: urllib.request.urlopen(url) except "Cannot fetch " + url. Raymond Hettinger often evaluates proposals like this by going through the standard library looking for code that would be improved by converting to use the new syntax. I suspect that this would be a good exercise for this proposal (grepping for except in the stdlib should be easy enough to start with, and would give you a pretty long list of examples you can look at :-) Paul From bruce at leapyear.org Tue Feb 18 09:08:57 2014 From: bruce at leapyear.org (Bruce Leban) Date: Tue, 18 Feb 2014 00:08:57 -0800 Subject: [Python-ideas] except expression In-Reply-To: <0309175B-998F-4E9D-8687-F536120CD568@yahoo.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <53017838.9050706@stoneleaf.us> <53028CA1.5080502@canterbury.ac.nz> <5302AE69.4080701@btinternet.com> <5302FF96.7050604@canterbury.ac.nz> <0309175B-998F-4E9D-8687-F536120CD568@yahoo.com> Message-ID: I think the idea of using commas for grouping in an except expression is a bad idea. To be clear when I say grouping what I mean is that as defined in the proposal when the comma is omitted between two except clauses, they form two separate except blocks and including the comma groups them into one. Compare: A , A , Here's color-coding to show the expressions we would have. A , A , This is nothing like how commas are used anywhere else in the language. I think it much better to use parenthesis for grouping: (A ) (A ) --- Bruce Learn how hackers think: http://j.mp/gruyere-security -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Tue Feb 18 13:49:17 2014 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 18 Feb 2014 12:49:17 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> Message-ID: <530356CD.3060506@mrabarnett.plus.com> On 2014-02-18 08:00, Paul Moore wrote: > On 18 February 2014 07:22, Chris Angelico wrote: >> On Tue, Feb 18, 2014 at 6:12 PM, Greg Ewing wrote: >>> Not in general, but in situations where you need the >>> conciseness of an except-expression? >>> >>> The examples in the PEP all seem to be variations on >>> >>> >>> value = next(it) except StopIteration as e: e.args[0] >>> >>> which seems a bit contrived. It's hard to think of a >>> situation where you'd want to treat the yielded values >>> and the return value of a generator in the same way. >> >> There's another one based on retrieving a document or its error text, >> and showing that to a human. Beyond that, I really need other people >> to make suggestions; there are those here who spend 99% of their time >> writing Python code and know all the ins and outs of the libraries, >> but I am not them. > > That was about the only even vaguely compelling one to me, and even > then I'm not convinced. Reproducing it here (btw, spacing out the > proposed and current syntax would be a huge help): > > Set a PyGTK label to a human-readable result from fetching a URL:: > > display.set_text( > urllib.request.urlopen(url) > except urllib.error.HTTPError as e: "Error %d: %s"%(x.getcode(), x.msg) > except (ValueError, urllib.error.URLError) as e: "Invalid URL: "+str(e) > ) > > > try: > display.set_text(urllib.request.urlopen(url)) > except urllib.error.HTTPError as e: > display.set_text("Error %d: %s"%(x.getcode(), x.msg)) > except (ValueError, urllib.error.URLError) as e: > display.set_text("Invalid URL: "+str(e)) > > However, your "current syntax" is not what I'd write at all - why not > use a temporary variable, as in: > > try: > text = urllib.request.urlopen(url) > except urllib.error.HTTPError as e: > text = "Error %d: %s"%(x.getcode(), x.msg) > except (ValueError, urllib.error.URLError) as e: > text = "Invalid URL: "+str(e) > display.set_text(text) > > That seems much clearer and natural to me than either of your two > examples. At this point the discussion starts to remind me of the "we > need multiline lambda" discussions, which fall apart when trying to > find compelling use cases where a simple def using a named function > wouldn't be better. > > I really think that this proposal needs to focus on the short, simple > use cases and *not* worry about too much generality. For example: > > sum(x[3] except 0 for x in list_of_tuples) > Shouldn't that be: sum(x[3] except: 0 for x in list_of_tuples) (colon added)? [snip] From p.f.moore at gmail.com Tue Feb 18 14:00:54 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 18 Feb 2014 13:00:54 +0000 Subject: [Python-ideas] except expression In-Reply-To: <530356CD.3060506@mrabarnett.plus.com> References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: On 18 February 2014 12:49, MRAB wrote: >> I really think that this proposal needs to focus on the short, simple >> use cases and *not* worry about too much generality. For example: >> >> sum(x[3] except 0 for x in list_of_tuples) >> > Shouldn't that be: > > > sum(x[3] except: 0 for x in list_of_tuples) > > (colon added)? Only if you feel that a colon is necessary for the syntax. I don't :-) Paul. From rosuav at gmail.com Tue Feb 18 14:37:04 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 00:37:04 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: On Wed, Feb 19, 2014 at 12:00 AM, Paul Moore wrote: > On 18 February 2014 12:49, MRAB wrote: >>> I really think that this proposal needs to focus on the short, simple >>> use cases and *not* worry about too much generality. For example: >>> >>> sum(x[3] except 0 for x in list_of_tuples) >>> >> Shouldn't that be: >> >> >> sum(x[3] except: 0 for x in list_of_tuples) >> >> (colon added)? > > Only if you feel that a colon is necessary for the syntax. I don't :-) The colon is necessary if the new syntax should allow the specification of the exception. If it's always equivalent to a bare except that doesn't capture the exception object, then it'd be unnecessary. Personally, I don't want to create a special syntax that does something that's strongly discouraged. I'm open to argument that the expression-except should accept just a single except clause; I'm open to argument that it shouldn't be able to capture (due to complications of creating sub-scopes), though that's more a nod to implementation/specification difficulty than any conceptual dislike of the syntax (it's clean, it's consistent, it's sometimes useful); but I don't want to narrow this down to always having to catch everything. ChrisA From p.f.moore at gmail.com Tue Feb 18 15:07:43 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 18 Feb 2014 14:07:43 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: On 18 February 2014 13:37, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 12:00 AM, Paul Moore wrote: >> On 18 February 2014 12:49, MRAB wrote: >>>> I really think that this proposal needs to focus on the short, simple >>>> use cases and *not* worry about too much generality. For example: >>>> >>>> sum(x[3] except 0 for x in list_of_tuples) >>>> >>> Shouldn't that be: >>> >>> >>> sum(x[3] except: 0 for x in list_of_tuples) >>> >>> (colon added)? >> >> Only if you feel that a colon is necessary for the syntax. I don't :-) > > The colon is necessary if the new syntax should allow the > specification of the exception. If it's always equivalent to a bare > except that doesn't capture the exception object, then it'd be > unnecessary. Well, yes and no. There are only 2 out of the 10 syntaxes originally proposed in the PEP at the start of this thread that use a colon. I've already pointed out that I don't like a colon, so assume I said "except return 0" if you must :-) > Personally, I don't want to create a special syntax that does > something that's strongly discouraged. I'm open to argument that the > expression-except should accept just a single except clause; I'm open > to argument that it shouldn't be able to capture (due to complications > of creating sub-scopes), though that's more a nod to > implementation/specification difficulty than any conceptual dislike of > the syntax (it's clean, it's consistent, it's sometimes useful); but I > don't want to narrow this down to always having to catch everything. That's a fair point, certainly, and probably worth capturing explicitly in the PEP if it's not already there. I could argue that "bare except" is more acceptable in an *expression* context because expressions should never be so complex that the reasons why it's bad in a statement context actually apply. But I'd only be playing devil's advocate by doing so. My main point here was to force attention back onto the *real* use cases which (as I see it) are very simple exception handling for a simple subexpression embedded in a compound expression. Examples of how you could catch 3 different exceptions, use exception data, etc, are to my mind missing the point. At that stage, factor out the expression and use try/except *statements* and a temporary variable. Also, "expr except fallback" is a very simple case of a keyword-based binary operation. So (ignoring the bare except issue) it's much less controversial. It's only when you need to specify the exception type that you get into ternary (and n-ary) territory, where there's a lot less consistency or prior art to base design decisions on. Hence all the multiple syntax proposals, fun debates and controversy ;-) Paul From alexander.belopolsky at gmail.com Tue Feb 18 15:27:50 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 18 Feb 2014 09:27:50 -0500 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: On Tue, Feb 18, 2014 at 9:07 AM, Paul Moore wrote: > Also, "expr except fallback" is a very simple case of a keyword-based > binary operation. > I agree and ISTM a natural generalization of this to allow exception type specification would be expr except(Exc1, Exc2) fallback or expr except[Exc1, Exc2] fallback I am not sure if any of these have been proposed, so apologies if they were. This said, I would still prefer the variant with the column as the closest to the existing syntax. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Tue Feb 18 16:05:01 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 02:05:01 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: On Wed, Feb 19, 2014 at 1:07 AM, Paul Moore wrote: > On 18 February 2014 13:37, Chris Angelico wrote: >> On Wed, Feb 19, 2014 at 12:00 AM, Paul Moore wrote: >>> On 18 February 2014 12:49, MRAB wrote: >>>>> I really think that this proposal needs to focus on the short, simple >>>>> use cases and *not* worry about too much generality. For example: >>>>> >>>>> sum(x[3] except 0 for x in list_of_tuples) >>>>> >>>> Shouldn't that be: >>>> >>>> >>>> sum(x[3] except: 0 for x in list_of_tuples) >>>> >>>> (colon added)? >>> >>> Only if you feel that a colon is necessary for the syntax. I don't :-) >> >> The colon is necessary if the new syntax should allow the >> specification of the exception. If it's always equivalent to a bare >> except that doesn't capture the exception object, then it'd be >> unnecessary. > > Well, yes and no. There are only 2 out of the 10 syntaxes originally > proposed in the PEP at the start of this thread that use a colon. I've > already pointed out that I don't like a colon, so assume I said > "except return 0" if you must :-) Whether it's a colon or another keyword, _something_ is necessary in there, if it's permissible to specify an exception type: sum(x[3] except IndexError 0 for x in list_of_tuples) sum(x[3] except 0 for x in list_of_tuples) How would you parse each of those, without a separator? One option might be to have a separator that's present only when the exception type is listed. For instance: sum(x[3] except(IndexError) 0 for x in list_of_tuples) sum(x[3] except 0 for x in list_of_tuples) Does that sort of thing have enough support to be added to the PEP? I'm not sure it gives us anything over the colon notation, which has the benefit of being consistent with the statement form (and is stylistically similar to lambda, so it's not grossly inappropriate to an expression context). The separator can only be omitted if there's exactly two parts to this new syntax, the "try-me" expression and the "give this instead if any exception is thrown" one. And that's the form that I'm strongly disliking, as I said in the next bit. >> Personally, I don't want to create a special syntax that does >> something that's strongly discouraged. I'm open to argument that the >> expression-except should accept just a single except clause; I'm open >> to argument that it shouldn't be able to capture (due to complications >> of creating sub-scopes), though that's more a nod to >> implementation/specification difficulty than any conceptual dislike of >> the syntax (it's clean, it's consistent, it's sometimes useful); but I >> don't want to narrow this down to always having to catch everything. > > That's a fair point, certainly, and probably worth capturing > explicitly in the PEP if it's not already there. I could argue that > "bare except" is more acceptable in an *expression* context because > expressions should never be so complex that the reasons why it's bad > in a statement context actually apply. But I'd only be playing devil's > advocate by doing so. My main point here was to force attention back > onto the *real* use cases which (as I see it) are very simple > exception handling for a simple subexpression embedded in a compound > expression. Examples of how you could catch 3 different exceptions, > use exception data, etc, are to my mind missing the point. At that > stage, factor out the expression and use try/except *statements* and a > temporary variable. Fair point. This should be kept simple. So, let's see. I don't feel like crafting a regular expression that searches the standard library for any one-line try block that assigns to something, so I guess I should figure out how to use ast features other than literal_eval... and it's not making it easy on me. But we'll get there. ChrisA From rosuav at gmail.com Tue Feb 18 16:07:53 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 02:07:53 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: On Wed, Feb 19, 2014 at 1:27 AM, Alexander Belopolsky wrote: > On Tue, Feb 18, 2014 at 9:07 AM, Paul Moore wrote: >> >> Also, "expr except fallback" is a very simple case of a keyword-based >> binary operation. > > > I agree and ISTM a natural generalization of this to allow exception type > specification would be > > expr except(Exc1, Exc2) fallback > > or > > expr except[Exc1, Exc2] fallback > > I am not sure if any of these have been proposed, so apologies if they were. Interestingly, I hadn't read your post when I wrote my own suggestion of parentheses around the exception type. I'm still not convinced that it's an improvement over matching the statement form (it still encourages the bare-except usage, which should be discouraged), but the fact that two people have come up with this means it can go into the PEP. ChrisA From python at mrabarnett.plus.com Tue Feb 18 16:13:30 2014 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 18 Feb 2014 15:13:30 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: <5303789A.7040406@mrabarnett.plus.com> On 2014-02-18 14:27, Alexander Belopolsky wrote: > > On Tue, Feb 18, 2014 at 9:07 AM, Paul Moore > wrote: > > Also, "expr except fallback" is a very simple case of a keyword-based > binary operation. > > > I agree and ISTM a natural generalization of this to allow exception > type specification would be > > expr except(Exc1, Exc2) fallback > Do you mean that the exception type would be optional? If so, then using parentheses would be a problem because the fallback itself could be in parentheses. > or > > expr except[Exc1, Exc2] fallback > > I am not sure if any of these have been proposed, so apologies if they were. > > This said, I would still prefer the variant with the column as the > closest to the existing syntax. > Another possibility would be to say that a bare except in an expression catches only certain "expression-oriented" exceptions, e.g. ValueError. The simplest way to do that would be to make them subclasses of an ExpressionError class. The question then becomes one of which exceptions are "expression-oriented"... From rosuav at gmail.com Tue Feb 18 16:25:15 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 02:25:15 +1100 Subject: [Python-ideas] except expression In-Reply-To: <5303789A.7040406@mrabarnett.plus.com> References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> Message-ID: On Wed, Feb 19, 2014 at 2:13 AM, MRAB wrote: > Another possibility would be to say that a bare except in an expression > catches only certain "expression-oriented" exceptions, e.g. ValueError. > The simplest way to do that would be to make them subclasses of an > ExpressionError class. The question then becomes one of which > exceptions are "expression-oriented"... Easier than fiddling with class inheritance would be to make ExpressionError into a tuple of exception types - at least for testing purposes. If this _is_ to be done, I would definitely advocate having that name (or some equivalent) as a built-in, so it's straight-forward to either manipulate it or use it in a statement-except. But you're absolutely right that the question of which ones are expression-oriented would be a big one. I could imagine catching (ValueError, Unicode{En,De}codeError, AttributeError, EOFError, IOError, OSError, LookupError, NameError, ZeroDivisionError) and that's just from a quick skim of the built-in names. Would you consider making a tuple like that and then using "except CommonErrors:" all over your code? Or more to the point, if it were really convenient to do that, would it improve your code? ChrisA From alexander.belopolsky at gmail.com Tue Feb 18 16:26:05 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 18 Feb 2014 10:26:05 -0500 Subject: [Python-ideas] except expression In-Reply-To: <5303789A.7040406@mrabarnett.plus.com> References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> Message-ID: On Tue, Feb 18, 2014 at 10:13 AM, MRAB wrote: > > Another possibility would be to say that a bare except in an expression > catches only certain "expression-oriented" exceptions, e.g. ValueError. > The simplest way to do that would be to make them subclasses of an > ExpressionError class. +1 > > The question then becomes one of which > exceptions are "expression-oriented"... Here is my attempt to answer: - ArithmeticError - AttributeError - LookupError - TypeError ? - ValueError ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rob.cliffe at btinternet.com Tue Feb 18 16:27:52 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Tue, 18 Feb 2014 15:27:52 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: <53037BF8.1050300@btinternet.com> On 18/02/2014 14:07, Paul Moore wrote: > On 18 February 2014 13:37, Chris Angelico wrote: >> On Wed, Feb 19, 2014 at 12:00 AM, Paul Moore wrote: >>> On 18 February 2014 12:49, MRAB wrote: >>>>> I really think that this proposal needs to focus on the short, simple >>>>> use cases and *not* worry about too much generality. For example: >>>>> >>>>> sum(x[3] except 0 for x in list_of_tuples) >>>>> >>>> Shouldn't that be: >>>> >>>> >>>> sum(x[3] except: 0 for x in list_of_tuples) >>>> >>>> (colon added)? >>> Only if you feel that a colon is necessary for the syntax. I don't :-) >> The colon is necessary if the new syntax should allow the >> specification of the exception. If it's always equivalent to a bare >> except that doesn't capture the exception object, then it'd be >> unnecessary. > Well, yes and no. There are only 2 out of the 10 syntaxes originally > proposed in the PEP at the start of this thread that use a colon. I've > already pointed out that I don't like a colon, so assume I said > "except return 0" if you must :-) > >> Personally, I don't want to create a special syntax that does >> something that's strongly discouraged. I'm open to argument that the >> expression-except should accept just a single except clause; I'm open >> to argument that it shouldn't be able to capture (due to complications >> of creating sub-scopes), though that's more a nod to >> implementation/specification difficulty than any conceptual dislike of >> the syntax (it's clean, it's consistent, it's sometimes useful); but I >> don't want to narrow this down to always having to catch everything. > That's a fair point, certainly, and probably worth capturing > explicitly in the PEP if it's not already there. I could argue that > "bare except" is more acceptable in an *expression* context because > expressions should never be so complex that the reasons why it's bad > in a statement context actually apply. But I'd only be playing devil's > advocate by doing so. My main point here was to force attention back > onto the *real* use cases which (as I see it) are very simple > exception handling for a simple subexpression embedded in a compound > expression. Examples of how you could catch 3 different exceptions, > use exception data, etc, are to my mind missing the point. At that > stage, factor out the expression and use try/except *statements* and a > temporary variable. > > Also, "expr except fallback" is a very simple case of a keyword-based > binary operation. So (ignoring the bare except issue) it's much less > controversial. It's only when you need to specify the exception type > that you get into ternary (and n-ary) territory, where there's a lot > less consistency or prior art to base design decisions on. Hence all > the multiple syntax proposals, fun debates and controversy ;-) > > Paul We *must* be able to specify the exception(s) to catch. Otherwise the whole thing is pretty useless: MisspeltVariableName / x except "You divided by zero." # No you didn't, you raised a NameError. Rob Cliffe > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > ----- > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6602 - Release Date: 02/17/14 > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rob.cliffe at btinternet.com Tue Feb 18 16:30:39 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Tue, 18 Feb 2014 15:30:39 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> Message-ID: <53037C9F.4050405@btinternet.com> On 18/02/2014 15:26, Alexander Belopolsky wrote: > > On Tue, Feb 18, 2014 at 10:13 AM, MRAB > wrote: > > > > Another possibility would be to say that a bare except in an expression > > catches only certain "expression-oriented" exceptions, e.g. ValueError. > > The simplest way to do that would be to make them subclasses of an > > ExpressionError class. > > > +1 > > > > > The question then becomes one of which > > exceptions are "expression-oriented"... > > Here is my attempt to answer: > > - ArithmeticError > - AttributeError > - LookupError > - TypeError ? > - ValueError ? > > Sorry, but this seems to me to be a futile discussion. Expressions can contain functions. Functions can raise any exception they like. So evaluating an expression can raise any exception you care to name. Rob Cliffe > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6602 - Release Date: 02/17/14 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.belopolsky at gmail.com Tue Feb 18 16:41:21 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 18 Feb 2014 10:41:21 -0500 Subject: [Python-ideas] except expression In-Reply-To: <53037C9F.4050405@btinternet.com> References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> Message-ID: On Tue, Feb 18, 2014 at 10:30 AM, Rob Cliffe wrote: > > Sorry, but this seems to me to be a futile discussion. > Expressions can contain functions. > Functions can raise any exception they like. > So evaluating an expression can raise any exception you care to name. This is true, but it does not follow that you need to be able to catch "any exception you care to name" in the expression. Exceptions that are caused by conditions not obvious from the expression are better handled at a higher level. Let's focus on the main use cases. E.g. a better spelling for x = d.get(key, default) -> x = d[key] except default try: x = 1/y except ArithmeticError: x=0; -> x = 1/y except 0 etc. I would not mind x = 1/f(y) except 0 not catching a TypeError that may come from f(y) in most cases. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Tue Feb 18 16:40:50 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 02:40:50 +1100 Subject: [Python-ideas] except expression In-Reply-To: <53037C9F.4050405@btinternet.com> References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> Message-ID: On Wed, Feb 19, 2014 at 2:30 AM, Rob Cliffe wrote: > Sorry, but this seems to me to be a futile discussion. > Expressions can contain functions. > Functions can raise any exception they like. > So evaluating an expression can raise any exception you care to name. Of course it could raise anything, but what are you likely to want to catch and handle by returning a different value? LookupError when your cache isn't populated UnicodeDecodeError to fall back on a different encoding ("try UTF-8; if that fails, assume the other end is Windows and try CP-1252" - I have that logic in some places) ZeroDivisionError and return float("inf"), float("nan"), or 0 But you wouldn't want to catch: SyntaxError ImportError AssertionError IndentationError ProgrammerIsAMoronAndHisCodeIsInError because most of them wouldn't come up in an expression context, and if they do, you aren't likely to have a useful alternate value to return. So it is conceivable that there be some tuple or superclass of "common errors worth catching". Trouble is, it still often wouldn't be appropriate to catch them all - maybe you want to hook into LookupError, but accidentally catching a UnicodeDecodeError along the way would mask a bug. ChrisA From rosuav at gmail.com Tue Feb 18 16:51:20 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 02:51:20 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> Message-ID: On Wed, Feb 19, 2014 at 2:41 AM, Alexander Belopolsky wrote: > I would not mind > > x = 1/f(y) except 0 > > not catching a TypeError that may come from f(y) in most cases. But should it catch a KeyError from inside f(y), based on the translation of d.get()? ChrisA From p.f.moore at gmail.com Tue Feb 18 16:56:10 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 18 Feb 2014 15:56:10 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: On 18 February 2014 15:05, Chris Angelico wrote: >> Well, yes and no. There are only 2 out of the 10 syntaxes originally >> proposed in the PEP at the start of this thread that use a colon. I've >> already pointed out that I don't like a colon, so assume I said >> "except return 0" if you must :-) > > Whether it's a colon or another keyword, _something_ is necessary in > there, if it's permissible to specify an exception type: > > sum(x[3] except IndexError 0 for x in list_of_tuples) > sum(x[3] except 0 for x in list_of_tuples) > > How would you parse each of those, without a separator? With "return" as a separator. I said that above. Was I not clear? > One option might be to have a separator that's present only when the > exception type is listed. For instance: > > sum(x[3] except(IndexError) 0 for x in list_of_tuples) > sum(x[3] except 0 for x in list_of_tuples) > > Does that sort of thing have enough support to be added to the PEP? > I'm not sure it gives us anything over the colon notation, which has > the benefit of being consistent with the statement form (and is > stylistically similar to lambda, so it's not grossly inappropriate to > an expression context) I remain less than happy with the colon notation, although I will concede that the basic x[3] except IndexError: 0 form is not horrible - it just becomes horrible very, very fast when people try to do anything more complicated than that. On the other hand, none of the examples of *any* of the proposed syntaxes look compelling to me - they are either too simple to be realistic, or so complex that they should be factored out into multiple statements, in my view. Paul From python at mrabarnett.plus.com Tue Feb 18 17:01:37 2014 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 18 Feb 2014 16:01:37 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> Message-ID: <530383E1.3060803@mrabarnett.plus.com> On 2014-02-18 15:40, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 2:30 AM, Rob Cliffe wrote: >> Sorry, but this seems to me to be a futile discussion. >> Expressions can contain functions. >> Functions can raise any exception they like. >> So evaluating an expression can raise any exception you care to name. > > Of course it could raise anything, but what are you likely to want to > catch and handle by returning a different value? > > LookupError when your cache isn't populated > UnicodeDecodeError to fall back on a different encoding ("try UTF-8; > if that fails, assume the other end is Windows and try CP-1252" - I > have that logic in some places) > ZeroDivisionError and return float("inf"), float("nan"), or 0 > > But you wouldn't want to catch: > > SyntaxError > ImportError > AssertionError > IndentationError > ProgrammerIsAMoronAndHisCodeIsInError > I think that last one should be: ProgrammerIsAMoronAndHisCodeIsInErrorError or just: PebkacError > because most of them wouldn't come up in an expression context, and if > they do, you aren't likely to have a useful alternate value to return. > > So it is conceivable that there be some tuple or superclass of "common > errors worth catching". Trouble is, it still often wouldn't be > appropriate to catch them all - maybe you want to hook into > LookupError, but accidentally catching a UnicodeDecodeError along the > way would mask a bug. > From rosuav at gmail.com Tue Feb 18 17:06:15 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 03:06:15 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: On Wed, Feb 19, 2014 at 2:56 AM, Paul Moore wrote: > On 18 February 2014 15:05, Chris Angelico wrote: >>> Well, yes and no. There are only 2 out of the 10 syntaxes originally >>> proposed in the PEP at the start of this thread that use a colon. I've >>> already pointed out that I don't like a colon, so assume I said >>> "except return 0" if you must :-) >> >> Whether it's a colon or another keyword, _something_ is necessary in >> there, if it's permissible to specify an exception type: >> >> sum(x[3] except IndexError 0 for x in list_of_tuples) >> sum(x[3] except 0 for x in list_of_tuples) >> >> How would you parse each of those, without a separator? > > With "return" as a separator. I said that above. Was I not clear? Sure. Sorry, got a bit of brain-fry at the moment; misinterpreted your last sentence three chevrons in. So yeah, that would be: sum(x[3] except IndexError return 0 for x in list_of_tuples) sum(x[3] except return 0 for x in list_of_tuples) > I remain less than happy with the colon notation, although I will > concede that the basic > > x[3] except IndexError: 0 > > form is not horrible - it just becomes horrible very, very fast when > people try to do anything more complicated than that. On the other > hand, none of the examples of *any* of the proposed syntaxes look > compelling to me - they are either too simple to be realistic, or so > complex that they should be factored out into multiple statements, in > my view. The same can be said for if/else and the short-circuiting uses of and/or, both of which have wide-spread support, and not just from folks like me who grew up with C. I'm currently working on finding a bunch of examples from the stdlib that could be translated. Will post them once I get the script sorted out. As it's currently 3AM, though, that means I need to fry up some bacon or something before I continue :) ChrisA From rosuav at gmail.com Tue Feb 18 17:08:55 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 03:08:55 +1100 Subject: [Python-ideas] except expression In-Reply-To: <530383E1.3060803@mrabarnett.plus.com> References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <530383E1.3060803@mrabarnett.plus.com> Message-ID: On Wed, Feb 19, 2014 at 3:01 AM, MRAB wrote: >> ProgrammerIsAMoronAndHisCodeIsInError >> > I think that last one should be: > > ProgrammerIsAMoronAndHisCodeIsInErrorError > > or just: > > PebkacError The error is that his code is in. If his code had been out, that would have been fine. When this exception is thrown, you need to hit it with your bat, otherwise your code will be out as soon as the exception hits the stumps. ChrisA From python at mrabarnett.plus.com Tue Feb 18 17:09:14 2014 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 18 Feb 2014 16:09:14 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> Message-ID: <530385AA.70703@mrabarnett.plus.com> On 2014-02-18 15:51, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 2:41 AM, Alexander Belopolsky > wrote: >> I would not mind >> >> x = 1/f(y) except 0 >> >> not catching a TypeError that may come from f(y) in most cases. > > But should it catch a KeyError from inside f(y), based on the > translation of d.get()? > The current code would use try...except. Would that catch a KeyError from inside f(y)? Yes. The problem is that sometimes you want to catch only ZeroDivisionError, so we need to be able to specify the exception. The question is whether it should be OK to allow a bare except, where that would catch a limited number of exceptions, in those cases where there won't be any risk. From p.f.moore at gmail.com Tue Feb 18 17:19:30 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 18 Feb 2014 16:19:30 +0000 Subject: [Python-ideas] except expression In-Reply-To: <53037BF8.1050300@btinternet.com> References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <53037BF8.1050300@btinternet.com> Message-ID: On 18 February 2014 15:27, Rob Cliffe wrote: > We must be able to specify the exception(s) to catch. Otherwise the whole > thing is pretty useless: > > MisspeltVariableName / x except "You divided by zero." # No you didn't, you > raised a NameError. Killer argument. You're right, absolutely. Paul From p.f.moore at gmail.com Tue Feb 18 17:25:44 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 18 Feb 2014 16:25:44 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> Message-ID: On 18 February 2014 15:51, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 2:41 AM, Alexander Belopolsky > wrote: >> I would not mind >> >> x = 1/f(y) except 0 >> >> not catching a TypeError that may come from f(y) in most cases. > > But should it catch a KeyError from inside f(y), based on the > translation of d.get()? Explicit is better than implicit - I think this discussion has done its job and established that trying to assume a subset of exceptions to catch isn't going to work. We either allow a bare except to mean "catch all exceptions" (which exactly the same risks and provisos as a bare except *statement*) or we make the exception to catch mandatory. OTOH, there's still an argument for only allowing a single exception name in the syntax (an "identifier" rather than an "expression" in syntax terms). If you must catch multiple exceptions, give the relevant tuple a name. Paul From rosuav at gmail.com Tue Feb 18 17:29:11 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 03:29:11 +1100 Subject: [Python-ideas] except expression In-Reply-To: <530385AA.70703@mrabarnett.plus.com> References: <52FE969D.5000002@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <530385AA.70703@mrabarnett.plus.com> Message-ID: On Wed, Feb 19, 2014 at 3:09 AM, MRAB wrote: > The question is whether it should be OK to allow a bare except, where > that would catch a limited number of exceptions, in those cases where > there won't be any risk. Yeah. I don't like the idea that omitting the exception name(s) would have a completely different effect in expression or statement form. Compare: value = true_value if condition else false_value # <-> if condition: value = true_value else: value = false_value func = lambda x: x+4 # <-> def func(x): return x+4 lst = [x*x for x in range(5)] # <-> lst = [] for x in range(5): lst.append(x*x) Each of them has a "corresponding statement form" that's more-or-less the same in effect (little stuff like name leakage aside), and which is spelled very similarly. You wouldn't expect, for instance, lambda functions to specify their args in reverse order compared to def functions. So if it's syntactically legal to have a bare except in an expression (still under debate), then that bare except should be equivalent to "except BaseException" - not "except Exception", not "except DWIMException", not anything else. Otherwise it'd just cause confusion when trying to decode a complex expression. ChrisA From rosuav at gmail.com Tue Feb 18 17:43:45 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 03:43:45 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> Message-ID: On Wed, Feb 19, 2014 at 3:25 AM, Paul Moore wrote: > Explicit is better than implicit - I think this discussion has done > its job and established that trying to assume a subset of exceptions > to catch isn't going to work. We either allow a bare except to mean > "catch all exceptions" (which exactly the same risks and provisos as a > bare except *statement*) or we make the exception to catch mandatory. Yep, agreed. I'm personally inclined to permit the bare except, and then advise against it in PEP 8, but both halves of that are debatable. I'm of the opinion that this would be risky: lst[x] except: "x is out of bounds" and would prefer the more verbose: lst[x] except KeyError: "x is out of bounds" but I can understand that some people will prefer the shorter form, even though it could mask a typo in either name. It's no different from any other uncatchable error: if html_tag == "": handle_script() Nobody expects the Python inquisition to catch that for them. A bare except lets you sacrifice some quick-error-catching-ness for some quick-code-typing-ness. [1] > OTOH, there's still an argument for only allowing a single exception > name in the syntax (an "identifier" rather than an "expression" in > syntax terms). If you must catch multiple exceptions, give the > relevant tuple a name. Hmm. Would that make anything any clearer? It feels like the sorts of crazy limitations that I've seen in some other languages, like how PHP up until relatively recently wouldn't let you subscript an array returned from a function without first assigning it to a variable: $x = func()[5]; $x = func(); $x = $x[5]; One of the things I like about Python is that anything is itself, regardless of its context. An expression yields a value, that value can be stored, and any expression yielding the same value will be exactly the same thing: func = obj.method func() # <-> obj.method() Contrast JavaScript, where those two are actually different. So in exception catching, *especially* in an expression context (where you can't assign anyway), is it really necessary to dump your two-name tuple out into a temporary name? [1] Now my fingers are wondering: Is there a global-interpreter-loch-ness monster? From tjreedy at udel.edu Tue Feb 18 17:46:19 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 18 Feb 2014 11:46:19 -0500 Subject: [Python-ideas] pickling obscured function references In-Reply-To: References: Message-ID: On 2/18/2014 1:41 AM, Henry Harrison wrote: > I can't be the first to have thought of this, so there must be a reason > this isn't the case, but I have to ask. Why is __main__ the fallback > when pickle can't find a function reference? > > Instead of something like: > os.path.basename(inspect.getsourcefile(func))[:-3] > > Thanks for humoring my curiosity, A place to ask questions and 'humor curiosity' is python-list. Python-ideas is for ideas for improving future version of python. -- Terry Jan Reedy From p.f.moore at gmail.com Tue Feb 18 17:56:41 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 18 Feb 2014 16:56:41 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> Message-ID: On 18 February 2014 16:43, Chris Angelico wrote: >> OTOH, there's still an argument for only allowing a single exception >> name in the syntax (an "identifier" rather than an "expression" in >> syntax terms). If you must catch multiple exceptions, give the >> relevant tuple a name. > > Hmm. Would that make anything any clearer? It feels like the sorts of > crazy limitations that I've seen in some other languages, like how PHP > up until relatively recently wouldn't let you subscript an array > returned from a function without first assigning it to a variable: Maybe not. Maybe again it's just a matter of a style recommendation. But the PEP itself has to tread a fine line between showing what is *possible* vs showing what is *intended* - I feel that the intention of the except construct should *not* be to do most of the crazy things people are talking about in this thread. Paul From rosuav at gmail.com Tue Feb 18 17:58:50 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 03:58:50 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> Message-ID: On Wed, Feb 19, 2014 at 3:56 AM, Paul Moore wrote: > On 18 February 2014 16:43, Chris Angelico wrote: >>> OTOH, there's still an argument for only allowing a single exception >>> name in the syntax (an "identifier" rather than an "expression" in >>> syntax terms). If you must catch multiple exceptions, give the >>> relevant tuple a name. >> >> Hmm. Would that make anything any clearer? It feels like the sorts of >> crazy limitations that I've seen in some other languages, like how PHP >> up until relatively recently wouldn't let you subscript an array >> returned from a function without first assigning it to a variable: > > Maybe not. Maybe again it's just a matter of a style recommendation. > But the PEP itself has to tread a fine line between showing what is > *possible* vs showing what is *intended* - I feel that the intention > of the except construct should *not* be to do most of the crazy things > people are talking about in this thread. That's fair. It's tricky to show that this really would be an improvement to the language when it's showing very little, but tricky to show that it'd be an improvement when the expressions are horrendously obfuscated. I expect that this would be used mostly in really really simple ways. ChrisA From alexander.belopolsky at gmail.com Tue Feb 18 18:01:29 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 18 Feb 2014 12:01:29 -0500 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> Message-ID: On Tue, Feb 18, 2014 at 11:25 AM, Paul Moore wrote: > On 18 February 2014 15:51, Chris Angelico wrote: > > On Wed, Feb 19, 2014 at 2:41 AM, Alexander Belopolsky > > wrote: > >> I would not mind > >> > >> x = 1/f(y) except 0 > >> > >> not catching a TypeError that may come from f(y) in most cases. > > > > But should it catch a KeyError from inside f(y), based on the > > translation of d.get()? > > Explicit is better than implicit - I think this discussion has done > its job and established that trying to assume a subset of exceptions > to catch isn't going to work. We either allow a bare except to mean > "catch all exceptions" (which exactly the same risks and provisos as a > bare except *statement*) or we make the exception to catch mandatory. I disagree. It is best to leave explicit selection of exceptions to catch outside of the expressions. This is job for things like decimal or numpy fp contexts. Always requiring a long camel-case name inside an expression will kill most of the advantages. For example, it would be nice to be able to write something like x = d1[key] except d2[key] except 0 but sprinkle this with repeated KeyError and what is important (d1 and d2) will be lost in the scaffolding. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemiant at hotmail.com Tue Feb 18 18:07:20 2014 From: lemiant at hotmail.com (Alex Rodrigues) Date: Tue, 18 Feb 2014 10:07:20 -0700 Subject: [Python-ideas] Inline Function - idea Message-ID: > Alex, can you explain the difference (if any) between your proposal and dynamic scoping? > > -- Steven Inline functions would be a bit like dynamic scoping and a bit like macros. The main difference from dynamic scoping is that they would not search up beyond their parent to find names, since they act exactly like code injected at the spot they are called it is expected that the variables they are using (like the ones in it's parent) are either locals or globals. I'm not sure that that has many advantages outside of demanding less from the runtime, but that's how I imagined it. IMO The biggest advantage to inline functions over other constructs is ease of understanding. We may you be used to understanding scoping after lots of programming, but it's not always intuitive. On the other hand it is extremely intuitive to understand "when you call this, all that code runs exactly as if you had typed it here". - Alex From abarnert at yahoo.com Tue Feb 18 18:45:22 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 18 Feb 2014 09:45:22 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> Message-ID: On Feb 18, 2014, at 7:26, Alexander Belopolsky wrote: > On Tue, Feb 18, 2014 at 10:13 AM, MRAB wrote: > > > > Another possibility would be to say that a bare except in an expression > > catches only certain "expression-oriented" exceptions, e.g. ValueError. > > The simplest way to do that would be to make them subclasses of an > > ExpressionError class. > > > +1 > > > > > The question then becomes one of which > > exceptions are "expression-oriented"... > > Here is my attempt to answer: > > - ArithmeticError > - AttributeError > - LookupError > - TypeError ? > - ValueError ? And Chris Angelico gave an overlapping but different list by scanning the built-in exception types. The fact that neither of them mentioned KeyError or IndexError, which are the two paradigm cases that brought up the proposal in the first place, implies to me that any attempt to define a collection of "expression errors" is going to be as hard and as contentious as coming up with a syntax. So, if the point of it was to allow us to come up with a new syntax that only handles one exception type, I don't think it's a promising avenue. If people do want to follow this, one thing to consider: we don't have to fiddle with the exception hierarchy; you can always override issubclass/isinstance by either using an ABC, or doing the same thing ABCMeta does. That would allow someone who wanted to catch some type of error that isn't usually considered an "expression error" but is commonly raised in expressions in their particular project to add that error. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Tue Feb 18 18:49:46 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 04:49:46 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> Message-ID: On Wed, Feb 19, 2014 at 4:45 AM, Andrew Barnert wrote: > The fact that neither of them mentioned KeyError or IndexError, which are > the two paradigm cases that brought up the proposal in the first place, > implies to me that any attempt to define a collection of "expression errors" > is going to be as hard and as contentious as coming up with a syntax. Both of us mentioned LookupError, which is superclass to both KeyError and IndexError, and so catches both of them. ChrisA From rosuav at gmail.com Tue Feb 18 18:51:22 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 04:51:22 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> Message-ID: On Wed, Feb 19, 2014 at 4:45 AM, Andrew Barnert wrote: > If people do want to follow this, one thing to consider: we don't have to > fiddle with the exception hierarchy; you can always override > issubclass/isinstance by either using an > ABC, or doing the same thing ABCMeta does. That would allow someone who > wanted to catch some type of error that isn't usually considered an > "expression error" but is commonly raised in expressions in their particular > project to add that error. Actually, I'm not sure that you can. Give it a try! The only thing you can try to catch is a subclass of BaseException. But creating a tuple would work. ChrisA From abarnert at yahoo.com Tue Feb 18 18:54:06 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 18 Feb 2014 09:54:06 -0800 Subject: [Python-ideas] pickling obscured function references In-Reply-To: References: Message-ID: <034F7C61-1F42-45BE-99C3-FE8809A85227@yahoo.com> On Feb 18, 2014, at 8:46, Terry Reedy wrote: > On 2/18/2014 1:41 AM, Henry Harrison wrote: > >> I can't be the first to have thought of this, so there must be a reason >> this isn't the case, but I have to ask. Why is __main__ the fallback >> when pickle can't find a function reference? >> >> Instead of something like: >> os.path.basename(inspect.getsourcefile(func))[:-3] >> >> Thanks for humoring my curiosity, > > A place to ask questions and 'humor curiosity' is python-list. Python-ideas is for ideas for improving future version of python. I think he may have actually wanted to propose this change, but posted it in an overly tentative way so that if it was a bad idea (as, after all, most language change proposals are...), people would explain gently why it's a bad idea instead of just rejecting it out of hand. (I don't think it's a good idea to post that way on this list--in fact, it's likely to have the opposite of the intended effect--but I can understand why someone might expect it to be.) From abarnert at yahoo.com Tue Feb 18 19:09:00 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 18 Feb 2014 10:09:00 -0800 Subject: [Python-ideas] Inline Function - idea In-Reply-To: References: Message-ID: <6B42AC9E-0032-46E4-A0CD-89A108B0AE3A@yahoo.com> On Feb 18, 2014, at 9:07, Alex Rodrigues wrote: >> Alex, can you explain the difference (if any) between your proposal and dynamic scoping? >> >> -- Steven > > Inline functions would be a bit like dynamic scoping and a bit like macros. The main difference from dynamic scoping is that they would not search up beyond their parent to find names, since they act exactly like code injected at the spot they are called it is expected that the variables they are using (like the ones in it's parent) are either locals or globals. I don't see how that's any different from a macro. If you're just proposing a limited form of macro that works entirely at the semantic or computational level (so it can't be used to add new syntax or do many of the other tricks that PyMacro, lisp macros, etc. do), that seems reasonable. But I don't think insisting that it's really a function, not a macro, makes it better. > I'm not sure that that has many advantages outside of demanding less from the runtime, but that's how I imagined it. If I remember correctly, in your proposal you wanted to be able to create these things at runtime, and convert regular functions into inline functions on the fly. If so, I think you're demanding a lot _more_ from the runtime in the service of trying to demand less. Is that a key part of your proposal? If it's _not_ a key part of the idea, you really should play with using PyMacro (limiting yourself to pure "function-like" usage) and see if it gives you everything you need. If so, you'll have some solid examples of what this can do, why it works, how it's safer than a more general macro system like PyMacro, etc. > IMO The biggest advantage to inline functions over other constructs is ease of understanding. We may you be used to understanding scoping after lots of programming, but it's not always intuitive. On the other hand it is extremely intuitive to understand "when you call this, all that code runs exactly as if you had typed it here". I'm not sure that it is. And it definitely gets in the way of reading code later on. That's exactly why C++ and C added inline functions, which do _not_ use the calling or defining scope, to replace function-like macros. Stroustrop even said that if you find it necessary to #define a macro, you've found a bug either in your code or in the language. I assume he later backed down from that assertion a bit, but the basic sentiment is shared by most of the community. It's also why later Lisp-family languages added hygienic macros either in place of or alongside nonhygienic macros. Capturing and modifying variables from a different scope is often useful, but it's even more often a mistake. That's why Python, Ruby, C++, and many other languages require you to make it explicit (all in different ways) even for simple closures over directly visible variables. From abarnert at yahoo.com Tue Feb 18 19:11:59 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 18 Feb 2014 10:11:59 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> Message-ID: <6DFDC144-A1CC-43B6-A751-7DCA699B917C@yahoo.com> On Feb 18, 2014, at 9:51, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 4:45 AM, Andrew Barnert wrote: >> If people do want to follow this, one thing to consider: we don't have to >> fiddle with the exception hierarchy; you can always override >> issubclass/isinstance by either using an >> ABC, or doing the same thing ABCMeta does. That would allow someone who >> wanted to catch some type of error that isn't usually considered an >> "expression error" but is commonly raised in expressions in their particular >> project to add that error. > > Actually, I'm not sure that you can. Give it a try! The only thing you > can try to catch is a subclass of BaseException. But creating a tuple > would work. But if I define two new BaseException subclasses, and register one as a subclass of the other after the fact, I believe it works. (I can't test until I get in front of a computer with Python 3 on it.) That's what I was suggesting--being able to add thirdpartymodule.error to the expression errors, not being able to add, say, deque. From abarnert at yahoo.com Tue Feb 18 19:13:26 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 18 Feb 2014 10:13:26 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> Message-ID: On Feb 18, 2014, at 9:49, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 4:45 AM, Andrew Barnert wrote: >> The fact that neither of them mentioned KeyError or IndexError, which are >> the two paradigm cases that brought up the proposal in the first place, >> implies to me that any attempt to define a collection of "expression errors" >> is going to be as hard and as contentious as coming up with a syntax. > > Both of us mentioned LookupError, which is superclass to both KeyError > and IndexError, and so catches both of them. Sorry, I didn't see LookupError in your list. But it's clearly there. NotEnoughCoffeeError. From rosuav at gmail.com Tue Feb 18 19:21:21 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 05:21:21 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> Message-ID: On Wed, Feb 19, 2014 at 5:13 AM, Andrew Barnert wrote: > On Feb 18, 2014, at 9:49, Chris Angelico wrote: > >> On Wed, Feb 19, 2014 at 4:45 AM, Andrew Barnert wrote: >>> The fact that neither of them mentioned KeyError or IndexError, which are >>> the two paradigm cases that brought up the proposal in the first place, >>> implies to me that any attempt to define a collection of "expression errors" >>> is going to be as hard and as contentious as coming up with a syntax. >> >> Both of us mentioned LookupError, which is superclass to both KeyError >> and IndexError, and so catches both of them. > > Sorry, I didn't see LookupError in your list. But it's clearly there. NotEnoughCoffeeError. Though you do (effectively) raise another objection to that proposal, namely that programmers will never be quite sure what's caught and what's not. It'll become magic. Can you, without looking anything up, name all the exception types that would be caught by (LookupError, UnicodeError, OSError) ? And that's only part of my list. ChrisA From haoyi.sg at gmail.com Tue Feb 18 19:27:38 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Tue, 18 Feb 2014 10:27:38 -0800 Subject: [Python-ideas] Inline Function - idea In-Reply-To: <6B42AC9E-0032-46E4-A0CD-89A108B0AE3A@yahoo.com> References: <6B42AC9E-0032-46E4-A0CD-89A108B0AE3A@yahoo.com> Message-ID: > I'm not sure that that has many advantages outside of demanding less from the runtime MacroPy doesn't demand anything from the runtime at all. *pip install macropy *and you're off to the races. I'm not sure how you can get any less demanding than that. If you really want inline functions as you described, just write a macro using an unhygienic quasiquote: *@macros.expr()* *def func(tree):* * return q[log(ast[tree], "i am a cow")]* This will inline the code *log(..., "i am a cow") *into anywhere you call *func(...)*. What you're asking for is already here, not tomorrow but today: you don't need to persuade anyone that it increases ease of understanding, or that it's a good idea, or write a PEP. If you think it'll be worth someone else's effort to fork the python interpreter, trying out the idea using MacroPy isn't a lot to do in advance. On Tue, Feb 18, 2014 at 10:09 AM, Andrew Barnert wrote: > On Feb 18, 2014, at 9:07, Alex Rodrigues wrote: > > >> Alex, can you explain the difference (if any) between your proposal and > dynamic scoping? > >> > >> -- Steven > > > > Inline functions would be a bit like dynamic scoping and a bit like > macros. The main difference from dynamic scoping is that they would not > search up beyond their parent to find names, since they act exactly like > code injected at the spot they are called it is expected that the variables > they are using (like the ones in it's parent) are either locals or globals. > > I don't see how that's any different from a macro. > > If you're just proposing a limited form of macro that works entirely at > the semantic or computational level (so it can't be used to add new syntax > or do many of the other tricks that PyMacro, lisp macros, etc. do), that > seems reasonable. But I don't think insisting that it's really a function, > not a macro, makes it better. > > > I'm not sure that that has many advantages outside of demanding less > from the runtime, but that's how I imagined it. > > If I remember correctly, in your proposal you wanted to be able to create > these things at runtime, and convert regular functions into inline > functions on the fly. If so, I think you're demanding a lot _more_ from the > runtime in the service of trying to demand less. Is that a key part of your > proposal? > > If it's _not_ a key part of the idea, you really should play with using > PyMacro (limiting yourself to pure "function-like" usage) and see if it > gives you everything you need. If so, you'll have some solid examples of > what this can do, why it works, how it's safer than a more general macro > system like PyMacro, etc. > > > IMO The biggest advantage to inline functions over other constructs is > ease of understanding. We may you be used to understanding scoping after > lots of programming, but it's not always intuitive. On the other hand it is > extremely intuitive to understand "when you call this, all that code runs > exactly as if you had typed it here". > > I'm not sure that it is. And it definitely gets in the way of reading code > later on. That's exactly why C++ and C added inline functions, which do > _not_ use the calling or defining scope, to replace function-like macros. > Stroustrop even said that if you find it necessary to #define a macro, > you've found a bug either in your code or in the language. I assume he > later backed down from that assertion a bit, but the basic sentiment is > shared by most of the community. > > It's also why later Lisp-family languages added hygienic macros either in > place of or alongside nonhygienic macros. > > Capturing and modifying variables from a different scope is often useful, > but it's even more often a mistake. That's why Python, Ruby, C++, and many > other languages require you to make it explicit (all in different ways) > even for simple closures over directly visible variables. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rob.cliffe at btinternet.com Tue Feb 18 19:30:21 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Tue, 18 Feb 2014 18:30:21 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: <5303A6BD.8030406@btinternet.com> On 18/02/2014 15:56, Paul Moore wrote: > On 18 February 2014 15:05, Chris Angelico wrote: >>> Well, yes and no. There are only 2 out of the 10 syntaxes originally >>> proposed in the PEP at the start of this thread that use a colon. I've >>> already pointed out that I don't like a colon, so assume I said >>> "except return 0" if you must :-) >> Whether it's a colon or another keyword, _something_ is necessary in >> there, if it's permissible to specify an exception type: >> >> sum(x[3] except IndexError 0 for x in list_of_tuples) >> sum(x[3] except 0 for x in list_of_tuples) >> >> How would you parse each of those, without a separator? > With "return" as a separator. I said that above. Was I not clear? > >> One option might be to have a separator that's present only when the >> exception type is listed. For instance: >> >> sum(x[3] except(IndexError) 0 for x in list_of_tuples) >> sum(x[3] except 0 for x in list_of_tuples) >> >> Does that sort of thing have enough support to be added to the PEP? >> I'm not sure it gives us anything over the colon notation, which has >> the benefit of being consistent with the statement form (and is >> stylistically similar to lambda, so it's not grossly inappropriate to >> an expression context) > I remain less than happy with the colon notation, although I will > concede that the basic > > x[3] except IndexError: 0 > > form is not horrible - it just becomes horrible very, very fast when > people try to do anything more complicated than that. I guess this is a subjective thing, no point in shouting too much about it. But given that this notation is (about) the most compact for its generality, wouldn't any alternative become horrible even faster? Rob Cliffe > On the other > hand, none of the examples of *any* of the proposed syntaxes look > compelling to me - they are either too simple to be realistic, or so > complex that they should be factored out into multiple statements, in > my view. > > Paul > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > ----- > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6602 - Release Date: 02/17/14 > > From rob.cliffe at btinternet.com Tue Feb 18 19:33:27 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Tue, 18 Feb 2014 18:33:27 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> Message-ID: <5303A777.3000306@btinternet.com> On 18/02/2014 16:43, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 3:25 AM, Paul Moore wrote: >> Explicit is better than implicit - I think this discussion has done >> its job and established that trying to assume a subset of exceptions >> to catch isn't going to work. We either allow a bare except to mean >> "catch all exceptions" (which exactly the same risks and provisos as a >> bare except *statement*) or we make the exception to catch mandatory. > Yep, agreed. I'm personally inclined to permit the bare except, and > then advise against it in PEP 8, but both halves of that are > debatable. I'm of the opinion that this would be risky: > > lst[x] except: "x is out of bounds" > > and would prefer the more verbose: > > lst[x] except KeyError: "x is out of bounds" > > but I can understand that some people will prefer the shorter form, > even though it could mask a typo in either name. It's no different > from any other uncatchable error: > > if html_tag == "": handle_script() > > Nobody expects the Python inquisition to catch that for them. A bare > except lets you sacrifice some quick-error-catching-ness for some > quick-code-typing-ness. [1] > >> OTOH, there's still an argument for only allowing a single exception >> name in the syntax (an "identifier" rather than an "expression" in >> syntax terms). If you must catch multiple exceptions, give the >> relevant tuple a name. > Hmm. Would that make anything any clearer? It feels like the sorts of > crazy limitations that I've seen in some other languages, like how PHP > up until relatively recently wouldn't let you subscript an array > returned from a function without first assigning it to a variable: > > $x = func()[5]; > > $x = func(); > $x = $x[5]; > > One of the things I like about Python is that anything is itself, > regardless of its context. An expression yields a value, that value > can be stored, and any expression yielding the same value will be > exactly the same thing: > > func = obj.method > func() > # <-> > obj.method() As anyone who has followed my contributions can probably guess, you're preaching to the converted here. Rob Cliffe > > Contrast JavaScript, where those two are actually different. > > So in exception catching, *especially* in an expression context (where > you can't assign anyway), is it really necessary to dump your two-name > tuple out into a temporary name? > > [1] Now my fingers are wondering: Is there a > global-interpreter-loch-ness monster? > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > ----- > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6602 - Release Date: 02/17/14 > > From rob.cliffe at btinternet.com Tue Feb 18 19:34:57 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Tue, 18 Feb 2014 18:34:57 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> Message-ID: <5303A7D1.60105@btinternet.com> On 18/02/2014 16:56, Paul Moore wrote: > On 18 February 2014 16:43, Chris Angelico wrote: >>> OTOH, there's still an argument for only allowing a single exception >>> name in the syntax (an "identifier" rather than an "expression" in >>> syntax terms). If you must catch multiple exceptions, give the >>> relevant tuple a name. >> Hmm. Would that make anything any clearer? It feels like the sorts of >> crazy limitations that I've seen in some other languages, like how PHP >> up until relatively recently wouldn't let you subscript an array >> returned from a function without first assigning it to a variable: > Maybe not. Maybe again it's just a matter of a style recommendation. > But the PEP itself has to tread a fine line between showing what is > *possible* vs showing what is *intended* - I feel that the intention > of the except construct should *not* be to do most of the crazy things > people are talking about in this thread. Almost any language feature can be abused. That doesn't mean we should regulate their use. Rob Cliffe > > Paul > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > ----- > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6602 - Release Date: 02/17/14 > > From p.f.moore at gmail.com Tue Feb 18 19:47:00 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 18 Feb 2014 18:47:00 +0000 Subject: [Python-ideas] except expression In-Reply-To: <5303A6BD.8030406@btinternet.com> References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303A6BD.8030406@btinternet.com> Message-ID: On 18 February 2014 18:30, Rob Cliffe wrote: >> I remain less than happy with the colon notation, although I will >> concede that the basic >> >> x[3] except IndexError: 0 >> >> form is not horrible - it just becomes horrible very, very fast when >> people try to do anything more complicated than that. > > I guess this is a subjective thing, no point in shouting too much about it. > But given that this notation is (about) the most compact for its generality, > wouldn't any alternative become horrible even faster? Not in my view, because I find keywords more readable than punctuation in this case. But yes, it is subjective. Even to the point that my bleary-eyed, pre-cup-of-tea early morning self has stronger views on the matter than I do right now :-) Paul From amber.yust at gmail.com Tue Feb 18 19:47:41 2014 From: amber.yust at gmail.com (Amber Yust) Date: Tue, 18 Feb 2014 18:47:41 +0000 Subject: [Python-ideas] except expression References: <52FE969D.5000002@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <5303A7D1.60105@btinternet.com> Message-ID: -1 to any proposal that a bare except in an expression should catch a different set of exceptions than a bare except in a statement. That's incredibly unintuitive. On Tue Feb 18 2014 at 10:37:24 AM, Rob Cliffe wrote: > > On 18/02/2014 16:56, Paul Moore wrote: > > On 18 February 2014 16:43, Chris Angelico wrote: > >>> OTOH, there's still an argument for only allowing a single exception > >>> name in the syntax (an "identifier" rather than an "expression" in > >>> syntax terms). If you must catch multiple exceptions, give the > >>> relevant tuple a name. > >> Hmm. Would that make anything any clearer? It feels like the sorts of > >> crazy limitations that I've seen in some other languages, like how PHP > >> up until relatively recently wouldn't let you subscript an array > >> returned from a function without first assigning it to a variable: > > Maybe not. Maybe again it's just a matter of a style recommendation. > > But the PEP itself has to tread a fine line between showing what is > > *possible* vs showing what is *intended* - I feel that the intention > > of the except construct should *not* be to do most of the crazy things > > people are talking about in this thread. > Almost any language feature can be abused. That doesn't mean we should > regulate their use. > Rob Cliffe > > > > Paul > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas at python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/ > > > > > > ----- > > No virus found in this message. > > Checked by AVG - www.avg.com > > Version: 2012.0.2247 / Virus Database: 3705/6602 - Release Date: 02/17/14 > > > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From storchaka at gmail.com Tue Feb 18 20:07:07 2014 From: storchaka at gmail.com (Serhiy Storchaka) Date: Tue, 18 Feb 2014 21:07:07 +0200 Subject: [Python-ideas] An Everything singleton Message-ID: This crazy idea is inspired by a discussing in Python-Dev which should be here. Currently we have a None singleton which representing nothing and raises an exception in almost all operations (except equality, etc). What about introducing its antonym, an Everything singleton (the name is discussable), which newer raises an exception in any operation, but returns consistent value? >>> Everything + 2 2 >>> Everything < 3 True >>> Everything * 'foo' 'foo' From amber.yust at gmail.com Tue Feb 18 20:09:45 2014 From: amber.yust at gmail.com (Amber Yust) Date: Tue, 18 Feb 2014 19:09:45 +0000 Subject: [Python-ideas] An Everything singleton References: Message-ID: Your examples do not make sense to me. On Tue Feb 18 2014 at 11:08:00 AM, Serhiy Storchaka wrote: > This crazy idea is inspired by a discussing in Python-Dev which should > be here. > > Currently we have a None singleton which representing nothing and raises > an exception in almost all operations (except equality, etc). What about > introducing its antonym, an Everything singleton (the name is > discussable), which newer raises an exception in any operation, but > returns consistent value? > > >>> Everything + 2 > 2 > >>> Everything < 3 > True > >>> Everything * 'foo' > 'foo' > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Tue Feb 18 20:27:05 2014 From: guido at python.org (Guido van Rossum) Date: Tue, 18 Feb 2014 11:27:05 -0800 Subject: [Python-ideas] An Everything singleton In-Reply-To: References: Message-ID: This idea comes up occasionally (though I've never heard it named "everything" before). I believe it is similar to a feature in Objective-C (or maybe just Apple's version) where the behavior is that if you send any message to nil it does nothing and returns nil. In Python I think it would be very disturbing though and would end up masking errors. (Python and ObjC seems to have very different attitudes about exceptions -- in Python they are used all the time while in ObjC the culture seems to favor the idea that exceptions are fatal and shouldn't even be caught.) On Tue, Feb 18, 2014 at 11:09 AM, Amber Yust wrote: > Your examples do not make sense to me. > > > On Tue Feb 18 2014 at 11:08:00 AM, Serhiy Storchaka > wrote: > >> This crazy idea is inspired by a discussing in Python-Dev which should >> be here. >> >> Currently we have a None singleton which representing nothing and raises >> an exception in almost all operations (except equality, etc). What about >> introducing its antonym, an Everything singleton (the name is >> discussable), which newer raises an exception in any operation, but >> returns consistent value? >> >> >>> Everything + 2 >> 2 >> >>> Everything < 3 >> True >> >>> Everything * 'foo' >> 'foo' >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From henry.schafer.harrison at gmail.com Tue Feb 18 20:35:27 2014 From: henry.schafer.harrison at gmail.com (Henry Harrison) Date: Tue, 18 Feb 2014 14:35:27 -0500 Subject: [Python-ideas] pickling obscured function references Message-ID: Apologies. Yes, I had in mind something in-between a proposal and a question, which in retrospect was not the way to go about it. And I was not aware of python-list, so apologies again for not researching the options for having this discussion. In any case after sleeping on it I'm pretty sure the answer to my question is "because backwards compatibility" which means that the proposal would be moot. Sorry again for wasting your time. - Henry > On Feb 18, 2014, at 8:46, Terry Reedy wrote: > > > On 2/18/2014 1:41 AM, Henry Harrison wrote: > > > >> I can't be the first to have thought of this, so there must be a reason > >> this isn't the case, but I have to ask. Why is __main__ the fallback > >> when pickle can't find a function reference? > >> > >> Instead of something like: > >> os.path.basename(inspect.getsourcefile(func))[:-3] > >> > >> Thanks for humoring my curiosity, > > > > A place to ask questions and 'humor curiosity' is python-list. > Python-ideas is for ideas for improving future version of python. > > I think he may have actually wanted to propose this change, but posted it > in an overly tentative way so that if it was a bad idea (as, after all, > most language change proposals are...), people would explain gently why > it's a bad idea instead of just rejecting it out of hand. (I don't think > it's a good idea to post that way on this list--in fact, it's likely to > have the opposite of the intended effect--but I can understand why someone > might expect it to be.) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Tue Feb 18 20:39:59 2014 From: guido at python.org (Guido van Rossum) Date: Tue, 18 Feb 2014 11:39:59 -0800 Subject: [Python-ideas] pickling obscured function references In-Reply-To: References: Message-ID: Actually there's more to it -- the source filename of a function may not always be available, and whether it's available may depend on how it was loaded. It's also a burden for Python implementations other than Python, which may make such introspections much more expensive or at times impossible. On Tue, Feb 18, 2014 at 11:35 AM, Henry Harrison < henry.schafer.harrison at gmail.com> wrote: > Apologies. Yes, I had in mind something in-between a proposal and a > question, which in retrospect was not the way to go about it. And I was not > aware of python-list, so apologies again for not researching the options > for having this discussion. > > In any case after sleeping on it I'm pretty sure the answer to my question > is "because backwards compatibility" which means that the proposal would be > moot. Sorry again for wasting your time. > > - Henry > > > >> On Feb 18, 2014, at 8:46, Terry Reedy wrote: >> >> > On 2/18/2014 1:41 AM, Henry Harrison wrote: >> > >> >> I can't be the first to have thought of this, so there must be a reason >> >> this isn't the case, but I have to ask. Why is __main__ the fallback >> >> when pickle can't find a function reference? >> >> >> >> Instead of something like: >> >> os.path.basename(inspect.getsourcefile(func))[:-3] >> >> >> >> Thanks for humoring my curiosity, >> > >> > A place to ask questions and 'humor curiosity' is python-list. >> Python-ideas is for ideas for improving future version of python. >> >> I think he may have actually wanted to propose this change, but posted it >> in an overly tentative way so that if it was a bad idea (as, after all, >> most language change proposals are...), people would explain gently why >> it's a bad idea instead of just rejecting it out of hand. (I don't think >> it's a good idea to post that way on this list--in fact, it's likely to >> have the opposite of the intended effect--but I can understand why someone >> might expect it to be.) >> > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Tue Feb 18 21:55:56 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 07:55:56 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: On Wed, Feb 19, 2014 at 3:06 AM, Chris Angelico wrote: > I'm currently working on finding a bunch of examples from the stdlib > that could be translated. Will post them once I get the script sorted > out. As it's currently 3AM, though, that means I need to fry up some > bacon or something before I continue :) Alright, results are in. Script: https://github.com/Rosuav/ExceptExpr/blob/master/find_except_expr.py Output: https://github.com/Rosuav/ExceptExpr/blob/master/candidates.txt Annotated examples: https://github.com/Rosuav/ExceptExpr/blob/master/examples.py The last one is the most readable. I've collected up a bunch of viable candidates for translation. (Note that I'm not advocating going through and editing these files for no other reason. That'd just be code churn. But these are cases where, had this feature existed when that code was written, it could have been taken advantage of.) My script is currently _very_ simplistic. It looks *only* for assignments, so it won't find something like this: try: foo.append(args[0]) except IndexError: foo.append('') which is correct, because it's impossible to know whether foo.append() will raise IndexError. (Chances are it won't, especially if we know foo is a list, for instance.) It's not safe to directly translate that sort of thing. It might, however, be something worth improving, as it narrows the scope of the except clause. But if that same code is written thus: try: tmp = args[0] except IndexError: tmp = '' foo.append(tmp) then the script _will_ find it, and then it really is a candidate for editing. Point to note: Apart from one instance, where it wasn't being used anyway, I found not a single instance of 'as' being used. There was one case where sys.exc_info() was referenced, though, so this may just mean that the stdlib files in question are maintaining compatibility with old versions of Python. I didn't look at most of the tests. The script found 195 plausible try/except blocks, of which 37 have the word "test" in the name;that leaves 158 that are likely to benefit from this change. There are a couple of false positives, but not many. Next is to figure out what else is a candidate for editing. Here's my current criteria, straight from the docstring: """Report on 'simple' try/except statements. The definition of simple is: 1. No else or finally clause 2. Only one except clause (currently) 3. Exactly one statement in the try clause 4. Exactly one statement in each except clause 5. Each of those statements is the same type. 6. That type is one that could be an expression. 7. Those statements are all compatible. The last two are the trickiest. Currently I'm looking only for assignments, where both try and except assign to the same target. This is, however, too narrow.""" Interestingly, removing criterion 2 gives us three additional examples out of the test suite, and nothing else. There are no cases outside of the test suite that look like this: try: x = some_calculation except ValueError: x = something_else except OverflowError: x = different_thing (The test suite has one case with those exact two exceptions, and a pair that use OverflowError and ZeroDivisionError. In each case, they're comparing two actions to make sure they give the same result, where "throwing OverflowError" is a result like any other.) Conclusions: The need for chained exception catching might not be so great after all, and even the 'as' keyword isn't as (heh heh) important as I thought it was. Alternate conclusion: My sample is too small. Need more. Data, you have the helm. ChrisA From bruce at leapyear.org Tue Feb 18 22:00:49 2014 From: bruce at leapyear.org (Bruce Leban) Date: Tue, 18 Feb 2014 13:00:49 -0800 Subject: [Python-ideas] An Everything singleton In-Reply-To: References: Message-ID: As Serhiy notes, Python's None value fails when trying to use it. In contrast, SQL's NULL does not fail and instead the NULL propagates. Note that this is not what Serhiy actually proposed. I can see that he wants Everything to be essentially an empty value for + and * but no idea why Everything < 3 is True. Unless of course he meant Everything <3 (the emoticon). :-) None + 3 => exception NULL + 3 => NULL In some sense, None is related to BOTTOM (?) and NULL is related to TOP (?). While I've sometimes wanted a None-like object that would not raise exceptions when operated on, I've usually thought it better to create an object that ignored specific expected operations not everything. Is there a use case or is this just an abstract idea? --- Bruce Learn how hackers think: http://j.mp/gruyere-security On Tue, Feb 18, 2014 at 11:27 AM, Guido van Rossum wrote: > This idea comes up occasionally (though I've never heard it named > "everything" before). I believe it is similar to a feature in Objective-C > (or maybe just Apple's version) where the behavior is that if you send any > message to nil it does nothing and returns nil. > > In Python I think it would be very disturbing though and would end up > masking errors. (Python and ObjC seems to have very different attitudes > about exceptions -- in Python they are used all the time while in ObjC the > culture seems to favor the idea that exceptions are fatal and shouldn't > even be caught.) > > > On Tue, Feb 18, 2014 at 11:09 AM, Amber Yust wrote: > >> Your examples do not make sense to me. >> >> >> On Tue Feb 18 2014 at 11:08:00 AM, Serhiy Storchaka >> wrote: >> >>> This crazy idea is inspired by a discussing in Python-Dev which should >>> be here. >>> >>> Currently we have a None singleton which representing nothing and raises >>> an exception in almost all operations (except equality, etc). What about >>> introducing its antonym, an Everything singleton (the name is >>> discussable), which newer raises an exception in any operation, but >>> returns consistent value? >>> >>> >>> Everything + 2 >>> 2 >>> >>> Everything < 3 >>> True >>> >>> Everything * 'foo' >>> 'foo' >>> >>> _______________________________________________ >>> Python-ideas mailing list >>> Python-ideas at python.org >>> https://mail.python.org/mailman/listinfo/python-ideas >>> Code of Conduct: http://python.org/psf/codeofconduct/ >>> >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > > -- > --Guido van Rossum (python.org/~guido) > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Tue Feb 18 22:20:04 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 18 Feb 2014 21:20:04 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: On 18 February 2014 20:55, Chris Angelico wrote: > Alright, results are in. > > Script: > https://github.com/Rosuav/ExceptExpr/blob/master/find_except_expr.py > Output: > https://github.com/Rosuav/ExceptExpr/blob/master/candidates.txt > Annotated examples: > https://github.com/Rosuav/ExceptExpr/blob/master/examples.py Great work! Looking at the annotated examples, my conclusions would be: 1. The "expr except Exception: default" syntax is actually a lot more readable in context than I expected. 2. But sometimes in more complex cases the similarity with the statement form hurts readability. 3. The win is noticeable on the one-line assignments 4. You had a few cases where you could have (should have?) translated to 3-arg getattr or similar. I'm not quite sure what that says ("if you have a hammer everything looks like a nail"?) 5. The longer examples typically look more confusing to me than the originals. 6. As you noted, nothing much used multiple exceptions or as. I think this implies they are of marginal benefit at best (but I'm guessing just as much as you). Based on this sample, my main worry with the new syntax is that people will over-use it. Up to line 125 of the examples I'd say about 50% seem like wins. After that point the only ones I like are Lib/tarfile.py and Tools/unicode/comparecodecs.py. Overall, I think this is a reasonable case for the PEP, but it's not overwhelming. Which I guess about mirrors my gut feeling for how useful the new construct would be. Paul From greg.ewing at canterbury.ac.nz Tue Feb 18 22:38:31 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 19 Feb 2014 10:38:31 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: <5303D2D7.1090400@canterbury.ac.nz> Chris Angelico wrote: > I don't want to narrow this down to always having to catch everything. Agreed, that would be unacceptably bad. What *might* be useful is to allow the exception type to be omitted, and have it default to LookupError. Then the most common anticipated use cases would be very concise: things[i] except: default_value -- Greg From rob.cliffe at btinternet.com Tue Feb 18 22:39:02 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Tue, 18 Feb 2014 21:39:02 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> Message-ID: <5303D2F6.8040807@btinternet.com> On 18/02/2014 17:01, Alexander Belopolsky wrote: > > On Tue, Feb 18, 2014 at 11:25 AM, Paul Moore > wrote: > > On 18 February 2014 15:51, Chris Angelico > wrote: > > On Wed, Feb 19, 2014 at 2:41 AM, Alexander Belopolsky > > > wrote: > >> I would not mind > >> > >> x = 1/f(y) except 0 > >> > >> not catching a TypeError that may come from f(y) in most cases. > > > > But should it catch a KeyError from inside f(y), based on the > > translation of d.get()? > > Explicit is better than implicit - I think this discussion has done > its job and established that trying to assume a subset of exceptions > to catch isn't going to work. We either allow a bare except to mean > "catch all exceptions" (which exactly the same risks and provisos as a > bare except *statement*) or we make the exception to catch mandatory. > > > I disagree. It is best to leave explicit selection of exceptions to > catch outside of the expressions. This is job for things like decimal > or numpy fp contexts. > > Always requiring a long camel-case name inside an expression will kill > most of the advantages. > > For example, it would be nice to be able to write something like > > x = d1[key] except d2[key] except 0 > > but sprinkle this with repeated KeyError and what is important (d1 and > d2) will be lost in the scaffolding. Hm. Currently you can write x = d1.get(key, d2.get(key, 0)) which is admirably concise, and perhaps you like it. Personally I don't find it particularly readable (and it is one of the API complications this proposal aims to make unnecessary). If we move on from that, this proposal means that, even with sprinkling keywords you can replace: try: x = d1[key] except KeyError: try: x = d2[key] except KeyError: x = 0 with: x = d1[key] except KeyError: (d2[key] except KeyError: 0) Again this is subjective, but I find that a significant gain. 7 lines replaced by 1 not terribly long line. Not many keystrokes saved, true. But no indents to type, and no visual jumping around the indents when reading it. The new version is dense, yes, but I find it readable (Westerners find it natural to read left-to-right), explicit, and with every bit of it except the colons meaningful. Perhaps to be fair the short variable names "d1" and "d2" are a bit swamped by the longer exception name "KeyError". To digress somewhat: I typed in the parentheses to convey my intention without analysing whether they were necessary. I don't think they are because, as far as I can see, the alternative reading x = (d1[key] except KeyError: d2[key]) except KeyError: 0 would have exactly the same semantics, and still would do if "d1" and "d2" were replaced by more complicated expressions which themselves might raise a KeyError (which I find kind of reassuring, it means we don't have to decide a precedence of one "except" over another). (Oops, now I seem to be almost arguing for the "d1.get(..." form which would propagate errors evaluating "d1" or "d2".) But if I am missing something, I'd be interested. Rob Cliffe > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6602 - Release Date: 02/17/14 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Tue Feb 18 22:39:48 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 08:39:48 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: On Wed, Feb 19, 2014 at 8:20 AM, Paul Moore wrote: > Great work! > > Looking at the annotated examples, my conclusions would be: > > 1. The "expr except Exception: default" syntax is actually a lot more > readable in context than I expected. > 2. But sometimes in more complex cases the similarity with the > statement form hurts readability. Interesting. I'm not sure that the similarity is itself hurting readability. I would agree, though, that several of the examples I gave should *not* be translated, that it's a judgment call. Especially where I said "Could become" rather than "Becomes", I'm dubious about the value of translation. > 3. The win is noticeable on the one-line assignments Absolutely. Converting eight lines with zig-zag indentation into two lines, still not exceeding 80 chars, is a huge win. I'm really liking this one: g = grp.getgrnam(tarinfo.gname)[2] except KeyError: tarinfo.gid u = pwd.getpwnam(tarinfo.uname)[2] except KeyError: tarinfo.uid I've had code exactly like this, where there are 4-8 parallel differences all down the line (in this case, that's g/u, grp/pwd, getgrnam/getpwnam, gname/uname, gid/uid - fiveof them). Having the two lines perfectly aligned up and down makes it really easy to see (a) the commonality, and (b) the differences. In the original, would you notice if one of the lines had the wrong variable name, for instance - if groups were being looked up with tarinfo.uname? You might not even notice for a long time, if your group and user names are the same. But laid out like this, it's much easier to see. > 4. You had a few cases where you could have (should have?) translated > to 3-arg getattr or similar. I'm not quite sure what that says ("if > you have a hammer everything looks like a nail"?) In examples.py? The code there is generally meant to be a direct translation of the try/except. Maybe there's an even better way to do some of it, but that's not the point of those snippets. (Though, if there's any that really definitely should be done differently, I'll just drop them, as they're not useful examples.) > 5. The longer examples typically look more confusing to me than the originals. Yeah, which is why I shoved those down to the bottom. They're not really there to support the proposal, they're just showing what it would look like if you did do it that way. Some of them might be deemed better, others will be deemed worse; definitely they don't justify adding syntax to the language. The value of this is the shorter ones. > 6. As you noted, nothing much used multiple exceptions or as. I think > this implies they are of marginal benefit at best (but I'm guessing > just as much as you). This could be a hugely biased sample. Does anyone want to grab that script and run it across a Py3-only codebase? I suspect quite a lot of the Python stdlib is kept Py2-compatible, if only because stuff hasn't needed to be edited since Py2 compat was important. That would explain the use of sys.exc_info rather than as. > Based on this sample, my main worry with the new syntax is that people > will over-use it. Up to line 125 of the examples I'd say about 50% > seem like wins. After that point the only ones I like are > Lib/tarfile.py and Tools/unicode/comparecodecs.py. Hmm. I'd have thought the proportion higher than that. Not worth doing would be the ones where it's too long to fit onto a single line, so the expression-except would need to be wrapped; pdb.py:461, pdb.py:644, and email/_header_value_parser.py:1644 would be like that. All the rest, though, I don't see a problem with - up to the line 125 boundary, that is. Agreed that after that it's a lot more spotty. > Overall, I think this is a reasonable case for the PEP, but it's not > overwhelming. Which I guess about mirrors my gut feeling for how > useful the new construct would be. I expect it'll be no more useful than ternary if... but also no less useful than ternary if. It'll have its place in the Python programmer's toolbox, without warping every single piece of code. ChrisA From steve at pearwood.info Tue Feb 18 22:41:28 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 19 Feb 2014 08:41:28 +1100 Subject: [Python-ideas] Commas [was Re: except expression] In-Reply-To: <5302ED60.9010307@canterbury.ac.nz> References: <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <5301DDC6.6020505@egenix.com> <53021DBE.5030006@btinternet.com> <5302240C.8030105@egenix.com> <20140217210717.GN4519@ando> <5302ED60.9010307@canterbury.ac.nz> Message-ID: <20140218214128.GQ4519@ando> On Tue, Feb 18, 2014 at 06:19:28PM +1300, Greg Ewing wrote: > Alexander Belopolsky wrote: > >Funny: less than an hour ago, I paused when writing a lambda to return a > >tuple. I conservatively put parentheses around the tuple and did not > >bother to check if they are required. Now I know that they are. > > It's even more confusing when you consider that > you can write > > lambda x, y: x + y, 2 > > and the first comma doesn't break up the lambda, > but the second one does! It shouldn't be confusing. It would be confusing if the expression was broken up in this way: (lambda x), (y: x+y), 2 since it is semantically meaningless. The parser here does the right thing, parsing an expression in a way that is semantically sound. > With oddities like this already in the language, > I don't think we need to worry too much about the > corner cases. But the corner cases could include bugs in the parser, at least theoretically, as well as bugs in people's code. And that is definitely not theoretical. For example, the old form of try...except blocks was a terrible bug magnet: try: stuff() except ValueError, error: print error.arguments People would write: except ValueError, TypeError: and the caught ValueError would be bound to the name TypeError, instead of catching TypeError. > People are always free to insert parens > to make things clearer. E.g. I would probably write > the above as > > (lambda (x, y): x + y), 2 Well, you could try to, but it won't work in Python 3. That's a SyntaxError. -- Steven From greg.ewing at canterbury.ac.nz Tue Feb 18 22:42:19 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 19 Feb 2014 10:42:19 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: <5303D3BB.1000709@canterbury.ac.nz> Paul Moore wrote: > I could argue that > "bare except" is more acceptable in an *expression* context because > expressions should never be so complex that the reasons why it's bad > in a statement context actually apply. The expression itself doesn't have to be complicated. Even a simple expression such as x[i] can invoke arbitrary code, and therefore raise arbitrary exceptions, including ones that represent bugs and therefore shouldn't be silenced. -- Greg From rymg19 at gmail.com Tue Feb 18 22:44:49 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Tue, 18 Feb 2014 15:44:49 -0600 Subject: [Python-ideas] An Everything singleton In-Reply-To: References: Message-ID: On Tue, Feb 18, 2014 at 1:07 PM, Serhiy Storchaka wrote: > This crazy idea is inspired by a discussing in Python-Dev which should be > here. > > Currently we have a None singleton which representing nothing and raises > an exception in almost all operations (except equality, etc). What about > introducing its antonym, an Everything singleton (the name is discussable), > which newer raises an exception in any operation, but returns consistent > value? > > >>> Everything + 2 > 2 > >>> Everything < 3 > True > >>> Everything * 'foo' > 'foo' > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > Sounds like Haskell's Maybe in a slight way. Personally, I'm favored against it, because, as Guido said, it can mask already hard to find bugs. I understand the idea of a null-ish value like this: def f(a,b): return a+b f(Everything,b) or: class x(some_base): def __eq__(self, rhs): return self.some_value == rhs.some_value x() == Everything # True A quick glance, however, would show that it could make writing libraries a pain(What if the user puts Everything? It should work...but it might not!). -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Tue Feb 18 22:46:08 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 08:46:08 +1100 Subject: [Python-ideas] except expression In-Reply-To: <5303D2D7.1090400@canterbury.ac.nz> References: <52FE969D.5000002@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303D2D7.1090400@canterbury.ac.nz> Message-ID: On Wed, Feb 19, 2014 at 8:38 AM, Greg Ewing wrote: > What *might* be useful is to allow the exception type to > be omitted, and have it default to LookupError. Then the > most common anticipated use cases would be very concise: > > things[i] except: default_value While that does seem very tempting, I'm strongly against having a dramatic-yet-likely-unnoticed difference between these two: _ = things[i] except: default_value and try: _ = things[i] except: _ = default_value By your suggestion, the first one is equivalent to: _ = things[i] except LookupError: default_value But by current rules of Python, the second is equivalent to: _ = things[i] except BaseException: default_value and that's really REALLY bad to do unexpectedly. Suppose the lookup into things[i] takes a long time (maybe the system's low on memory and has to page stuff back in), and the user hits Ctrl-C while it's doing it. Catching BaseException means getting back the default_value there; catching LookupError means having KeyboardInterrupt propagate upward. Same goes for typoing 'things', or any other sort of error. I want to be able to explain the exception-expression in terms of a similarly-spelled exception-statement, which means that every piece of common syntax should have parallel semantics - just as lambda and def do. That means having a bare except either cause an error, or do the same thing a bare except does in the statement form. ChrisA From abarnert at yahoo.com Tue Feb 18 22:45:45 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 18 Feb 2014 13:45:45 -0800 (PST) Subject: [Python-ideas] An Everything singleton In-Reply-To: References: Message-ID: <1392759945.28315.YahooMailNeo@web181002.mail.ne1.yahoo.com> From: Serhiy Storchaka Sent: Tuesday, February 18, 2014 11:07 AM >T his crazy idea is inspired by a discussing in Python-Dev which should be here. > > Currently we have a None singleton which representing nothing and raises an > exception in almost all operations (except equality, etc). What about > introducing its antonym, an Everything singleton (the name is discussable), > which newer raises an exception in any operation, but returns consistent value? > >>>> Everything + 2 > 2 >>>> Everything < 3 > True >>>> Everything * 'foo' > 'foo' What about these: ? ? >>> Everything + None ? ? >>> 1 if Everything else 0 ? ? >>> int(Everything) ? ? >>> iter(Everything) ? ? >>> next(Everything) ? ? >>> Everything(23, kw=42) ? ? >>> Everything[-1, 1:3, ...] ? ? >>> range(Everything) ? ? >>> spam(1, Everything, 2) ? ? >>> eggs[Everything:-1] None acts a lot like a singleton value of an ultimate superclass, if the language had such a thing. In fact, it acts almost exactly like, e.g., Scala's null, where null is a singleton instance of the ultimate superclass Null (well, it's really a penultimate superclass; there's Nothing, with _no_ instances, above it? but that isn't relevant here). So Everything should act like a singleton value of an ultimate subclass, right? But Python has an ultimate subclass, object, and it already works almost exactly like Scala's equivalent, Any. You can create lots of object instances, and it can make good sense to do so (e.g., as separate sentinels for separate functions at the same scope). These instances support only a bare minimum set of operations (equality testing, type testing, hashing, string-representing, and introspection). In Scala, Any is also useful as a declared type for variables, as a way to wedge duck typing into static typing (a function can take an Any parameter, a collection can be parameterized on Any element type, etc., in which case they effectively act like Python functions and collections), and there are a few places in or near Python (e.g., parameter attributes used for type annotation, Cython variable types, the PyObject* type and "o"-related arg types in the C API, etc.) where it's used for the same thing. So, Everything clearly isn't the opposite of None in that sense. So? what type _is_ it? Maybe Objective C's nil is a better model. It's a legal value of all types (even those that aren't derived from NSObject or any alternate real class root), and you can call any method on it and it will return nil. But you still can't pass it as an argument to any arbitrary function or operator and expect to get back anything consistent (and, to the extent that you _do_ expect something consistent, it's nil, not some other value from the expression). And the same is true with SQL NULL, IEEE NaN, and every other similar value I can think of: they all avoid exceptions by "infecting" every expression as much as possible, not as little as possible.? For a type like that, Everything is easier, but there are still some problems: ? ? >>> Everything + 2 ? ? Everything ? ? >>> Everything < 3 ? ? ??? (could be Everything, but you still have to decide whether that's truthy?) ? ? >>> Everything * 'foo' ? ? Everything ? ? >>> Everything + None ? ? Everything ? ? >>> 1 if Everything else 0 ? ? 1 # assuming Everything is truthy, as above ? ? >>> int(Everything) ? ? Everything # fine, but what about Everything.__index__()? ? ? >>> iter(Everything) ? ? Everything? ? ? >>> next(Everything) ? ? Everything? ? ? >>> Everything(23, kw=42) ? ? Everything? ? ? >>> Everything[-1, 1:3, ...] ? ? Everything? ? ? >>> range(Everything) ? ? ??? ? ? >>> spam(1, Everything, 2) ? ? ??? # often Everything, but no guarantee ? ? >>> eggs[Everything:-1] ? ? ??? But anyway, even if it's easier to define, it's not what you wanted? From rosuav at gmail.com Tue Feb 18 22:52:14 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 08:52:14 +1100 Subject: [Python-ideas] Commas [was Re: except expression] In-Reply-To: <20140218214128.GQ4519@ando> References: <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <5301DDC6.6020505@egenix.com> <53021DBE.5030006@btinternet.com> <5302240C.8030105@egenix.com> <20140217210717.GN4519@ando> <5302ED60.9010307@canterbury.ac.nz> <20140218214128.GQ4519@ando> Message-ID: On Wed, Feb 19, 2014 at 8:41 AM, Steven D'Aprano wrote: >> People are always free to insert parens >> to make things clearer. E.g. I would probably write >> the above as >> >> (lambda (x, y): x + y), 2 > > Well, you could try to, but it won't work in Python 3. That's a > SyntaxError. And it's semantically different in Py2 - it takes a two-element tuple, rather than two args. ChrisA From steve at pearwood.info Tue Feb 18 22:57:23 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 19 Feb 2014 08:57:23 +1100 Subject: [Python-ideas] An Everything singleton In-Reply-To: References: Message-ID: <20140218215723.GR4519@ando> On Tue, Feb 18, 2014 at 09:07:07PM +0200, Serhiy Storchaka wrote: > This crazy idea is inspired by a discussing in Python-Dev which should > be here. > > Currently we have a None singleton which representing nothing and raises > an exception in almost all operations (except equality, etc). What about > introducing its antonym, an Everything singleton (the name is > discussable), which newer raises an exception in any operation, but > returns consistent value? > > >>> Everything + 2 > 2 Here, your Everything value acts like the Null Object pattern. > >>> Everything < 3 > True Here, your Everything value acts like a Smallest object. But if you then do Everything > 5, presumably you would want it to return True, so it also acts like a Biggest object. Why have Everything < 3 return True? Why not False? > >>> Everything * 'foo' > 'foo' Here, your Everything object acts like an Identity object. But earlier you had it acting like a Null object. How does the Everything object decide whether it should behave as Null or Identity for a certain operation? E.g. should 23**Everything return 1 or 23? 23 == Everything == 42 will presumably return True. What should these things do? some_dict.pop(Everything) # Is this the same as some_dict.clear()? Everything & 42 '+'.join(['a', 'b', Everything, 'c']) I don't think this idea is either useful or self-consistent. -- Steven From greg.ewing at canterbury.ac.nz Tue Feb 18 23:02:44 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 19 Feb 2014 11:02:44 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> Message-ID: <5303D884.3070501@canterbury.ac.nz> Alexander Belopolsky wrote: > - ArithmeticError > - AttributeError > - LookupError > - TypeError ? > - ValueError ? I'd also include EnvironmentError. I wouldn't include TypeError -- that's much more likely to indicate a bug. But this list is getting rather long, which makes me think it's not such a good idea to have a default exception list after all. (BTW, why doesn't ArithmeticError inherit from ValueError?) -- Greg From abarnert at yahoo.com Tue Feb 18 23:02:29 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 18 Feb 2014 14:02:29 -0800 (PST) Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: <1392760949.60646.YahooMailNeo@web181002.mail.ne1.yahoo.com> From: Chris Angelico Sent: Tuesday, February 18, 2014 12:55 PM > Alright, results are in. > > Script: > https://github.com/Rosuav/ExceptExpr/blob/master/find_except_expr.py > Output: > https://github.com/Rosuav/ExceptExpr/blob/master/candidates.txt > Annotated examples: > https://github.com/Rosuav/ExceptExpr/blob/master/examples.py > > The last one is the most readable. I've collected up a bunch of viable > candidates for translation. (Note that I'm not advocating going > through and editing these files for no other reason. That'd just be > code churn. But these are cases where, had this feature existed when > that code was written, it could have been taken advantage of.) It's interesting that some of these call functions that have a fallback-value parameter, so it may be worth asking why they weren't already written as expressions, and whether the new exception expression would be better. Taking one example at random: # Lib/pdb.py:433: ????????try: ????????????func = getattr(self, 'do_' + cmd) ????????except AttributeError: ????????????func = self.default ? ? func = getattr(self, 'do_' + cmd, self.default) ? ? func = getattr(self, 'do_' + cmd) except AttributeError: self.default These aren't identical: using the existing default parameter to getattr would require accessing self.default. Is that guaranteed to exist? (And to be relatively cheap to access?)?If so, then presumably there's some reason for the more explicit form, and I don't think the except-expression version would be acceptable if the three-argument version isn't. But if not, I think that's a clear win for this proposal: Here's something we would have liked to write as an expression, but, even with a function that already has a default parameter, we couldn't do so without the except-expression. There are even more cases like this: # Lib/sysconfig.py:529: ????????try: ????????????_CONFIG_VARS['abiflags'] = sys.abiflags ????????except AttributeError: ????????????# sys.abiflags may not be defined on all platforms. ????????????_CONFIG_VARS['abiflags'] = '' ? while you _could_ replace this with getattr(sys, 'abiflags', ''), only a madman would do so? but in a language with ".?"-type syntax you'd almost certainly use that, and with this proposal,?sys.abiflags except AttributeError: '' is equivalent and much more readable. So, that looks like another win. > My script is currently _very_ simplistic. It looks *only* for > assignments, so it won't find something like this: > > try: > ? ? foo.append(args[0]) > except IndexError: > ? ? foo.append('') > > which is correct, because it's impossible to know whether foo.append() > will raise IndexError. (Chances are it won't, especially if we know > foo is a list, for instance.) It's not safe to directly translate that > sort of thing. It might, however, be something worth improving, as it > narrows the scope of the except clause. But if that same code is > written thus: > > try: > ? ? tmp = args[0] > except IndexError: > ? ? tmp = '' > foo.append(tmp) > > then the script _will_ find it, and then it really is a candidate for editing. > > Point to note: Apart from one instance, where it wasn't being used > anyway, I found not a single instance of 'as' being used. There was > one case where sys.exc_info() was referenced, though, so this may just > mean that the stdlib files in question are maintaining compatibility > with old versions of Python. > > I didn't look at most of the tests. The script found 195 plausible > try/except blocks, of which 37 have the word "test" in the name;that > leaves 158 that are likely to benefit from this change. There are a > couple of false positives, but not many. > > Next is to figure out what else is a candidate for editing. Here's my > current criteria, straight from the docstring: > > """Report on 'simple' try/except statements. > > The definition of simple is: > 1. No else or finally clause > 2. Only one except clause (currently) > 3. Exactly one statement in the try clause > 4. Exactly one statement in each except clause > 5. Each of those statements is the same type. > 6. That type is one that could be an expression. > 7. Those statements are all compatible. > > The last two are the trickiest. Currently I'm looking > only for assignments, where both try and except assign > to the same target. This is, however, too narrow.""" > > Interestingly, removing criterion 2 gives us three additional examples > out of the test suite, and nothing else. There are no cases outside of > the test suite that look like this: > > try: > ? ? x = some_calculation > except ValueError: > ? ? x = something_else > except OverflowError: > ? ? x = different_thing > > (The test suite has one case with those exact two exceptions, and a > pair that use OverflowError and ZeroDivisionError. In each case, > they're comparing two actions to make sure they give the same result, > where "throwing OverflowError" is a result like any other.) > > Conclusions: The need for chained exception catching might not be so > great after all, and even the 'as' keyword isn't as (heh heh) > important as I thought it was. > > Alternate conclusion: My sample is too small. Need more. Data, you > have the helm. > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > From greg.ewing at canterbury.ac.nz Tue Feb 18 23:21:43 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 19 Feb 2014 11:21:43 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> Message-ID: <5303DCF7.3030404@canterbury.ac.nz> Alexander Belopolsky wrote: > For example, it would be nice to be able to write something like > > x = d1[key] except d2[key] except 0 > > but sprinkle this with repeated KeyError and what is important (d1 and > d2) will be lost in the scaffolding. Another crazy thought: Make the default exception depend on what kind of expression the except clause is attached to. things[i] except: default # catches LookupError distance / time except: very_fast # catches ZeroDivisionError Anything else would require an explicit exception type. -- Greg From rymg19 at gmail.com Tue Feb 18 23:25:28 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Tue, 18 Feb 2014 16:25:28 -0600 Subject: [Python-ideas] Function to return first(or last) true value from list Message-ID: It isn't uncommon to try and get either the first or the last True value from a list. In Python 2, you'd do this: next((x for x in mylist if x)) And, in Python 3, thanks to filter returning an iterator, you'd do this: next(filter(bool,mylist)) It still is pretty common. Common enough to make it aggravating to write a function to do that nearly every time. My idea is to add first and lastfunctions that do just that. Stuff that's open to lots of debate: - Names. They're not very creative; I know. - Builtin or itertools. I'm personally leaning towards the latter at the moment. Implementing it in itertools would be simple: def first(it): return next(filter(bool,it)) def last(it): return next(reversed(filter(bool,it))) -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Tue Feb 18 23:37:12 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 18 Feb 2014 14:37:12 -0800 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: Message-ID: On Feb 18, 2014, at 14:25, Ryan Gonzalez wrote: > It isn't uncommon to try and get either the first or the last True value from a list. In Python 2, you'd do this: > next((x for x in mylist if x)) > And, in Python 3, thanks to filter returning an iterator, you?d do this: > > next(filter(bool,mylist)) > It still is pretty common. Common enough to make it aggravating to write a function to do that nearly every time. My idea is to add first and last functions that do just that. > > Stuff that?s open to lots of debate: > > Names. They?re not very creative; I know. > Builtin or itertools. I?m personally leaning towards the latter at the moment. If suggest putting it on PyPI, or submitting it to an existing project like more-itertools, and seeing what kind of uptake it has, before trying to decide whether it's necessary to add to the stdlib. Except I think first may already be in more-itertools. > > Implementing it in itertools would be simple: > > def first(it): > return next(filter(bool,it)) > > def last(it): > return next(reversed(filter(bool,it))) But reversed takes a sequence, not an iterator. So you'd have to call it on list(filter(...)), and surely you don't want to build the whole list just to get the last value. You could build it by using a single-element peek wrapper, or just an explicit for loop that keeps track of "last" as it goes along. > -- > Ryan > If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.peters at gmail.com Tue Feb 18 23:42:21 2014 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 18 Feb 2014 16:42:21 -0600 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: Message-ID: [Ryan Gonzalez ] > It isn't uncommon to try and get either the first or the last True value > from a list. In Python 2, you'd do this: > > next((x for x in mylist if x)) > > And, in Python 3, thanks to filter returning an iterator, you'd do this: > > next(filter(bool,mylist)) > ... See here: http://bugs.python.org/issue18652 Never heard anyone ask for "last" before "first" has been available in a PyPI package for over a year. Detail in the bug report. From steve at pearwood.info Tue Feb 18 23:45:14 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 19 Feb 2014 09:45:14 +1100 Subject: [Python-ideas] Inline Function - idea In-Reply-To: References: Message-ID: <20140218224514.GS4519@ando> On Tue, Feb 18, 2014 at 10:07:20AM -0700, Alex Rodrigues wrote: > > Alex, can you explain the difference (if any) between your proposal and dynamic scoping? > > > > -- Steven > > Inline functions would be a bit like dynamic scoping and a bit like > macros. The main difference from dynamic scoping is that they would > not search up beyond their parent to find names, What is "their parent"? Is that the function they are defined in, or the function they are called in? > since they act > exactly like code injected at the spot they are called it is expected > that the variables they are using (like the ones in it's parent) are > either locals or globals. But that is *not how it works* if you actually type the code in that spot. If you have code that looks like this: def spam(): def eggs(): def cheese(): ### <= insert code here then code inserted into cheese *will* search the eggs and spam local variables as well as globals. Having "inline code" skip over eggs and spam namespaces is a major difference in behaviour. The advantage of dynamic scoping is that it is a standard, well-understood execution model that can easily be described. The pros and cons are well known. There are more cons than pros, which is why very few modern languages support it. Your suggestion seems to be half dynamic, half lexical (static), but not quite the same as either. So it would be completely innovative, and since nobody has used this before, nobody can know how well it will work in practice. > I'm not sure that that has many advantages > outside of demanding less from the runtime, How do you know it demands less from the runtime? > but that's how I imagined > it. IMO The biggest advantage to inline functions over other > constructs is ease of understanding. Well I don't understand it. Flat scoping, like in Python before nested_scopes was introduced, that's easy. Nested scopes (static or lexical scoping), that's also easy to understand: scope follows the layout of your code when indented. That's extremely intuitive. Dynamic scoping is a bit harder to grasp, but not very much so. A function sees the variables in the code that calls it, not the code where it was defined. So if you have a function spam that calls eggs that calls cheese, cheese sees the local variables inside eggs and spam. But yours, well, I'm not quite sure how it's supposed to work, beause it is half static and half dynamic and all confusing. I believe that you think this is a simple model to understand because you're only thinking of simple cases. But what happens if you have an inline function call an inline function? def regular(): x = 42 inline() @make_inline def inline(): another_line() @make_inline def another_inline(): print(x) According to your comment above, another_inline should NOT follow the normal dynamic scoping search path: - does another_inline have a local variable x? - if not, does inline have a local variable x? - if not, does regular have a local variable x? <=== FOUND - if not, look for a global instead it should follow: - does another_inline have a local variable x? - if not, does inline have a local variable x? - do not search regular, since that is too deep - skip straight to globals But that contradicts the idea that the code in another_inline should behave just as if it were typed inside inline. So your description is ambiguous -- will calling regular() above raise a NameError, or will it print 42? There are *all sorts* of odd corners to be considered. What happens if another_inline is declared *inside* inline, rather than outside? What happens if another_inline is declared inside *regular*? What if it's a closure that has come from somewhere else? I don't have any intuition to be able to immediately guess what the behaviour in these cases will be. I think that either static or dynamic scoping is easy to reason about, but your hybrid is not. > We may you be used to > understanding scoping after lots of programming, but it's not always > intuitive. On the other hand it is extremely intuitive to understand > "when you call this, all that code runs exactly as if you had typed it > here". But that's not what you described earlier. See my earlier comment. -- Steven From ncoghlan at gmail.com Tue Feb 18 23:51:22 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 19 Feb 2014 08:51:22 +1000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: On 19 Feb 2014 07:20, "Paul Moore" wrote: > > On 18 February 2014 20:55, Chris Angelico wrote: > > Alright, results are in. > > > > Script: > > https://github.com/Rosuav/ExceptExpr/blob/master/find_except_expr.py > > Output: > > https://github.com/Rosuav/ExceptExpr/blob/master/candidates.txt > > Annotated examples: > > https://github.com/Rosuav/ExceptExpr/blob/master/examples.py > > Great work! Indeed! On the syntax front, one option to keep in mind is the "yield expression" solution: require the surrounding parentheses, except when it is used as a standalone statement, as the sole expression on the RHS of an assignment statement, or (perhaps) as the sole positional argument to a function call. That approach keeps the simple cases clean, while making more complex cases a bit easier to read. > > Looking at the annotated examples, my conclusions would be: > > 1. The "expr except Exception: default" syntax is actually a lot more > readable in context than I expected. > 2. But sometimes in more complex cases the similarity with the > statement form hurts readability. > 3. The win is noticeable on the one-line assignments > 4. You had a few cases where you could have (should have?) translated > to 3-arg getattr or similar. I'm not quite sure what that says ("if > you have a hammer everything looks like a nail"?) > 5. The longer examples typically look more confusing to me than the originals. > 6. As you noted, nothing much used multiple exceptions or as. I think > this implies they are of marginal benefit at best (but I'm guessing > just as much as you). I think it's worth keeping the name binding (since that would be very hard to add later), but restricting the initial proposal to a single except clause would make sense. Without the block structure to help, it is awfully difficult to associate multiple except clauses with their corresponding initial expression, and if you really want to have different results for different exceptions, you can pass the caught exception to a translator function. > > Based on this sample, my main worry with the new syntax is that people > will over-use it. Up to line 125 of the examples I'd say about 50% > seem like wins. After that point the only ones I like are > Lib/tarfile.py and Tools/unicode/comparecodecs.py. This was one of the main concerns with PEP 308 as well, and the general principle of "use it only if it helps readability" seems to have kept that under control. > Overall, I think this is a reasonable case for the PEP, but it's not > overwhelming. Which I guess about mirrors my gut feeling for how > useful the new construct would be. As with PEP 308, I think this is more about the existing workarounds for the lack of a language level solution causing enough problems that the new syntax is being suggested primarily to say "stop doing that!". Cheers, Nick. > > Paul > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Feb 18 23:56:46 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 19 Feb 2014 08:56:46 +1000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: On 19 Feb 2014 06:56, "Chris Angelico" wrote: > > My script is currently _very_ simplistic. It looks *only* for > assignments, so it won't find something like this: > > try: > foo.append(args[0]) > except IndexError: > foo.append('') > > which is correct, because it's impossible to know whether foo.append() > will raise IndexError. (Chances are it won't, especially if we know > foo is a list, for instance.) It's not safe to directly translate that > sort of thing. It might, however, be something worth improving, as it > narrows the scope of the except clause. But if that same code is > written thus: > > try: > tmp = args[0] > except IndexError: > tmp = '' > foo.append(tmp) > > then the script _will_ find it, and then it really is a candidate for editing. This example should go in the PEP - the except clause in the example code is almost certainly too broad, but the fix is rather ugly. With the PEP, the except clause can easily be moved inside the expression so it doesn't cover the append() call: foo.append((args[0] except IndexError: "")) Cheers, Nick. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Wed Feb 19 00:01:10 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 19 Feb 2014 10:01:10 +1100 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: Message-ID: <20140218230110.GT4519@ando> On Tue, Feb 18, 2014 at 04:25:28PM -0600, Ryan Gonzalez wrote: > It isn't uncommon to try and get either the first or the last True value > from a list. I'm not sure that I've ever wanted to do either. If I've ever wanted the first true value, it was so uncommon I've forgotten. But I'm pretty confident I've never wanted the *last* true value. That would be a strange thing to do. > In Python 2, you'd do this: > > next((x for x in mylist if x)) That works fine in Python 3 too. > And, in Python 3, thanks to filter returning an iterator, you'd do this: > > next(filter(bool,mylist)) > > It still is pretty common. Common enough to make it aggravating to write a > function to do that nearly every time. But you aren't writing a function. It's a simple, trivial, one-line operation, a single expression. Not every trivial one-line operation needs to be a function. Just write "next(filter(bool, mylist))" in-place, where you want it to appear. It's only a couple of characters longer than "itertools.first(mylist)", and is one less thing to memorize. [...] > Stuff that's open to lots of debate: > > - Names. They're not very creative; I know. > - Builtin or itertools. I'm personally leaning towards the latter at the > moment. You missed the most important question: whether or not this is worth doing at all. -- Steven From denis.spir at gmail.com Wed Feb 19 00:06:22 2014 From: denis.spir at gmail.com (spir) Date: Wed, 19 Feb 2014 00:06:22 +0100 Subject: [Python-ideas] An Everything singleton In-Reply-To: References: Message-ID: <5303E76E.3040603@gmail.com> On 02/18/2014 08:27 PM, Guido van Rossum wrote: > This idea comes up occasionally (though I've never heard it named > "everything" before). I believe it is similar to a feature in Objective-C > (or maybe just Apple's version) where the behavior is that if you send any > message to nil it does nothing and returns nil. > > In Python I think it would be very disturbing though and would end up > masking errors. Yes; and unfortunately this often happens (also with none in python, when misused); there are related discussion about NaN as well (eg see Eiffel), and with funcs which try to guess clients' needs instead of just failing. > (Python and ObjC seems to have very different attitudes > about exceptions -- in Python they are used all the time while in ObjC the > culture seems to favor the idea that exceptions are fatal and shouldn't > even be caught.) (Rather, they are for cases of failures impredictable on the client side (eg file not found), which are not programmer errors (the cases belong to the app logic); in such a case, one may catch and deal with the case (eg create the file)). d From ncoghlan at gmail.com Wed Feb 19 00:11:51 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 19 Feb 2014 09:11:51 +1000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303D2D7.1090400@canterbury.ac.nz> Message-ID: On 19 Feb 2014 07:48, "Chris Angelico" wrote: > > On Wed, Feb 19, 2014 at 8:38 AM, Greg Ewing wrote: > > What *might* be useful is to allow the exception type to > > be omitted, and have it default to LookupError. Then the > > most common anticipated use cases would be very concise: > > > > things[i] except: default_value > > While that does seem very tempting, I'm strongly against having a > dramatic-yet-likely-unnoticed difference between these two: > > _ = things[i] except: default_value > > and > > try: > _ = things[i] > except: > _ = default_value > > By your suggestion, the first one is equivalent to: > > _ = things[i] except LookupError: default_value > > But by current rules of Python, the second is equivalent to: > > _ = things[i] except BaseException: default_value > > and that's really REALLY bad to do unexpectedly. Suppose the lookup > into things[i] takes a long time (maybe the system's low on memory and > has to page stuff back in), and the user hits Ctrl-C while it's doing > it. Catching BaseException means getting back the default_value there; > catching LookupError means having KeyboardInterrupt propagate upward. > Same goes for typoing 'things', or any other sort of error. I want to > be able to explain the exception-expression in terms of a > similarly-spelled exception-statement, which means that every piece of > common syntax should have parallel semantics - just as lambda and def > do. That means having a bare except either cause an error, or do the > same thing a bare except does in the statement form. +1000 from me for disallowing bare except in the expression form - suppressing SystemExit and KeyError is bad, and if an explicit exception is required, people are more likely to spell "catch everything" as "except Exception:" rather than "except BaseException:" . Cheers, Nick. > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Wed Feb 19 00:27:06 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 19 Feb 2014 10:27:06 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: <20140218232706.GU4519@ando> On Tue, Feb 18, 2014 at 02:07:43PM +0000, Paul Moore wrote: > Also, "expr except fallback" is a very simple case of a keyword-based > binary operation. So (ignoring the bare except issue) it's much less > controversial. But you can't ignore that issue. That issue is *critical* to why it is a simple binary operation. I think I would almost prefer the status quo of no except-expression than one which encourages people to implicitly catch all exceptions. Yes, for a trivially simple case, it's harmless: 1/x except INFINITY but you're teaching people bad habits. It doesn't take a much more complex version to show why it's a bad habit: 1/x + mylits[0] except INFINITY If not for backwards compatibility, I think bare excepts should be removed from the language altogether. They are an attractive nuisance. I don't want to add even more of them. If you truly want to catch everything, catch BaseException. The fact that this is longer to write than Exception is a *good thing*. Making it easier to do the wrong thing is the wrong thing to do. -1 on bare excepts in the expression form. If that makes it harder to sell the PEP, oh well, better that than adding a misfeature to the language. -- Steven From rymg19 at gmail.com Wed Feb 19 00:30:29 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Tue, 18 Feb 2014 17:30:29 -0600 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: Message-ID: Sorry! I was Googling it, but I must have missed that bug report. On Tue, Feb 18, 2014 at 4:42 PM, Tim Peters wrote: > [Ryan Gonzalez ] > > It isn't uncommon to try and get either the first or the last True value > > from a list. In Python 2, you'd do this: > > > > next((x for x in mylist if x)) > > > > And, in Python 3, thanks to filter returning an iterator, you'd do this: > > > > next(filter(bool,mylist)) > > ... > > See here: > > http://bugs.python.org/issue18652 > > Never heard anyone ask for "last" before "first" has been available > in a PyPI package for over a year. Detail in the bug report. > -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Wed Feb 19 00:34:09 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 19 Feb 2014 10:34:09 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> Message-ID: <20140218233408.GV4519@ando> On Wed, Feb 19, 2014 at 02:25:15AM +1100, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 2:13 AM, MRAB wrote: > > Another possibility would be to say that a bare except in an expression > > catches only certain "expression-oriented" exceptions, e.g. ValueError. > > The simplest way to do that would be to make them subclasses of an > > ExpressionError class. The question then becomes one of which > > exceptions are "expression-oriented"... > > Easier than fiddling with class inheritance would be to make > ExpressionError into a tuple of exception types - at least for testing > purposes. If this _is_ to be done, I would definitely advocate having > that name (or some equivalent) as a built-in, so it's straight-forward > to either manipulate it or use it in a statement-except. But you're > absolutely right that the question of which ones are > expression-oriented would be a big one. I could imagine catching > (ValueError, Unicode{En,De}codeError, AttributeError, EOFError, > IOError, OSError, LookupError, NameError, ZeroDivisionError) and > that's just from a quick skim of the built-in names. Would you > consider making a tuple like that and then using "except > CommonErrors:" all over your code? Or more to the point, if it were > really convenient to do that, would it improve your code? I hope that you were making a reductio ad absurdum argument as to why this is a bad idea. Exception handlers should ALWAYS catch the FEWEST errors that you KNOW you need to handle. (Excuse my shouting, but this is important.) Encouraging people to catch all sorts of unrelated errors which, if they ever did occur, would absolutely definitely without a doubt represent an actual bug in their code is a terrible, terrible idea. If you wrote 1/x except CommonErrors: 0 expecting that x was a number, and IOError was raised, caught and swallowed, wouldn't you rather know about it? -- Steven From ethan at stoneleaf.us Wed Feb 19 00:30:08 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 18 Feb 2014 15:30:08 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> Message-ID: <5303ED00.6060605@stoneleaf.us> On 02/18/2014 07:25 AM, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 2:13 AM, MRAB wrote: >> Another possibility would be to say that a bare except in an expression >> catches only certain "expression-oriented" exceptions, e.g. ValueError. >> The simplest way to do that would be to make them subclasses of an >> ExpressionError class. The question then becomes one of which >> exceptions are "expression-oriented"... > > Easier than fiddling with class inheritance would be to make > ExpressionError into a tuple of exception types - at least for testing > purposes. If this _is_ to be done, I would definitely advocate having > that name (or some equivalent) as a built-in, so it's straight-forward > to either manipulate it or use it in a statement-except. But you're > absolutely right that the question of which ones are > expression-oriented would be a big one. I could imagine catching > (ValueError, Unicode{En,De}codeError, AttributeError, EOFError, > IOError, OSError, LookupError, NameError, ZeroDivisionError) and > that's just from a quick skim of the built-in names. Would you > consider making a tuple like that and then using "except > CommonErrors:" all over your code? Or more to the point, if it were > really convenient to do that, would it improve your code? If you're catching that many exceptions, you may as well just use a bare except. And be prepared to mask bugs. -- ~Ethan~ From greg.ewing at canterbury.ac.nz Wed Feb 19 00:57:37 2014 From: greg.ewing at canterbury.ac.nz (Greg) Date: Wed, 19 Feb 2014 12:57:37 +1300 Subject: [Python-ideas] An Everything singleton In-Reply-To: <20140218215723.GR4519@ando> References: <20140218215723.GR4519@ando> Message-ID: <5303F371.6050305@canterbury.ac.nz> On 19/02/2014 10:57 a.m., Steven D'Aprano wrote: > Why have Everything < 3 return True? Why not False? Why not Everything? -- Greg From python at mrabarnett.plus.com Wed Feb 19 00:58:46 2014 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 18 Feb 2014 23:58:46 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: <5303F3B6.6070202@mrabarnett.plus.com> On 2014-02-18 20:55, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 3:06 AM, Chris Angelico wrote: >> I'm currently working on finding a bunch of examples from the stdlib >> that could be translated. Will post them once I get the script sorted >> out. As it's currently 3AM, though, that means I need to fry up some >> bacon or something before I continue :) > > Alright, results are in. > > Script: > https://github.com/Rosuav/ExceptExpr/blob/master/find_except_expr.py > Output: > https://github.com/Rosuav/ExceptExpr/blob/master/candidates.txt > Annotated examples: > https://github.com/Rosuav/ExceptExpr/blob/master/examples.py > > The last one is the most readable. I've collected up a bunch of viable > candidates for translation. (Note that I'm not advocating going > through and editing these files for no other reason. That'd just be > code churn. But these are cases where, had this feature existed when > that code was written, it could have been taken advantage of.) > At least one of the examples doesn't need this proposed addition. "getattr" can accept a default, so: func = getattr(self, 'do_' + cmd) except AttributeError: self.default can be written as: func = getattr(self, 'do_' + cmd, self.default) [snip] From zuo at chopin.edu.pl Wed Feb 19 00:58:40 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Wed, 19 Feb 2014 00:58:40 +0100 Subject: [Python-ideas] except expression In-Reply-To: <5303789A.7040406@mrabarnett.plus.com> References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> Message-ID: 18.02.2014 16:13, MRAB wrote: > Another possibility would be to say that a bare except in an > expression > catches only certain "expression-oriented" exceptions, e.g. > ValueError. > The simplest way to do that would be to make them subclasses of an > ExpressionError class. The question then becomes one of which > exceptions are "expression-oriented"... I believe it is not a good idea -- as: * Explicit is better than implicit?. * When dealing with exceptions, catching too much is in fact much worse than catching too little (at least most often). Cheers. *j From ethan at stoneleaf.us Wed Feb 19 00:43:54 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 18 Feb 2014 15:43:54 -0800 Subject: [Python-ideas] except expression In-Reply-To: <20140218232706.GU4519@ando> References: <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> Message-ID: <5303F03A.9080801@stoneleaf.us> On 02/18/2014 03:27 PM, Steven D'Aprano wrote: > > -1 on bare excepts in the expression form. Agreed. -- ~Ethan~ From steve at pearwood.info Wed Feb 19 01:10:49 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 19 Feb 2014 11:10:49 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> Message-ID: <20140219001048.GX4519@ando> On Tue, Feb 18, 2014 at 12:01:29PM -0500, Alexander Belopolsky wrote: > I disagree. It is best to leave explicit selection of exceptions to catch > outside of the expressions. This is job for things like decimal or numpy > fp contexts. I don't understand what relevance fp contexts have here. > Always requiring a long camel-case name inside an expression will kill most > of the advantages. I doubt that. And they're not typically that long. TypeError is nine characters, no longer than your name :-) > For example, it would be nice to be able to write something like > > x = d1[key] except d2[key] except 0 Which is harmful, because it masks bugs. If you want to save typing, define K = KeyError at the top of your module, and write: x = d1[key] except K d2[key] except K 0 which by the way looks horrible without the colons (sorry Paul, you have not convinced me that the Tim Peters prohibition on grit applies here). Even with three levels of indentation, mandatory brackets, and colons, it still fits within 79 columns and reads quite well: if condition: while something(): for key in sequence: x = (d1[key] except KeyError: (d2[key] except KeyError: 0)) Take the colons and brackets out, and I think it is less readable: x = d1[key] except KeyError d2[key] except KeyError 0 and ambiguous. By the way, I think this is a good example for the PEP (Chris are you reading?). You might be tempted to re-write this as: x = d1.get(key, d2.get(key, 0)) which is shorter, but the semantics are different. If the second case, you have to pre-calculate the fallback, which may be expensive, while in the exception form, you only calculate the fallback if the first lookup actually fails. > but sprinkle this with repeated KeyError and what is important (d1 and d2) > will be lost in the scaffolding. I think that is wrong. I realise it is a matter of taste, but in this instance I think my taste is much closer to the rest of Python's syntax than yours is. (But you probably think the same, yes?) -- Steven From zuo at chopin.edu.pl Wed Feb 19 01:11:52 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Wed, 19 Feb 2014 01:11:52 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> Message-ID: <64da979e889af212ba4a98c5da91a38f@chopin.edu.pl> 18.02.2014 17:43, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 3:25 AM, Paul Moore > wrote: >> Explicit is better than implicit - I think this discussion has done >> its job and established that trying to assume a subset of exceptions >> to catch isn't going to work. We either allow a bare except to mean >> "catch all exceptions" (which exactly the same risks and provisos as >> a >> bare except *statement*) or we make the exception to catch >> mandatory. > > Yep, agreed. I'm personally inclined to permit the bare except, and > then advise against it in PEP 8, but both halves of that are > debatable. I'm of the opinion that this would be risky: IMHO bare except is practically always a very bad practice unless the exception is immediately re-raised: try: foo() except: logger.exception('error:') raise ...and AFAIK, up to now, nobody proposed any syntax that would make it possible to re-raise an exception in an except expression. Therefore I believe that bare except should *not* be allowed in except expressions at all. My-3-cents-ly yours *j From zuo at chopin.edu.pl Wed Feb 19 01:19:49 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Wed, 19 Feb 2014 01:19:49 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> Message-ID: <8b4fe0d54b73e5e881131998739fef61@chopin.edu.pl> 18.02.2014 17:25, Paul Moore napisa?: > OTOH, there's still an argument for only allowing a single exception > name in the syntax (an "identifier" rather than an "expression" in > syntax terms). If you must catch multiple exceptions, give the > relevant tuple a name. I believe that at this point (what the exception spec would be allowed to be: identifier?, tuple?, any expression?) the syntax should be identical to the statement syntax (i.e.: any expression). Less special cases to remember. For the same reason, I believe that tuple expressions ("except (ValueError, TypeError)") should be obligatorily enclosed with parens as long as they are obligatorily enclosed with parens in the statement syntax (i.e., probably till Python 3.13 :)). *j From ethan at stoneleaf.us Wed Feb 19 00:58:45 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 18 Feb 2014 15:58:45 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: <5303F3B5.9030801@stoneleaf.us> On 02/18/2014 12:55 PM, Chris Angelico wrote: > > Conclusions: The need for chained exception catching might not be so > great after all, and even the 'as' keyword isn't as (heh heh) > important as I thought it was. I wouldn't base that conclusion on the lack of seeing the new syntax in the stdlib. The stdlib works -- going through and changing working code to use the new syntax is at best code churn, and at worst (and more likely) introducing new bugs. -- ~Ethan~ From steve at pearwood.info Wed Feb 19 00:50:45 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 19 Feb 2014 10:50:45 +1100 Subject: [Python-ideas] except expression In-Reply-To: <530385AA.70703@mrabarnett.plus.com> References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <530385AA.70703@mrabarnett.plus.com> Message-ID: <20140218235045.GW4519@ando> On Tue, Feb 18, 2014 at 04:09:14PM +0000, MRAB wrote: > The question is whether it should be OK to allow a bare except, where > that would catch a limited number of exceptions, in those cases where > there won't be any risk. Having a bare except catch everything in a try...except block, and only a few things in an except expression, is a violation of the Principle of Least Surprise. Not to mention the Zen about implicit vs explicit. And the idea that the language designers -- that's us -- might be able to predict ahead of time which exceptions aren't risky is laughable. We really can't. ANY exception could be a bug in the code, and therefore wrong to mask by default. The only person capable of correctly deciding which exceptions to catch is the person writing the code. (And not even always them, but we don't have to solve the problem of poor coders writing poor code. It's enough to avoid encouraging it.) -- Steven From rob.cliffe at btinternet.com Wed Feb 19 01:28:30 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Wed, 19 Feb 2014 00:28:30 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: <5303FAAE.90205@btinternet.com> On 18/02/2014 22:56, Nick Coghlan wrote: > > > On 19 Feb 2014 06:56, "Chris Angelico" > wrote: > > > > My script is currently _very_ simplistic. It looks *only* for > > assignments, so it won't find something like this: > > > > try: > > foo.append(args[0]) > > except IndexError: > > foo.append('') > > > > which is correct, because it's impossible to know whether foo.append() > > will raise IndexError. (Chances are it won't, especially if we know > > foo is a list, for instance.) It's not safe to directly translate that > > sort of thing. It might, however, be something worth improving, as it > > narrows the scope of the except clause. But if that same code is > > written thus: > > > > try: > > tmp = args[0] > > except IndexError: > > tmp = '' > > foo.append(tmp) > > > > then the script _will_ find it, and then it really is a candidate > for editing. > > This example should go in the PEP - the except clause in the example > code is almost certainly too broad, but the fix is rather ugly. > > With the PEP, the except clause can easily be moved inside the > expression so it doesn't cover the append() call: > > foo.append((args[0] except IndexError: "")) > > Cheers, > Nick. > Beautiful catch, Nick! Shows that this proposal not only gives a neater way of writing code without changing the semantics, but may actually point the way to _improving_ existing (or new) code. Rob Cliffe > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6604 - Release Date: 02/18/14 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Wed Feb 19 01:02:00 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 18 Feb 2014 16:02:00 -0800 Subject: [Python-ideas] except expression In-Reply-To: <5303F3B6.6070202@mrabarnett.plus.com> References: <52FE969D.5000002@canterbury.ac.nz> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303F3B6.6070202@mrabarnett.plus.co m> Message-ID: <5303F478.7070902@stoneleaf.us> On 02/18/2014 03:58 PM, MRAB wrote: > On 2014-02-18 20:55, Chris Angelico wrote: >> On Wed, Feb 19, 2014 at 3:06 AM, Chris Angelico wrote: >>> I'm currently working on finding a bunch of examples from the stdlib >>> that could be translated. Will post them once I get the script sorted >>> out. As it's currently 3AM, though, that means I need to fry up some >>> bacon or something before I continue :) >> >> Alright, results are in. >> >> Script: >> https://github.com/Rosuav/ExceptExpr/blob/master/find_except_expr.py >> Output: >> https://github.com/Rosuav/ExceptExpr/blob/master/candidates.txt >> Annotated examples: >> https://github.com/Rosuav/ExceptExpr/blob/master/examples.py >> >> The last one is the most readable. I've collected up a bunch of viable >> candidates for translation. (Note that I'm not advocating going >> through and editing these files for no other reason. That'd just be >> code churn. But these are cases where, had this feature existed when >> that code was written, it could have been taken advantage of.) >> > At least one of the examples doesn't need this proposed addition. > > "getattr" can accept a default, so: > > func = getattr(self, 'do_' + cmd) except AttributeError: self.default > > can be written as: > > func = getattr(self, 'do_' + cmd, self.default) That depends on whether it's okay to evaluate self.default when it's not needed. -- ~Ethan~ From zuo at chopin.edu.pl Wed Feb 19 01:40:43 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Wed, 19 Feb 2014 01:40:43 +0100 Subject: [Python-ideas] except expression In-Reply-To: <20140219001048.GX4519@ando> References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> Message-ID: <43f5f27531a95efe1e76fb96babffa49@chopin.edu.pl> 19.02.2014 01:10, Steven D'Aprano wrote: > On Tue, Feb 18, 2014 at 12:01:29PM -0500, Alexander Belopolsky wrote: > >> I disagree. It is best to leave explicit selection of exceptions to >> catch >> outside of the expressions. This is job for things like decimal or >> numpy >> fp contexts. > > I don't understand what relevance fp contexts have here. > > >> Always requiring a long camel-case name inside an expression will >> kill most >> of the advantages. > > I doubt that. And they're not typically that long. TypeError is nine > characters, no longer than your name :-) > > >> For example, it would be nice to be able to write something like >> >> x = d1[key] except d2[key] except 0 > > Which is harmful, because it masks bugs. If you want to save typing, > define K = KeyError at the top of your module, and write: > > x = d1[key] except K d2[key] except K 0 > > > which by the way looks horrible without the colons (sorry Paul, you > have > not convinced me that the Tim Peters prohibition on grit applies > here). > Even with three levels of indentation, mandatory brackets, and > colons, it still fits within 79 columns and reads quite well: > > if condition: > while something(): > for key in sequence: > x = (d1[key] except KeyError: (d2[key] except KeyError: > 0)) > > Take the colons and brackets out, and I think it is less readable: > > x = d1[key] except KeyError d2[key] except KeyError 0 > > and ambiguous. +1. Though I still like the paren-after-except syntax more. :) x = d1[key] except (KeyError: d2[key] except (KeyError: 0)) An advantage of this one is IMHO that: 1. first we see the basic expression (d1[key]) 2. but, immediately, we notice the 'except' keyword ("a-ha: generally it is *d1[key]* but with reservations for some special cases, let's see them...") 3. then, we read what are those "special cases" (KeyError: ...) which visually are nicely separated from the basic expression. Cheers. *j PS. Of course it could be laid out with some line breaks, e.g.: x = (d1[key] except (KeyError: d2[key] except (KeyError: 0))) PS2. Yes, I also see some visual advantages of the paren-enclosing-whole-expression syntax when applying that kind of layout: x = (d1[key] except KeyError: (d2[key] except KeyError: 0)) PS3. Anyway, IMHO *obligatory* parens would be a good thing. Also because they "neutralize" the visual connotations of colon with its typical role of block-statement indicator (as it's obvious that a block cannot start within parens). From steve at pearwood.info Wed Feb 19 01:43:57 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 19 Feb 2014 11:43:57 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: <20140219004357.GY4519@ando> On Tue, Feb 18, 2014 at 03:56:10PM +0000, Paul Moore wrote: > On 18 February 2014 15:05, Chris Angelico wrote: > >> Well, yes and no. There are only 2 out of the 10 syntaxes originally > >> proposed in the PEP at the start of this thread that use a colon. I've > >> already pointed out that I don't like a colon, so assume I said > >> "except return 0" if you must :-) > > > > Whether it's a colon or another keyword, _something_ is necessary in > > there, if it's permissible to specify an exception type: > > > > sum(x[3] except IndexError 0 for x in list_of_tuples) > > sum(x[3] except 0 for x in list_of_tuples) > > > > How would you parse each of those, without a separator? > > With "return" as a separator. I said that above. Was I not clear? Yes you were clear, but this is a huge complicated thread and if Chris is the least bit like me, he's probably feeling a bit overloaded :-) Using return to not actually return seems completely wrong to me. This is the problem with all the proposals to overload some arbitrary keyword *just because it is available*. To me, all of these are equally as arbitrary and silly: expression except return 42 expression except pass 42 expression except del 42 expression except import 42 expression except class 42 For each keyword in (return, pass, del, import, ...), we are attempting to overload it to mean something different inside an except expression to what it does outside of one. With return and pass, if we squint and turn our heads to the side, we might be able to persuade ourselves that there is a vague analogy between what the keyword does outside of expression to what it does inside ("it returns from the expression, but not from the function" sort of thing). But I don't consider vague analogies to be a good enough reason to overload a keyword with two distinct uses. Is anyone going to propose some other, new, keyword? Although the barrier to new keywords is high, it's not impossibly high. It's worth at least considering. > I remain less than happy with the colon notation, although I will > concede that the basic > > x[3] except IndexError: 0 > > form is not horrible - it just becomes horrible very, very fast when > people try to do anything more complicated than that. Really? I think adding a second exception to the same clause is not horrible at all. x[3] except IndexError, KeyError: 0 Parens around the exception list should be permitted, but I don't think they need to be mandatory: x[3] except (IndexError, KeyError): 0 If we allow multiple except clauses -- and I am +1 on that -- then it should be recommended to put one except clause per line, indented for clarity where needed: my_values = [# existing syntax lambda a, b: (a+b)/(a*b), somelist[2:-1], {key: value for key in sequence}, some_function(x) if x else -1, # proposed new syntax (minmax(sequence) except ValueError: (-1, -1)), (str(x[3]) except IndexError, KeyError: 0, except NameError: UNDEF, except UnicodeDecodeError: '?', ), ] As you can see from my examples, using colons is hardly unheard of, and while multiple except clauses does make the except expression more complicated, the use of parens and sensible formatting allows you to digest it a bit at a time. Of course people might abuse this. But they might abuse list comps, generator expressions, ternary if, and even mathematical expressions too. In practice we solve that by slapping them with a halibut :-) and saying "don't do that". -- Steven From rosuav at gmail.com Wed Feb 19 01:48:04 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 11:48:04 +1100 Subject: [Python-ideas] except expression In-Reply-To: <20140218233408.GV4519@ando> References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <20140218233408.GV4519@ando> Message-ID: On Wed, Feb 19, 2014 at 10:34 AM, Steven D'Aprano wrote: >> I could imagine catching >> (ValueError, Unicode{En,De}codeError, AttributeError, EOFError, >> IOError, OSError, LookupError, NameError, ZeroDivisionError) and >> that's just from a quick skim of the built-in names. Would you >> consider making a tuple like that and then using "except >> CommonErrors:" all over your code? Or more to the point, if it were >> really convenient to do that, would it improve your code? > > I hope that you were making a reductio ad absurdum argument as to why > this is a bad idea. > > Exception handlers should ALWAYS catch the FEWEST errors that you KNOW > you need to handle. (Excuse my shouting, but this is important.) Pretty much. I started out saying that it doesn't need to be a superclass (thus there's no need to "infect" every exception definition with this check), but by the time I'd written out that list of stuff that would make sense to catch, it was pretty obvious that such a diverse list would make a terrible default. Also, compare this list against the first 125 lines of my stdlib conversion examples (the "easy ones", the mostly-non-controversial ones). Even ignoring the ones that are too broad for their needs, there are some that legitimately catch Exception, TypeError, StopIteration, and a custom error from the email module (which would be reasonably well handled by a subclassing solution but not really with the tuple). There's no way to restrict this without making it simultaneously dangerous AND useless. ChrisA From zuo at chopin.edu.pl Wed Feb 19 01:58:50 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Wed, 19 Feb 2014 01:58:50 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: <74600d07883badf73aa04623511abcec@chopin.edu.pl> 18.02.2014 16:56, Paul Moore wrote: > On 18 February 2014 15:05, Chris Angelico wrote: >>> Well, yes and no. There are only 2 out of the 10 syntaxes >>> originally >>> proposed in the PEP at the start of this thread that use a colon. >>> I've >>> already pointed out that I don't like a colon, so assume I said >>> "except return 0" if you must :-) >> >> Whether it's a colon or another keyword, _something_ is necessary in >> there, if it's permissible to specify an exception type: >> >> sum(x[3] except IndexError 0 for x in list_of_tuples) >> sum(x[3] except 0 for x in list_of_tuples) >> >> How would you parse each of those, without a separator? > > With "return" as a separator. I said that above. Was I not clear? IMHO it would be an abuse of the keyword. "return" is very strong signal to programmers eyes: "here we exit the body of the function" (and the function must be defined in the statement-based way: by def, not by lambda; return is not used in lambda so in an except expression it would even more alien and out of place). >> One option might be to have a separator that's present only when the >> exception type is listed. For instance: >> >> sum(x[3] except(IndexError) 0 for x in list_of_tuples) >> sum(x[3] except 0 for x in list_of_tuples) >> >> Does that sort of thing have enough support to be added to the PEP? >> I'm not sure it gives us anything over the colon notation, which has >> the benefit of being consistent with the statement form (and is >> stylistically similar to lambda, so it's not grossly inappropriate >> to >> an expression context) > > I remain less than happy with the colon notation, although I will > concede that the basic > > x[3] except IndexError: 0 > > form is not horrible - it just becomes horrible very, very fast when > people try to do anything more complicated than that. [snip] IMHO it'll not become horible if some parens are added... ...either after 'except' (my favorite variant): x[3] except (IndexError: 0) sum(x[3] except (IndexError: 0) for x in list_of_tuples) ...or before 'except': x[3] (except IndexError: 0) sum(x[3] (except IndexError: 0) for x in list_of_tuples) ...or just enclosing the whole expression: (x[3] except IndexError: 0) sum((x[3] except IndexError: 0) for x in list_of_tuples) Cheers. *j From rosuav at gmail.com Wed Feb 19 02:00:02 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 12:00:02 +1100 Subject: [Python-ideas] except expression In-Reply-To: <20140219001048.GX4519@ando> References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> Message-ID: On Wed, Feb 19, 2014 at 11:10 AM, Steven D'Aprano wrote: > for key in sequence: > x = (d1[key] except KeyError: (d2[key] except KeyError: 0)) > > Take the colons and brackets out, and I think it is less readable: > > x = d1[key] except KeyError d2[key] except KeyError 0 > > and ambiguous. > > By the way, I think this is a good example for the PEP (Chris are you > reading?). You might be tempted to re-write this as: > > x = d1.get(key, d2.get(key, 0)) > > which is shorter, but the semantics are different. If the second case, > you have to pre-calculate the fallback, which may be expensive, while in > the exception form, you only calculate the fallback if the first lookup > actually fails. I certainly am reading :) Going a bit further by having the final fall-back be a function call (highlighting the fact that it may be expensive). """ Consider this example of a two-level cache:: for key in sequence: x = (lvl1[key] except KeyError: (lvl2[key] except KeyError: f(key))) This cannot be rewritten as:: x = lvl1.get(key, lvl2.get(key, f(key))) which, despite being shorter, defeats the purpose of the cache, as it must calculate a default value to pass to get(). The .get() version calculates backwards; the exception-testing version calculates forwards, as would be expected. """ ChrisA From rosuav at gmail.com Wed Feb 19 02:08:52 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 12:08:52 +1100 Subject: [Python-ideas] except expression In-Reply-To: <64da979e889af212ba4a98c5da91a38f@chopin.edu.pl> References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <64da979e889af212ba4a98c5da91a38f@chopin.edu.pl> Message-ID: On Wed, Feb 19, 2014 at 11:11 AM, Jan Kaliszewski wrote: > IMHO bare except is practically always a very bad practice unless the > exception is immediately re-raised: > > try: > foo() > except: > logger.exception('error:') > raise > > ...and AFAIK, up to now, nobody proposed any syntax that would make it > possible to re-raise an exception in an except expression. > > Therefore I believe that bare except should *not* be allowed in > except expressions at all. Reraising inside an expression doesn't make a huge amount of sense. If you want to do something and then reraise, what's the value of the expression? Go statement-form and make it clearer. But there are legit uses of broad except. One is to use the exception itself as the value. (See Lib/imaplib.py:568, quoted in the examples.) Another is for a "this must never fail" code like repr (Lib/asyncore.py:482, the next example). Arguably both should be catching Exception, not BaseException; but the recommendation to "log it and reraise" should apply just as much to catching Exception as to bare-excepting, as an unexpected TypeError or AttributeError could indicate a significant bug. IMO the validity of bare except in an expression should be based on readability and bug-magnet possibility, not whether or not the exception can be reraised. ChrisA From rosuav at gmail.com Wed Feb 19 02:11:15 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 12:11:15 +1100 Subject: [Python-ideas] except expression In-Reply-To: <8b4fe0d54b73e5e881131998739fef61@chopin.edu.pl> References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <8b4fe0d54b73e5e881131998739fef61@chopin.edu.pl> Message-ID: On Wed, Feb 19, 2014 at 11:19 AM, Jan Kaliszewski wrote: > 18.02.2014 17:25, Paul Moore napisa?: > > >> OTOH, there's still an argument for only allowing a single exception >> name in the syntax (an "identifier" rather than an "expression" in >> syntax terms). If you must catch multiple exceptions, give the >> relevant tuple a name. > > > I believe that at this point (what the exception spec would be > allowed to be: identifier?, tuple?, any expression?) the syntax should > be identical to the statement syntax (i.e.: any expression). > > Less special cases to remember. Yes, definitely. I see little value in forcing single-exception catching. > For the same reason, I believe that tuple expressions > ("except (ValueError, TypeError)") should be obligatorily > enclosed with parens as long as they are obligatorily enclosed > with parens in the statement syntax (i.e., probably till Python 3.13 > :)). AFAIK, the only reason to mandate the parens is to specifically disallow the Py2 syntax: except Exception, e: pass If that's the case, they could be optional in the expression form, as that has no Py2 equivalent. ChrisA From rosuav at gmail.com Wed Feb 19 02:16:04 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 12:16:04 +1100 Subject: [Python-ideas] except expression In-Reply-To: <74600d07883badf73aa04623511abcec@chopin.edu.pl> References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <74600d07883badf73aa04623511abcec@chopin.edu.pl> Message-ID: On Wed, Feb 19, 2014 at 11:58 AM, Jan Kaliszewski wrote: > sum((x[3] except IndexError: 0) > for x in list_of_tuples) Really, I don't think this example is all that bad even without the parens: sum(x[3] except IndexError: 0 for x in list_of_tuples) Compare: sum(x[3] if len(x)>3 else 0 for x in list_of_tuples) which is a roughly-comparable LBLY check doing the same thing, and which doesn't need parens. If you want 'em, go ahead, put 'em in, but it's not horrible without them. ChrisA From zuo at chopin.edu.pl Wed Feb 19 02:16:01 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Wed, 19 Feb 2014 02:16:01 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: <972338288aa415df5f02c4493f0c1661@chopin.edu.pl> 18.02.2014 21:55, Chris Angelico wrote: > Script: > https://github.com/Rosuav/ExceptExpr/blob/master/find_except_expr.py > Output: > https://github.com/Rosuav/ExceptExpr/blob/master/candidates.txt > Annotated examples: > https://github.com/Rosuav/ExceptExpr/blob/master/examples.py Excellent! :) One more minor motivating case: q = queue.Queue(maxsize=10) ... item = q.get_nowait() except Full: default (in case of a list you could check its length to avoid catching exception; but Queue's usage cases are typically multi-threading -related so the "check and then get" way is not an option then) Cheers. *j From rosuav at gmail.com Wed Feb 19 02:20:03 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 12:20:03 +1100 Subject: [Python-ideas] except expression In-Reply-To: <972338288aa415df5f02c4493f0c1661@chopin.edu.pl> References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <972338288aa415df5f02c4493f0c1661@chopin.edu.pl> Message-ID: On Wed, Feb 19, 2014 at 12:16 PM, Jan Kaliszewski wrote: >> Annotated examples: >> https://github.com/Rosuav/ExceptExpr/blob/master/examples.py > > > Excellent! :) > > One more minor motivating case: > > q = queue.Queue(maxsize=10) > ... > item = q.get_nowait() except Full: default > > (in case of a list you could check its length to avoid catching > exception; but Queue's usage cases are typically multi-threading > -related so the "check and then get" way is not an option then) The examples I'm quoting there are from actual real-world code - in this case, *.py in the cpython source tree. Do you have some real-world code that does the above with a try/except, and which is available for viewing and analysis? If so, I'd be happy to add that to the examples! Otherwise, if this is just a contrived/theoretical example, I can still add it someplace, but we already have a good few of those. (BTW, should that be "except Empty"?) ChrisA From rob.cliffe at btinternet.com Wed Feb 19 02:22:17 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Wed, 19 Feb 2014 01:22:17 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> Message-ID: <53040749.6030606@btinternet.com> On 19/02/2014 01:00, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 11:10 AM, Steven D'Aprano wrote: >> for key in sequence: >> x = (d1[key] except KeyError: (d2[key] except KeyError: 0)) >> >> Take the colons and brackets out, and I think it is less readable: >> >> x = d1[key] except KeyError d2[key] except KeyError 0 >> >> and ambiguous. >> >> By the way, I think this is a good example for the PEP (Chris are you >> reading?). You might be tempted to re-write this as: >> >> x = d1.get(key, d2.get(key, 0)) >> >> which is shorter, but the semantics are different. If the second case, >> you have to pre-calculate the fallback, which may be expensive, while in >> the exception form, you only calculate the fallback if the first lookup >> actually fails. Yo! I actually mentioned this way of rewriting it, but I missed that the semantics were different and more expensive. Long live this proposal! (Although to carp, I think that the colons are desirable but the brackets are not.) > I certainly am reading :) Going a bit further by having the final > fall-back be a function call (highlighting the fact that it may be > expensive). > > """ > Consider this example of a two-level cache:: > for key in sequence: > x = (lvl1[key] except KeyError: (lvl2[key] except KeyError: f(key))) > > This cannot be rewritten as:: > x = lvl1.get(key, lvl2.get(key, f(key))) > > which, despite being shorter, defeats the purpose of the cache, as it must > calculate a default value to pass to get(). The .get() version calculates > backwards; the exception-testing version calculates forwards, as would be > expected. > """ > > ChrisA Again, yo! Surely we are on to a good thing here with this proposal? Rob Cliffe > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > ----- > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6604 - Release Date: 02/18/14 > > From rob.cliffe at btinternet.com Wed Feb 19 02:25:23 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Wed, 19 Feb 2014 01:25:23 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: <53040803.9090003@btinternet.com> Well done, Chris! (Some poor sod does all the hard work while the rest of us chip in our 2c as and when we feel like it, making more work for him. A bit like bear-baiting.) My first thoughts are similar to Paul's: The shorter the examples, the more convincing they are. Dramatically so. And you say "there are a huge number" of them. I think this shows the value of the proposal. Except that I also _love_ the 2-line examples that line up (short or not, mostly not). (I'm a big fan of lining things up, so I will write 2, 3 or 4 statements on a line when they show a consistent pattern. I believe it makes the code much more comprehensible, and much easier to spot typos. Too bad if multiple statements on a line is almost a taboo for the rest of you plonkers - I know I'm right and the rest of the planet is wrong. ) Whereas when the new form is multi-line it seems harder to understand than the original (though this may be partly because the new syntax is simply unfamiliar. Writing the target variable only once is a good thing as far as it goes, but that is not very far). I have to say I'm not keen on the cases where you introduce an extra contraction, simply because it's possible (although the fact that the proposal makes it possible is sort of a point in its favour, in a perverted way). E.g. given try: netloc_enc = netloc.encode("ascii") except UnicodeEncodeError: netloc_enc = netloc.encode("idna") self.putheader('Host', netloc_enc) which would contract (consistently with the simpler examples) to: netloc_enc = netloc.encode("ascii") except UnicodeEncodeError:netloc.encode("idna") self.putheader('Host', netloc_enc) you go one step further and write self.putheader('Host', netloc.encode("ascii") except UnicodeEncodeError: netloc.encode("idna") ) I confess I have the (bad) habit of writing "slick" (i.e. as few statements as possible) code like this myself (probably to try to show how clever I am ). But I know in my heart that it's slightly harder to write and significantly harder to understand and maintain, and the performance advantage _if any_ is negligible. It really is a mistake to put too much content into one statement, and I try to fight my tendency to do it. Thanks again, Rob Cliffe On 18/02/2014 20:55, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 3:06 AM, Chris Angelico wrote: >> I'm currently working on finding a bunch of examples from the stdlib >> that could be translated. Will post them once I get the script sorted >> out. As it's currently 3AM, though, that means I need to fry up some >> bacon or something before I continue :) > Alright, results are in. > > Script: > https://github.com/Rosuav/ExceptExpr/blob/master/find_except_expr.py > Output: > https://github.com/Rosuav/ExceptExpr/blob/master/candidates.txt > Annotated examples: > https://github.com/Rosuav/ExceptExpr/blob/master/examples.py > > The last one is the most readable. I've collected up a bunch of viable > candidates for translation. (Note that I'm not advocating going > through and editing these files for no other reason. That'd just be > code churn. But these are cases where, had this feature existed when > that code was written, it could have been taken advantage of.) > > My script is currently _very_ simplistic. It looks *only* for > assignments, so it won't find something like this: > > try: > foo.append(args[0]) > except IndexError: > foo.append('') > > which is correct, because it's impossible to know whether foo.append() > will raise IndexError. (Chances are it won't, especially if we know > foo is a list, for instance.) It's not safe to directly translate that > sort of thing. It might, however, be something worth improving, as it > narrows the scope of the except clause. But if that same code is > written thus: > > try: > tmp = args[0] > except IndexError: > tmp = '' > foo.append(tmp) > > then the script _will_ find it, and then it really is a candidate for editing. > > Point to note: Apart from one instance, where it wasn't being used > anyway, I found not a single instance of 'as' being used. There was > one case where sys.exc_info() was referenced, though, so this may just > mean that the stdlib files in question are maintaining compatibility > with old versions of Python. > > I didn't look at most of the tests. The script found 195 plausible > try/except blocks, of which 37 have the word "test" in the name;that > leaves 158 that are likely to benefit from this change. There are a > couple of false positives, but not many. > > Next is to figure out what else is a candidate for editing. Here's my > current criteria, straight from the docstring: > > """Report on 'simple' try/except statements. > > The definition of simple is: > 1. No else or finally clause > 2. Only one except clause (currently) > 3. Exactly one statement in the try clause > 4. Exactly one statement in each except clause > 5. Each of those statements is the same type. > 6. That type is one that could be an expression. > 7. Those statements are all compatible. > > The last two are the trickiest. Currently I'm looking > only for assignments, where both try and except assign > to the same target. This is, however, too narrow.""" > > Interestingly, removing criterion 2 gives us three additional examples > out of the test suite, and nothing else. There are no cases outside of > the test suite that look like this: > > try: > x = some_calculation > except ValueError: > x = something_else > except OverflowError: > x = different_thing > > (The test suite has one case with those exact two exceptions, and a > pair that use OverflowError and ZeroDivisionError. In each case, > they're comparing two actions to make sure they give the same result, > where "throwing OverflowError" is a result like any other.) > > Conclusions: The need for chained exception catching might not be so > great after all, and even the 'as' keyword isn't as (heh heh) > important as I thought it was. > > Alternate conclusion: My sample is too small. Need more. Data, you > have the helm. > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct:http://python.org/psf/codeofconduct/ > > > ----- > No virus found in this message. > Checked by AVG -www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6604 - Release Date: 02/18/14 > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rob.cliffe at btinternet.com Wed Feb 19 02:33:30 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Wed, 19 Feb 2014 01:33:30 +0000 Subject: [Python-ideas] except expression In-Reply-To: <20140219004357.GY4519@ando> References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140219004357.GY4519@ando> Message-ID: <530409EA.3060103@btinternet.com> On 19/02/2014 00:43, Steven D'Aprano wrote: > On Tue, Feb 18, 2014 at 03:56:10PM +0000, Paul Moore wrote: >> On 18 February 2014 15:05, Chris Angelico wrote: >>>> Well, yes and no. There are only 2 out of the 10 syntaxes originally >>>> proposed in the PEP at the start of this thread that use a colon. I've >>>> already pointed out that I don't like a colon, so assume I said >>>> "except return 0" if you must :-) >>> Whether it's a colon or another keyword, _something_ is necessary in >>> there, if it's permissible to specify an exception type: >>> >>> sum(x[3] except IndexError 0 for x in list_of_tuples) >>> sum(x[3] except 0 for x in list_of_tuples) >>> >>> How would you parse each of those, without a separator? >> With "return" as a separator. I said that above. Was I not clear? > Yes you were clear, but this is a huge complicated thread and if Chris > is the least bit like me, he's probably feeling a bit overloaded :-) > > Using return to not actually return seems completely wrong to me. This > is the problem with all the proposals to overload some arbitrary keyword > *just because it is available*. To me, all of these are equally as > arbitrary and silly: > > expression except return 42 > expression except pass 42 > expression except del 42 > expression except import 42 > expression except class 42 I agree. For me, "return" would be the least indigestible - at least it suggests getting a value and parking it somewhere. My support for the colon is based on (1) its conciseness (2) its natural meaning in my mind, something like "here follows" (3) a somewhat fanatical hankering for consistency - there is a colon at the end of an "except" statement. (Although actually I would prefer that "if", "while", "try", "except" etc. statements _didn't_ require a colon - I still forget to put them in sometimes. Not sure where that leaves me.) > > For each keyword in (return, pass, del, import, ...), we are attempting > to overload it to mean something different inside an except expression > to what it does outside of one. > > With return and pass, if we squint and turn our heads to the side, we > might be able to persuade ourselves that there is a vague analogy > between what the keyword does outside of expression to what it does > inside ("it returns from the expression, but not from the function" sort > of thing). But I don't consider vague analogies to be a good enough > reason to overload a keyword with two distinct uses. > > Is anyone going to propose some other, new, keyword? Although the > barrier to new keywords is high, it's not impossibly high. It's worth at > least considering. I confess I don't understand what the issues are that makes the barrier so high, and I willingly defer to those who know more about it than I do. But if we admit the possibility, "then" is the best candidate I can think of: x = d[y] except KeyError then z > > >> I remain less than happy with the colon notation, although I will >> concede that the basic >> >> x[3] except IndexError: 0 >> >> form is not horrible - it just becomes horrible very, very fast when >> people try to do anything more complicated than that. > Really? I think adding a second exception to the same clause is not > horrible at all. > > x[3] except IndexError, KeyError: 0 > > Parens around the exception list should be permitted, but I don't think > they need to be mandatory: > > x[3] except (IndexError, KeyError): 0 You strike a chord with me, if only because I spend half my time with a language that requires parentheses round everything, ergo I'm biased. But more seriously, Python generally does not require parentheses unless there's a very good reason. > > If we allow multiple except clauses -- and I am +1 on that -- then it > should be recommended to put one except clause per line, indented for > clarity where needed: > > > my_values = [# existing syntax > lambda a, b: (a+b)/(a*b), > somelist[2:-1], > {key: value for key in sequence}, > some_function(x) if x else -1, > # proposed new syntax > (minmax(sequence) except ValueError: (-1, -1)), > (str(x[3]) except IndexError, KeyError: 0, > except NameError: UNDEF, > except UnicodeDecodeError: '?', > ), > ] > > > As you can see from my examples, using colons is hardly unheard of, and > while multiple except clauses does make the except expression more > complicated, the use of parens and sensible formatting allows you to > digest it a bit at a time. > > Of course people might abuse this. But they might abuse list comps, > generator expressions, ternary if, and even mathematical expressions > too. In practice we solve that by slapping them with a halibut :-) and > saying "don't do that". > > > I pretty much agree with everything you have said in this post. (Kinda nice, as we had a violent disagreement in the past. :-) ) Rob Cliffe -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.belopolsky at gmail.com Wed Feb 19 02:34:25 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 18 Feb 2014 20:34:25 -0500 Subject: [Python-ideas] except expression In-Reply-To: <20140219001048.GX4519@ando> References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> Message-ID: On Tue, Feb 18, 2014 at 7:10 PM, Steven D'Aprano wrote: > > I disagree. It is best to leave explicit selection of exceptions to > catch > > outside of the expressions. This is job for things like decimal or numpy > > fp contexts. > > I don't understand what relevance fp contexts have here. >>> numpy.seterr(divide='ignore') {'over': 'warn', 'divide': 'raise', 'invalid': 'warn', 'under': 'ignore'} >>> 1/numpy.array(0.0) inf >>> numpy.seterr(divide='raise') {'over': 'warn', 'divide': 'ignore', 'invalid': 'warn', 'under': 'ignore'} >>> 1/numpy.array(0.0) Traceback (most recent call last): File "", line 1, in FloatingPointError: divide by zero encountered in divide -------------- next part -------------- An HTML attachment was scrubbed... URL: From zuo at chopin.edu.pl Wed Feb 19 02:37:29 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Wed, 19 Feb 2014 02:37:29 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <64da979e889af212ba4a98c5da91a38f@chopin.edu.pl> Message-ID: <8f563e4e0298c73e55419ff9c4fb76d2@chopin.edu.pl> 19.02.2014 02:08, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 11:11 AM, Jan Kaliszewski > wrote: >> IMHO bare except is practically always a very bad practice unless >> the >> exception is immediately re-raised: >> >> try: >> foo() >> except: >> logger.exception('error:') >> raise >> >> ...and AFAIK, up to now, nobody proposed any syntax that would make >> it >> possible to re-raise an exception in an except expression. >> >> Therefore I believe that bare except should *not* be allowed in >> except expressions at all. > > Reraising inside an expression doesn't make a huge amount of sense. > If > you want to do something and then reraise, what's the value of the > expression? Go statement-form and make it clearer. Exactly. That's why I believe bare except should be disallowed in the expression form. > But there are legit uses of broad except. Apart from immediately-re-raising, base except are practically never a legit use IMHO. > One is to use the exception > itself as the value. (See Lib/imaplib.py:568, quoted in the > examples.) > Another is for a "this must never fail" code like repr > (Lib/asyncore.py:482, the next example). Arguably both should be > catching Exception, not BaseException; I believe they should indeed. > but the recommendation to "log > it and reraise" should apply just as much to catching Exception as to > bare-excepting, as an unexpected TypeError or AttributeError could > indicate a significant bug. "Log it" -- yes. "Reraise" -- not necessarily for Exception-based ones, if the are logged, IMHO (see sources of logging, asyncio, tornado...). > IMO the validity of bare except in an expression should be based on > readability and bug-magnet possibility, not whether or not the > exception can be reraised. Bare except statement is indeed a bug magnet, but: * it's a legacy that cannot be removed from the language easily, * at least there are cases when it is valid and also seems to be somewhat elegant: except: # *bare* except ... raise # *bare* raise On the other hand, bare except expression would be a bug magnet, without any legit use cases on horizon. And, I suppose, the magnet would be even stronger -- expecially for keystroke-saving-lovers. :) Let's do not introduce an attractive nuisance whose positive value is near zero. Cheers. *j From zuo at chopin.edu.pl Wed Feb 19 02:40:38 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Wed, 19 Feb 2014 02:40:38 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <8b4fe0d54b73e5e881131998739fef61@chopin.edu.pl> Message-ID: 19.02.2014 02:11, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 11:19 AM, Jan Kaliszewski > wrote: >> 18.02.2014 17:25, Paul Moore napisa?: >> >> >>> OTOH, there's still an argument for only allowing a single >>> exception >>> name in the syntax (an "identifier" rather than an "expression" in >>> syntax terms). If you must catch multiple exceptions, give the >>> relevant tuple a name. >> >> >> I believe that at this point (what the exception spec would be >> allowed to be: identifier?, tuple?, any expression?) the syntax >> should >> be identical to the statement syntax (i.e.: any expression). >> >> Less special cases to remember. > > Yes, definitely. I see little value in forcing single-exception > catching. > >> For the same reason, I believe that tuple expressions >> ("except (ValueError, TypeError)") should be obligatorily >> enclosed with parens as long as they are obligatorily enclosed >> with parens in the statement syntax (i.e., probably till Python 3.13 >> :)). > > AFAIK, the only reason to mandate the parens is to specifically > disallow the Py2 syntax: > > except Exception, e: > pass > > If that's the case, they could be optional in the expression form, as > that has no Py2 equivalent. But then you must remember: in expression yes, in statement no; + additional trouble when you refactor transforming the former to the latter... Cheers. *j From zuo at chopin.edu.pl Wed Feb 19 02:48:24 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Wed, 19 Feb 2014 02:48:24 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <972338288aa415df5f02c4493f0c1661@chopin.edu.pl> Message-ID: 19.02.2014 02:20, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 12:16 PM, Jan Kaliszewski > wrote: >>> Annotated examples: >>> https://github.com/Rosuav/ExceptExpr/blob/master/examples.py >> >> >> Excellent! :) >> >> One more minor motivating case: >> >> q = queue.Queue(maxsize=10) >> ... >> item = q.get_nowait() except Full: default >> >> (in case of a list you could check its length to avoid catching >> exception; but Queue's usage cases are typically multi-threading >> -related so the "check and then get" way is not an option then) > > The examples I'm quoting there are from actual real-world code - in > this case, *.py in the cpython source tree. Do you have some > real-world code that does the above with a try/except, and which is > available for viewing and analysis? If so, I'd be happy to add that > to > the examples! Then I was writing that it seemed to me that I encountered such cases in some code I saw not so long ago. But I don't remember the details and indeed am not sure. So -- nevermind. :) > Otherwise, if this is just a contrived/theoretical example, I can > still add it someplace, but we already have a good few of those. > > (BTW, should that be "except Empty"?) Yes, it should. Sorry, it's 2:46 *a.m.* here... You just caught my TooLateNightError :) Cheers. *j From rob.cliffe at btinternet.com Wed Feb 19 02:52:30 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Wed, 19 Feb 2014 01:52:30 +0000 Subject: [Python-ideas] except expression In-Reply-To: <5303F03A.9080801@stoneleaf.us> References: <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> Message-ID: <53040E5E.5020901@btinternet.com> On 18/02/2014 23:43, Ethan Furman wrote: > On 02/18/2014 03:27 PM, Steven D'Aprano wrote: >> >> -1 on bare excepts in the expression form. > > Agreed. > > -- > ~Ethan~ I'm prepared to concede this point (as if my opinion as a very junior member of the Python community counted for much :-) ). Nick Coghlan and Steven D'Aprano made a very strong case in their posts. My only real objection is that I would like consistency between "except" statements and "except" expressions. But I suspect that you (plural), if you had the chance to rewrite history, would also want to disallow a bare "except" statement. (Would you?) Nick, Steven, Ethan (and anybody else), are you willing to answer this question? No offence intended, but I'm curious. No, I should be honest: I'm hoping to score a debating point. Best wishes, Rob Cliffe > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > ----- > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6604 - Release Date: 02/18/14 > > From zuo at chopin.edu.pl Wed Feb 19 02:56:58 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Wed, 19 Feb 2014 02:56:58 +0100 Subject: [Python-ideas] except expression In-Reply-To: <5303D3BB.1000709@canterbury.ac.nz> References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303D3BB.1000709@canterbury.ac.nz> Message-ID: 18.02.2014 22:42, Greg Ewing wrote: > Paul Moore wrote: >> I could argue that >> "bare except" is more acceptable in an *expression* context because >> expressions should never be so complex that the reasons why it's bad >> in a statement context actually apply. > > The expression itself doesn't have to be complicated. > Even a simple expression such as x[i] can invoke arbitrary > code, and therefore raise arbitrary exceptions, including > ones that represent bugs and therefore shouldn't be > silenced. And including KeyboardInterrupt which, with the default SIGINT handler, can be raised *at any point*, beetween any bytecode instructions (what is a serious trouble for critical with|finally-based cleanups but it's another story). Cheers. *j From rob.cliffe at btinternet.com Wed Feb 19 02:59:42 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Wed, 19 Feb 2014 01:59:42 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <8b4fe0d54b73e5e881131998739fef61@chopin.edu.pl> Message-ID: <5304100E.8070207@btinternet.com> On 19/02/2014 01:40, Jan Kaliszewski wrote: > 19.02.2014 02:11, Chris Angelico wrote: > >> On Wed, Feb 19, 2014 at 11:19 AM, Jan Kaliszewski >> wrote: >>> 18.02.2014 17:25, Paul Moore napisa?: >>> >>> >>>> OTOH, there's still an argument for only allowing a single exception >>>> name in the syntax (an "identifier" rather than an "expression" in >>>> syntax terms). If you must catch multiple exceptions, give the >>>> relevant tuple a name. >>> >>> >>> I believe that at this point (what the exception spec would be >>> allowed to be: identifier?, tuple?, any expression?) the syntax should >>> be identical to the statement syntax (i.e.: any expression). >>> >>> Less special cases to remember. >> >> Yes, definitely. I see little value in forcing single-exception >> catching. Uh yes. I thought/hoped the debate had got past this point. >> >>> For the same reason, I believe that tuple expressions >>> ("except (ValueError, TypeError)") should be obligatorily >>> enclosed with parens as long as they are obligatorily enclosed >>> with parens in the statement syntax (i.e., probably till Python 3.13 >>> :)). >> I think I agree on grounds of (sorry if I'm becoming a bore, but you guessed it!) Consistency! But I don't see this as a critical issue, particularly as Python 3.13 is probably a few weeks away :-) . I think putting parens around a list of exceptions would be good style in any case. Rob Cliffe >> AFAIK, the only reason to mandate the parens is to specifically >> disallow the Py2 syntax: >> >> except Exception, e: >> pass >> >> If that's the case, they could be optional in the expression form, as >> that has no Py2 equivalent. > > But then you must remember: in expression yes, in statement no; > + additional trouble when you refactor transforming the former to > the latter... > > Cheers. > *j > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > ----- > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6604 - Release Date: 02/18/14 From zuo at chopin.edu.pl Wed Feb 19 03:04:32 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Wed, 19 Feb 2014 03:04:32 +0100 Subject: [Python-ideas] except expression In-Reply-To: <53040E5E.5020901@btinternet.com> References: <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> Message-ID: 19.02.2014 02:52, Rob Cliffe wrote: [...] > I would like consistency between > "except" statements and "except" expressions. Me too -- *except* the bare except syntax which, in case of except expression, would be a serious nuisance without any real advantage. (IMHO) Cheers. *j From rosuav at gmail.com Wed Feb 19 03:09:29 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 13:09:29 +1100 Subject: [Python-ideas] except expression In-Reply-To: <53040803.9090003@btinternet.com> References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <53040803.9090003@btinternet.com> Message-ID: On Wed, Feb 19, 2014 at 12:25 PM, Rob Cliffe wrote: > Well done, Chris! (Some poor sod does all the hard work while the rest of > us chip in our 2c as and when we feel like it, making more work for him. A > bit like bear-baiting.) It may be more work for me, but it's also the validation of that work. This isn't a case of "little me against the throng, I'm only one and possibly I'm wrong" [1] - it's something that's generating several hundred emails' worth of interest. And once it quiets down here, I can post to python-dev and start it all over again :) > The shorter the examples, the more convincing they are. Dramatically so. > And you say "there are a huge number" of them. I think this shows the value > of the proposal. Yep. I'd guess that there are over a hundred examples in the stdlib that would become one-liners with this proposal. Quite a few of them have long assignment targets like _CONFIG_VARS['abiflags'] (and there was a longer one, but I don't have it handy), so the expression form encourages DRY. > Too > bad if multiple statements on a line is almost a taboo for the rest of you > plonkers - I know I'm right and the rest of the planet is wrong. I'll join you on the 'right' side of the planet :) If putting two statements on a line makes it easier to spot bugs, I'll do it. > I have to say I'm not keen on the cases where you introduce an extra > contraction, simply because it's possible (although the fact that the > proposal makes it possible is sort of a point in its favour, in a perverted > way). E.g. given > > try: > netloc_enc = netloc.encode("ascii") > except UnicodeEncodeError: > &nb > sp; netloc_enc = netloc.encode("idna") > self.putheader('Host', netloc_enc) > > > which would contract (consistently with the simpler examples) to: > > netloc_enc = netloc.encode("ascii") except > UnicodeEncodeError: netloc.encode("idna") > self.putheader('Host', > netloc_enc) > > you go one step further and write > > self.putheader('Host', > netloc.encode("ascii") except UnicodeEncodeError: > netloc.encode("idna") > ) This is where it gets strongly debatable. Obviously, since this is an expression, it can be used in any context where an expression is valid, not just an assignment. However, part of the issue may be that my script was looking only for assignments. A lot of cases were clearly worth mentioning, some others are doubtful, a number of them just not worth the hassle. I'd like some good non-assignment examples. ChrisA [1] Theresa's song from The Mountebanks: http://math.boisestate.edu/gas/gilbert/plays/mountebanks/webopera/mount_07.html From rosuav at gmail.com Wed Feb 19 03:17:59 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 13:17:59 +1100 Subject: [Python-ideas] except expression In-Reply-To: <530409EA.3060103@btinternet.com> References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140219004357.GY4519@ando> <530409EA.3060103@btinternet.com> Message-ID: On Wed, Feb 19, 2014 at 12:33 PM, Rob Cliffe wrote: > > I agree. For me, "return" would be the least indigestible - at least it > suggests getting a value and parking it somewhere. My support for the colon > is based on (1) its conciseness (2) its natural meaning in my mind, > something like "here follows" (3) a somewhat fanatical hankering for > consistency - there is a colon at the end of an "except" statement. > (Although actually I would prefer that "if", "while", "try", "except" etc. > statements didn't require a colon - I still forget to put them in sometimes. > Not sure where that leaves me.) Compare all of these: def f(x): return x + 2 f = lambda x: x + 2 def f(x): try: return x+2 except TypeError: return x f = lambda x: x+2 except TypeError: x In each case, the expression form implicitly "gives back" an expression after a colon, while the statement form implicitly goes and executes the statement after the colon. Does that make reasonable sense? "Here we are executing statements. Here, go execute that suite after that colon." vs "Here we are evaluating an expression. Here, go evaluate that expression after that colon." > I confess I don't understand what the issues are that makes the barrier so > high, and I willingly defer to those who know more about it than I do. But > if we admit the possibility, "then" is the best candidate I can think of: > > x = d[y] except KeyError then z The biggest barrier to adding a keyword is code that uses that word as an identifier. Imagine if "id" were changed from a built-in function (which can be shadowed) to a keyword (which can't). Every piece of databasing code that stores the primary key in "id" would be broken, as would innumerable other codebases. So the trick is to find something that makes really REALLY good sense in the context it's being used in, and really bad sense as an identifier. Currently, the PEP mentions "then", "use", and "when", and my preference would be in that order. I suspect "then" is unlikely to be used as a name in as many places as "when" will be, but I can't be sure without actually searching the code of the world. ChrisA From rosuav at gmail.com Wed Feb 19 03:20:32 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 13:20:32 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> Message-ID: On Wed, Feb 19, 2014 at 12:34 PM, Alexander Belopolsky wrote: > On Tue, Feb 18, 2014 at 7:10 PM, Steven D'Aprano > wrote: >> >> > I disagree. It is best to leave explicit selection of exceptions to >> > catch >> > outside of the expressions. This is job for things like decimal or >> > numpy >> > fp contexts. >> >> I don't understand what relevance fp contexts have here. > > >>>> numpy.seterr(divide='ignore') > {'over': 'warn', 'divide': 'raise', 'invalid': 'warn', 'under': 'ignore'} >>>> 1/numpy.array(0.0) > inf >>>> numpy.seterr(divide='raise') > {'over': 'warn', 'divide': 'ignore', 'invalid': 'warn', 'under': 'ignore'} >>>> 1/numpy.array(0.0) > Traceback (most recent call last): > File "", line 1, in > FloatingPointError: divide by zero encountered in divide > That's solving the same problem in a slightly different way. Instead of quickly trapping one exception right here, you set a state flag that controls them globally. In a computationally-heavy program, that's going to work out far FAR cleaner than this; but suppose most of the time you want them to raise, and then you have one little calculation in the middle where you don't. That's where this would come in handy. ChrisA From rosuav at gmail.com Wed Feb 19 03:26:29 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 13:26:29 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <8b4fe0d54b73e5e881131998739fef61@chopin.edu.pl> Message-ID: On Wed, Feb 19, 2014 at 12:40 PM, Jan Kaliszewski wrote: >> AFAIK, the only reason to mandate the parens is to specifically >> disallow the Py2 syntax: >> >> except Exception, e: >> pass >> >> If that's the case, they could be optional in the expression form, as >> that has no Py2 equivalent. > > > But then you must remember: in expression yes, in statement no; > + additional trouble when you refactor transforming the former to > the latter... Perhaps. But I could imagine the need for parens being weakened in a future version. Let's say 3.4/3.5 is billed as the primary target for migration from 2.7, and that after 3.7, Python 3 will fly free and be itself without worrying too much about how hard it is to port. (That quite probably won't happen, but let's pretend.) In that case, 3.8 would be allowed to relax that restriction, which would then bring the statement and expression forms in line. Alternatively, the expression form could simply have the same arbitrary requirement, just for consistency, and they could both lose it at once... or the expression form could technically not require the parens, but style guides recommend using them anyway, in case you need to change it to a statement. I'm generally not a fan of making parens mandatory. Let the style guides argue that out. ChrisA From python at mrabarnett.plus.com Wed Feb 19 03:50:41 2014 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 19 Feb 2014 02:50:41 +0000 Subject: [Python-ideas] except expression In-Reply-To: <74600d07883badf73aa04623511abcec@chopin.edu.pl> References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <74600d07883badf73aa04623511abcec@chopin.edu.pl> Message-ID: <53041C01.2010902@mrabarnett.plus.com> On 2014-02-19 00:58, Jan Kaliszewski wrote: > 18.02.2014 16:56, Paul Moore wrote: > >> On 18 February 2014 15:05, Chris Angelico wrote: >>>> Well, yes and no. There are only 2 out of the 10 syntaxes >>>> originally >>>> proposed in the PEP at the start of this thread that use a colon. >>>> I've >>>> already pointed out that I don't like a colon, so assume I said >>>> "except return 0" if you must :-) >>> >>> Whether it's a colon or another keyword, _something_ is necessary in >>> there, if it's permissible to specify an exception type: >>> >>> sum(x[3] except IndexError 0 for x in list_of_tuples) >>> sum(x[3] except 0 for x in list_of_tuples) >>> >>> How would you parse each of those, without a separator? >> >> With "return" as a separator. I said that above. Was I not clear? > > IMHO it would be an abuse of the keyword. "return" is very strong > signal to programmers eyes: "here we exit the body of the function" > (and the function must be defined in the statement-based way: by def, > not by lambda; return is not used in lambda so in an except > expression it would even more alien and out of place). > >>> One option might be to have a separator that's present only when the >>> exception type is listed. For instance: >>> >>> sum(x[3] except(IndexError) 0 for x in list_of_tuples) >>> sum(x[3] except 0 for x in list_of_tuples) >>> >>> Does that sort of thing have enough support to be added to the PEP? >>> I'm not sure it gives us anything over the colon notation, which has >>> the benefit of being consistent with the statement form (and is >>> stylistically similar to lambda, so it's not grossly inappropriate >>> to >>> an expression context) >> >> I remain less than happy with the colon notation, although I will >> concede that the basic >> >> x[3] except IndexError: 0 >> >> form is not horrible - it just becomes horrible very, very fast when >> people try to do anything more complicated than that. > [snip] > > IMHO it'll not become horible if some parens are added... > > ...either after 'except' (my favorite variant): > > x[3] except (IndexError: 0) > > sum(x[3] except (IndexError: 0) > for x in list_of_tuples) > -1 That would preclude a tuple of exceptions. > ...or before 'except': > > x[3] (except IndexError: 0) > > sum(x[3] (except IndexError: 0) > for x in list_of_tuples) > -1 That looks like a call. > ...or just enclosing the whole expression: > > (x[3] except IndexError: 0) > > sum((x[3] except IndexError: 0) > for x in list_of_tuples) > No complaints there! From rosuav at gmail.com Wed Feb 19 03:53:21 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 13:53:21 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> Message-ID: On Wed, Feb 19, 2014 at 1:04 PM, Jan Kaliszewski wrote: > Me too -- *except* the bare except syntax which, in case > of except expression, would be a serious nuisance without > any real advantage. (IMHO) Jan, and everyone else who's expressed opinions on the use of bare except: https://raw2.github.com/Rosuav/ExceptExpr/master/pep-0463.txt (note that the file name and URL changed when a PEP number was assigned) I've added two sections, one in "Open Issues" and the other in "Rejected sub-proposals", on this topic. Have I covered the salient points? ChrisA From python at mrabarnett.plus.com Wed Feb 19 03:55:22 2014 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 19 Feb 2014 02:55:22 +0000 Subject: [Python-ideas] except expression In-Reply-To: <8f563e4e0298c73e55419ff9c4fb76d2@chopin.edu.pl> References: <52FE969D.5000002@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <64da979e889af212ba4a98c5da91a38f@chopin.edu.pl> <8f563e4e0298c73e55419ff9c4fb76d2@chopin.edu.pl> Message-ID: <53041D1A.9010103@mrabarnett.plus.com> On 2014-02-19 01:37, Jan Kaliszewski wrote: > 19.02.2014 02:08, Chris Angelico wrote: > >> On Wed, Feb 19, 2014 at 11:11 AM, Jan Kaliszewski >> wrote: >>> IMHO bare except is practically always a very bad practice unless >>> the >>> exception is immediately re-raised: >>> >>> try: >>> foo() >>> except: >>> logger.exception('error:') >>> raise >>> >>> ...and AFAIK, up to now, nobody proposed any syntax that would make >>> it >>> possible to re-raise an exception in an except expression. >>> >>> Therefore I believe that bare except should *not* be allowed in >>> except expressions at all. >> >> Reraising inside an expression doesn't make a huge amount of sense. >> If >> you want to do something and then reraise, what's the value of the >> expression? Go statement-form and make it clearer. > > Exactly. That's why I believe bare except should be disallowed in the > expression form. > >> But there are legit uses of broad except. > > Apart from immediately-re-raising, base except are practically never > a legit use IMHO. > +1 On the one hand, allowing a bare except would be consistent with the statement form. On the other hand, without the ability to re-raise, it's just asking for trouble, although there _is_ that "consenting adults" thing! :-) From rosuav at gmail.com Wed Feb 19 04:05:09 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 14:05:09 +1100 Subject: [Python-ideas] except expression In-Reply-To: <53041D1A.9010103@mrabarnett.plus.com> References: <52FE969D.5000002@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <64da979e889af212ba4a98c5da91a38f@chopin.edu.pl> <8f563e4e0298c73e55419ff9c4fb76d2@chopin.edu.pl> <53041D1A.9010103@mrabarnett.plus.com> Message-ID: On Wed, Feb 19, 2014 at 1:55 PM, MRAB wrote: > On the one hand, allowing a bare except would be consistent with the > statement form. > > On the other hand, without the ability to re-raise, it's just asking > for trouble, although there _is_ that "consenting adults" thing! :-) Regardless of the ability to re-raise, I wouldn't be against disallowing a bare except, and insisting that it be spelled "except BaseException:" instead. The main argument against that is consistency; in fact, possibly the *only* viable argument against that. Obviously backward compatibility is a strong reason to keep support in the statement form, but how important is it to be consistent with something that's strongly discouraged anyway? Hmm. Actually, how strongly *is* the bare except discouraged? There are a good lot of them in the stdlib, and quite a few probably should be "except Exception" anyway. Currently, PEP 8 permits two use-cases (log and continue, and clean-up and reraise), but then maybe discourages one of them. Core devs, what's your opinion on new code with "except:" in it? Would you prefer to see it spelled "except BaseException:"? ChrisA From steve at pearwood.info Wed Feb 19 04:25:33 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 19 Feb 2014 14:25:33 +1100 Subject: [Python-ideas] except expression In-Reply-To: <53040E5E.5020901@btinternet.com> References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> Message-ID: <20140219032533.GA1458@ando> On Wed, Feb 19, 2014 at 01:52:30AM +0000, Rob Cliffe wrote: > > On 18/02/2014 23:43, Ethan Furman wrote: > >On 02/18/2014 03:27 PM, Steven D'Aprano wrote: > >> > >>-1 on bare excepts in the expression form. > > > >Agreed. > > > >-- > >~Ethan~ > I'm prepared to concede this point (as if my opinion as a very junior > member of the Python community counted for much :-) ). Nick Coghlan and > Steven D'Aprano made a very strong case in their posts. > My only real objection is that I would like consistency between "except" > statements and "except" expressions. > But I suspect that you (plural), if you had the chance to rewrite > history, would also want to disallow a bare "except" statement. (Would you?) > Nick, Steven, Ethan (and anybody else), are you willing to answer this > question? > No offence intended, but I'm curious. No, I should be honest: I'm > hoping to score a debating point. No offence taken. I think bare excepts are a bug magnet and violate "Explicit is better than implicit", and I would like to see them gone, even though they do have a use at the interactive interpreter for lazy people. Including me. py> try: this() ... except: None Yes, I sometimes use that sort of quick-and-dirty construction, but if the language didn't have it, I wouldn't mind writing Exception after the except. Well, maybe a tiny bit. But not enough to complain. I think bare excepts were useful back in ancient days when you could raise strings, and they were caught by identity not value. But I think there is strong evidence that this was a mistake: raising strings was removed in Python 2.6, rather than waiting for Python 3000. I expect that most people don't even remember that you ever could write things like raise "This is an error" and thank goodness for that :-) -- Steven From rosuav at gmail.com Wed Feb 19 04:37:25 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 14:37:25 +1100 Subject: [Python-ideas] except expression In-Reply-To: <20140219032533.GA1458@ando> References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> Message-ID: On Wed, Feb 19, 2014 at 2:25 PM, Steven D'Aprano wrote: > I think bare excepts were useful back in ancient days when you > could raise strings, and they were caught by identity not value. But I > think there is strong evidence that this was a mistake: raising strings > was removed in Python 2.6, rather than waiting for Python 3000. I expect > that most people don't even remember that you ever could write things > like > > raise "This is an error" > > and thank goodness for that :-) They were caught by identity? Ouch. Definite bug magnet. In that case, yeah, you'd normally want to just put a bare "except:" and then decide what to do with the exception... only... a bare except can't capture the thing thrown, so how do you decide? IMO having to turn to sys.exc_info is wrong; you should be able to work within the construct of the try/except block. ChrisA From abarnert at yahoo.com Wed Feb 19 06:06:09 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 18 Feb 2014 21:06:09 -0800 (PST) Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> Message-ID: <1392786369.4527.YahooMailNeo@web181006.mail.ne1.yahoo.com> From: Chris Angelico Sent: Tuesday, February 18, 2014 7:37 PM > On Wed, Feb 19, 2014 at 2:25 PM, Steven D'Aprano > wrote: >> I think bare excepts were useful back in ancient days when you >> could raise strings, and they were caught by identity not value. But I >> think there is strong evidence that this was a mistake: raising strings >> was removed in Python 2.6, rather than waiting for Python 3000. I expect >> that most people don't even remember that you ever could write things >> like >> >> ? ? raise "This is an error" >> >> and thank goodness for that :-) > > They were caught by identity? Ouch. Definite bug magnet. In that case, > yeah, you'd normally want to just put a bare "except:" and then > decide > what to do with the exception... only... a bare except can't capture > the thing thrown, so how do you decide? IMO having to turn to > sys.exc_info is wrong; you should be able to work within the construct > of the try/except block. As far as I know, wrong or not, that's how you had to do it. I worked on at least one project that actually used a bare except, fished the string out of exc_info, and then did regexp matching on it. Because some people are so clever they're stupid. But anyway, string exceptions are long gone, and good riddance. If you really want the same functionality, you can do it much cleaner by just defining a trivial StringException class, doing raise StringException("This is an error"), and using?"except StringException as e" and fishing the string out of e. Only a little bit more verbose than a bare except plus exc_info, and a lot cleaner and more readable? From anikom15 at gmail.com Wed Feb 19 07:05:49 2014 From: anikom15 at gmail.com (=?ISO-8859-1?Q?Westley_Mart=EDnez?=) Date: Tue, 18 Feb 2014 22:05:49 -0800 Subject: [Python-ideas] An Everything singleton In-Reply-To: References: Message-ID: Is there a strong use case for this? How did this idea come about? On Feb 18, 2014 11:08 AM, "Serhiy Storchaka" wrote: > > This crazy idea is inspired by a discussing in Python-Dev which should be here. > > Currently we have a None singleton which representing nothing and raises an exception in almost all operations (except equality, etc). What about introducing its antonym, an Everything singleton (the name is discussable), which newer raises an exception in any operation, but returns consistent value? > > >>> Everything + 2 > 2 > >>> Everything < 3 > True > >>> Everything * 'foo' > 'foo' > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Wed Feb 19 07:10:34 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 17:10:34 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> Message-ID: In the interests of a larger sample size, I've enhanced my "simple try/except clause finder" script to locate the following: 1) Assignments to the same "thing" (which might be a simple name, an attribute, an item, whatever) 2) Augmented assignments, as long as they use the same operator 3) Return values 4) Expressions The last one gets a *lot* of false positives. As far as the AST is concerned, both of these are try/except blocks with a simple expression in each: try: big.complex.function.call(int(x)) except ValueError: big.complex.function.call(x) # and # try: some_file.write(some_data) except IOError: print("Unable to write data", file=sys.stderr) The former might well want to be changed (though I've put those into my "semantic changes" section, see below), but the latter definitely can't. For the moment, I'm not trying to make the script recognize those, so the human has to skim over all of them. But the other three are fairly useful. Once again, there are a good number of try/except blocks that match these fairly restrictive rules - around 300. (THIS! IS! PYTHON!) And once again, they overwhelmingly *do not* use the 'as' clause. For reference, I had the script collect stats on except clauses overall. The same files contain 4617 except clauses, of which 1007 use 'as'. Based on that approximate one-in-five ratio, I would expect there to be about 60 uses of 'as' in the simple try/excepts; but this is not the case. In fact, there are only 34 out of 301, roughly half the proportion required (note that this is skipping all files with "/test/" in their names; including those gives 52/390 which is still way lower than 1/5); and skipping all the two-expression blocks (most of which are false positives) brings it down to just 7 out of 235. Seven. That's how few simple try/except clauses use 'as'. So, I'm really needing someone's big Python project to run this on, to get an alternative data set. Someone want to help out with stats? Annotated examples: https://github.com/Rosuav/ExceptExpr/blob/master/examples.py All files: https://github.com/Rosuav/ExceptExpr/ ChrisA From rosuav at gmail.com Wed Feb 19 08:06:05 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 18:06:05 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> Message-ID: On Wed, Feb 19, 2014 at 5:10 PM, Chris Angelico wrote: > Once again, there are a good number of try/except blocks that match > these fairly restrictive rules - around 300. (THIS! IS! PYTHON!) And > once again, they overwhelmingly *do not* use the 'as' clause. Despite the numbers saying that 'as' is almost completely unnecessary to the proposal, I'm leaving it in for the moment, pending more data. However, I'm removing two parts of the proposal: bare except clauses (use "except BaseException:" if you really want that) and chaining of excepts (use parens, "(expr except Exception1: default1) except Exception2: default2"). Current PEP draft: https://raw2.github.com/Rosuav/ExceptExpr/master/pep-0463.txt ChrisA From p.f.moore at gmail.com Wed Feb 19 08:25:13 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 19 Feb 2014 07:25:13 +0000 Subject: [Python-ideas] except expression In-Reply-To: <20140219001048.GX4519@ando> References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> Message-ID: On 19 February 2014 00:10, Steven D'Aprano wrote: > x = d1[key] except K d2[key] except K 0 > > > which by the way looks horrible without the colons (sorry Paul, you have > not convinced me that the Tim Peters prohibition on grit applies here). While I did say later on yesterday that I'd become less uncomfortable with the except Whatever: form, I was never advocating just removing the colons - pick your favourite keyword-based proposal and compare with that if you want to see what I was talking about. But that ship has sailed, for better or worse, and it looks like the except-colon form is the winner. So be it. Paul From rosuav at gmail.com Wed Feb 19 08:34:28 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 18:34:28 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> Message-ID: On Wed, Feb 19, 2014 at 6:25 PM, Paul Moore wrote: > On 19 February 2014 00:10, Steven D'Aprano wrote: >> x = d1[key] except K d2[key] except K 0 >> >> >> which by the way looks horrible without the colons (sorry Paul, you have >> not convinced me that the Tim Peters prohibition on grit applies here). > > While I did say later on yesterday that I'd become less uncomfortable > with the except Whatever: form, I was never advocating just removing > the colons - pick your favourite keyword-based proposal and compare > with that if you want to see what I was talking about. > > But that ship has sailed, for better or worse, and it looks like the > except-colon form is the winner. So be it. The except-colon form is *my* currently-preferred spelling. But when Guido took a quick look at it, he didn't like the colon form, and his preference makes a lot more difference than mine :) It remains to be seen whether the PEP convinces him one way or the other. Have you a preferred keyword-based proposal? Does it use an existing keyword or create a new one? The floor is yours, Paul: sell me your keyword :) Bear in mind, I used to be in favour of the "expr except Exception pass default" form, so it shouldn't be too hard to push me back to there. ChrisA From p.f.moore at gmail.com Wed Feb 19 08:53:07 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 19 Feb 2014 07:53:07 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> Message-ID: On 19 February 2014 07:34, Chris Angelico wrote: > Have you a preferred keyword-based proposal? Does it use an existing > keyword or create a new one? The floor is yours, Paul: sell me your > keyword :) Bear in mind, I used to be in favour of the "expr except > Exception pass default" form, so it shouldn't be too hard to push me > back to there. Honestly? No, I don't. If I need an example of non-colon syntax I choose "return" and consider "pass" because those are the 2 that seem most reasonable, and I remember "return" fastest. But I can't really argue strongly for them - whoever said it was right, *all* of the keyword approaches are picking "something that's not awful" from what we have. My list of new keywords is basically the same as yours, but I don't think the construct is important enough to warrant a new keyword (if the if-expression couldn't justify "then", then we don't stand a chance!) So having struggled to find objections to the colon syntax, I've reached a point where I think it's the best of the alternatives. "I can't find a good enough objection" isn't exactly a ringing endorsement, but it's the best I've got :-) Paul From steve at pearwood.info Wed Feb 19 09:14:59 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 19 Feb 2014 19:14:59 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> Message-ID: <20140219081458.GD1458@ando> On Wed, Feb 19, 2014 at 06:06:05PM +1100, Chris Angelico wrote: > Despite the numbers saying that 'as' is almost completely unnecessary > to the proposal, I'm leaving it in for the moment, pending more data. > However, I'm removing two parts of the proposal: bare except clauses > (use "except BaseException:" if you really want that) and chaining of > excepts (use parens, "(expr except Exception1: default1) except > Exception2: default2"). I think it will be helpful to include a section with a few definitions, because the term "chaining" is ambiguous. It could mean either: * Multiple except clauses in a single expression (that is, n-ary operator, for n > ). Example: (expression except SpamError: spam, except EggsError: eggs) where the 5 operands here are - expression - SpamError - EggsError - spam - eggs Or chaining could mean wrapping one except-expression inside another, such as: (expression except SpamError: spam) except EggsError: eggs which has two expressions each with three operands: - expression - SpamError - spam and - (expression except SpamError: spam) - EggsError - eggs I think it is important to decide which one is chaining, and which one is not, include it in the PEP as a definition, and stick with it. I *strongly* suggest that chaining means the second case. That's what chaining means when we talk about method chaining: obj.spam().eggs() # call eggs method of the result returned by spam and exception chaining: raise ThisError from some_exception I don't think that we should prohibit passing one except-expression as argument to another. The first case, the n-ary form, can be deferred for later. But I don't think that's chaining. It is also important to note that this are not, in general, the same thing! -- Steven From rosuav at gmail.com Wed Feb 19 09:17:45 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 19:17:45 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> Message-ID: On Wed, Feb 19, 2014 at 6:53 PM, Paul Moore wrote: > So having struggled to find objections to the colon syntax, I've > reached a point where I think it's the best of the alternatives. "I > can't find a good enough objection" isn't exactly a ringing > endorsement, but it's the best I've got :-) Hehe. That's one way to settle it! If anyone does have a strong objection, I do want to hear. Sometimes a thread like this starts piling off in one direction, and any voice going the other way may feel like yelling into a hurricane, but a lot of the decisions made in the PEP have been on a fairly light balance of evidence. One good shout in a different direction could change some of them. The 'as' clause is currently hanging in the balance, for instance: Pro: Consistency with try/except statement, functionality. Con: Introduces complexity, functionality isn't used. It's on the knife-edge. Currently it's in, but could go out more easily than a Pommie batsman. *dives for cover* ChrisA From rosuav at gmail.com Wed Feb 19 09:29:00 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 19:29:00 +1100 Subject: [Python-ideas] except expression In-Reply-To: <20140219081458.GD1458@ando> References: <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <20140219081458.GD1458@ando> Message-ID: On Wed, Feb 19, 2014 at 7:14 PM, Steven D'Aprano wrote: > I think it will be helpful to include a section with a few definitions, > because the term "chaining" is ambiguous. It could mean either: > > * Multiple except clauses in a single expression (that is, n-ary > operator, for n > ). Example: > > (expression except SpamError: spam, > except EggsError: eggs) > > > where the 5 operands here are > > - expression > - SpamError > - EggsError > - spam > - eggs > > > > Or chaining could mean wrapping one except-expression inside another, > such as: > > (expression except SpamError: spam) except EggsError: eggs > > which has two expressions each with three operands: > > - expression > - SpamError > - spam > > and > > - (expression except SpamError: spam) > - EggsError > - eggs > > > I think it is important to decide which one is chaining, and which one > is not, include it in the PEP as a definition, and stick with it. I > *strongly* suggest that chaining means the second case. I've been using it to mean the first case. The second case is simply what happens when you use an except-expression as an operand to another except-expression, and I'd be inclined to call that "nesting", as it's effectively this: try: try: _ = expression except SpamError: _ = spam except EggsError: _ = eggs But if that form is called "chaining", then what's the other form called? Whatever it is, I need a word for it, because that one is the one that entails additional syntax. > That's what > chaining means when we talk about method chaining: > > obj.spam().eggs() # call eggs method of the result returned by spam > > and exception chaining: > > raise ThisError from some_exception But not comparison chaining, which is this: 1 < x < 2 I'm okay with changing the word, but I need something to change it _to_. I can't just 'del chaining' - without some other reference, the whole concept would be garbage collected :) > I don't think that we should prohibit passing one except-expression as > argument to another. Of course not, any more than anyone prohibits the nesting of try blocks. > The first case, the n-ary form, can be deferred for later. But I don't > think that's chaining. > > It is also important to note that this are not, in general, the same > thing! Indeed, they're not. ChrisA From rosuav at gmail.com Wed Feb 19 09:39:37 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 19:39:37 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> Message-ID: On Wed, Feb 19, 2014 at 7:23 PM, Bruce Leban wrote: > On Tue, Feb 18, 2014 at 11:06 PM, Chris Angelico wrote: >> >> However, I'm removing two parts of the proposal: ... chaining of >> >> excepts (use parens, "(expr except Exception1: default1) except >> Exception2: default2"). > > > There are two problems with trying to leave this out. > > First, you say "use parens" but no use of parenthesis can have the same > effect. The alternative you give: > (expr except E1: default1) except E2: default2 > is not the same as the chained form. This catches E2 in both the evaluation > of E1 and the evaluation of default1 while the chained form catches both > exceptions only in the evaluation of E1. And putting parens anywhere else > doesn't do it either. Correct. > Second, isn't an except expression like any other expression which can be > used anywhere an expression can be? So that means that writing something > that looks like a chained expression without parenthesis will be a valid > expression and will have to have some interpretation (depending on whether > except is left associative or right associative. Not quite; it could be made a SyntaxError to abut two except expressions without parens. Failing that, the without-parens form would always mean nesting (or what Steven calls chaining), and a future development that puts a single try with multiple excepts would have to be done with commas. But there's still one big question: What use-case have you for the multi-catch form? (Maybe call it "continued"? "extended"??) I went through the standard library and found pretty much nothing that even tripped my checks, and those were sufficiently borderline that they could just be left as statements. Do you have a good-sized Python codebase that you could either point me to, or run my little finder script on? I want to see more code that would benefit from this, and to see if there are any current use-cases that need multiple except clauses. I'd be looking for something like this: try: foo = expression except SomeException: foo = another_expression except SomeOtherException: foo = some_other_expression Any real-world examples would be appreciated. Even if that sub-proposal doesn't get accepted, it'd still be improved by examples. ChrisA From mal at egenix.com Wed Feb 19 10:10:47 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 19 Feb 2014 10:10:47 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> Message-ID: <53047517.6070007@egenix.com> On 19.02.2014 08:34, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 6:25 PM, Paul Moore wrote: >> On 19 February 2014 00:10, Steven D'Aprano wrote: >>> x = d1[key] except K d2[key] except K 0 >>> >>> >>> which by the way looks horrible without the colons (sorry Paul, you have >>> not convinced me that the Tim Peters prohibition on grit applies here). >> >> While I did say later on yesterday that I'd become less uncomfortable >> with the except Whatever: form, I was never advocating just removing >> the colons - pick your favourite keyword-based proposal and compare >> with that if you want to see what I was talking about. >> >> But that ship has sailed, for better or worse, and it looks like the >> except-colon form is the winner. So be it. > > The except-colon form is *my* currently-preferred spelling. But when > Guido took a quick look at it, he didn't like the colon form, and his > preference makes a lot more difference than mine :) It remains to be > seen whether the PEP convinces him one way or the other. > > Have you a preferred keyword-based proposal? Does it use an existing > keyword or create a new one? The floor is yours, Paul: sell me your > keyword :) Bear in mind, I used to be in favour of the "expr except > Exception pass default" form, so it shouldn't be too hard to push me > back to there. Overall, I think the proposed syntax is too complex and offers too many options. The purpose of an except expression would be to solve a single common problem: that of adding default values for situations where an exception is raised. For this purpose, all you need is a very simple form: mylist[1] except IndexError return 0 in the same spirit as the conditional expression if - else: http://docs.python.org/2.7/reference/expressions.html#conditional-expressions You don't need: - support for "as" - support for multiple except clauses - introduction of a confusing colon block - new keywords If you do need any of these, write a regular try-except block :-) As grammar you'd get something like this: conditional_expression ::= or_test [(if_else_expression | except_expression)] if_else_expression ::= "if" or_test "else" expression except_expression ::= "except" expression "return" expression expression ::= conditional_expression | lambda_expr -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 19 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mertz at gnosis.cx Wed Feb 19 10:04:54 2014 From: mertz at gnosis.cx (David Mertz) Date: Wed, 19 Feb 2014 01:04:54 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> Message-ID: I don't know if this really amounts to a *strong* objection. To me, as much as I try to like it reading this thread, the colon just continues to feel wrong to me. Yes, of course I know the analogy with lambda, and I can even see a certain analogy with dict literals. However, far more compelling to me is making it look more like the ternary expression (which it is really basically a special case of. In terms of keywords to put in place of the colon, the "least bad" in my mind is "return." Yes, of course, we don't actually return out of a function call or remove a call stack element (well, unless we wind up doing so in the implementation). But without fuzzing one's brain *too much* one can think of the except expression as kind of like a function call, and though of that way, 'return' makes sense. The next "least bad" in my mind is 'else' because if preserved the parallel with 'val if cond else fallback' most closely. Sure, we need a bit more verbosity in the expression, especially when 'as' is used (and it should be included), i.e.: x = val except SomeError as e else something(e) But still it's basically only substituting the one 'if' for an 'except' and otherwise keeping the familiar ternary expression. Of course, my opinion here is of very slight value, since the intuitions of our BDFL will decide the outcome, and the various alternatives are present in the PEP. And indeed, if the colon is what "wins", I'll learn to use it and be more comfortable with it. On Wed, Feb 19, 2014 at 12:17 AM, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 6:53 PM, Paul Moore wrote: > > So having struggled to find objections to the colon syntax, I've > > reached a point where I think it's the best of the alternatives. "I > > can't find a good enough objection" isn't exactly a ringing > > endorsement, but it's the best I've got :-) > > Hehe. That's one way to settle it! > > If anyone does have a strong objection, I do want to hear. Sometimes a > thread like this starts piling off in one direction, and any voice > going the other way may feel like yelling into a hurricane, but a lot > of the decisions made in the PEP have been on a fairly light balance > of evidence. One good shout in a different direction could change some > of them. The 'as' clause is currently hanging in the balance, for > instance: > > Pro: Consistency with try/except statement, functionality. > > Con: Introduces complexity, functionality isn't used. > > It's on the knife-edge. Currently it's in, but could go out more > easily than a Pommie batsman. > > *dives for cover* > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bruce at leapyear.org Wed Feb 19 09:23:47 2014 From: bruce at leapyear.org (Bruce Leban) Date: Wed, 19 Feb 2014 00:23:47 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> Message-ID: On Tue, Feb 18, 2014 at 11:06 PM, Chris Angelico wrote: > However, I'm removing two parts of the proposal: ... chaining of > excepts (use parens, "(expr except Exception1: default1) except > Exception2: default2"). > There are two problems with trying to leave this out. First, you say "use parens" but no use of parenthesis can have the same effect. The alternative you give: (expr except E1: default1) except E2: default2 is not the same as the chained form. This catches E2 in both the evaluation of E1 *and* the evaluation of default1 while the chained form catches both exceptions only in the evaluation of E1. And putting parens anywhere else doesn't do it either. Second, isn't an except expression like any other expression which can be used anywhere an expression can be? So that means that writing something that looks like a chained expression without parenthesis *will* be a valid expression and will have to have some interpretation (depending on whether except is left associative or right associative. Given that, I think it is better to define it as chained now rather than leaving people to think it's chained and never being able to add it in the future (since that will change the semantics of existing code). This is analogous to handling a < b < c: it's a bug magnet in C for inexperienced programmers but can't be changed without breaking code. Starting from scratch in Python, it could be chained. --- Bruce Learn how hackers think: http://j.mp/gruyere-security -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Wed Feb 19 11:01:50 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 19 Feb 2014 21:01:50 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <20140219081458.GD1458@ando> Message-ID: <20140219100150.GA3684@ando> On Wed, Feb 19, 2014 at 07:29:00PM +1100, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 7:14 PM, Steven D'Aprano wrote: > > I think it will be helpful to include a section with a few definitions, > > because the term "chaining" is ambiguous. It could mean either: > > > > * Multiple except clauses in a single expression [...] > > Or chaining could mean wrapping one except-expression inside another, > > such as: > > > > (expression except SpamError: spam) except EggsError: eggs > > > > which has two expressions each with three operands [...] > > I think it is important to decide which one is chaining, and which one > > is not, include it in the PEP as a definition, and stick with it. I > > *strongly* suggest that chaining means the second case. > > I've been using it to mean the first case. The second case is simply > what happens when you use an except-expression as an operand to > another except-expression, and I'd be inclined to call that "nesting", Nested expressions is also a good term for it. [...] > But if that form is called "chaining", then what's the other form > called? Whatever it is, I need a word for it, because that one is the > one that entails additional syntax. I don't think it needs a single word, especially since it's being deferred for later. I'm partial to describing it as "multiple except clauses", since that's exactly what it is. As far as I can tell, nobody refers to this as exception chaining: try: this except A: a except B: b except C: c ... Exception chaining is something very different, "raise A from B", and I think it is harmful for comprehension to have "exception chaining" and "chained excepts" mean radically different things. > I'm okay with changing the word, but I need something to change it > _to_. I can't just 'del chaining' - without some other reference, the > whole concept would be garbage collected :) Yes, but it doesn't have to be a single word. -- Steven From rob.cliffe at btinternet.com Wed Feb 19 11:46:52 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Wed, 19 Feb 2014 10:46:52 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> Message-ID: <53048B9C.4070504@btinternet.com> On 19/02/2014 08:23, Bruce Leban wrote: > > On Tue, Feb 18, 2014 at 11:06 PM, Chris Angelico > wrote: > > However, I'm removing two parts of the proposal: ... chaining of > > excepts (use parens, "(expr except Exception1: default1) except > Exception2: default2"). > > > There are two problems with trying to leave this out. > > First, you say "use parens" but no use of parenthesis can have the > same effect. The alternative you give: > (expr except E1: default1) except E2: default2 > is not the same as the chained form. This catches E2 in both the > evaluation of E1 *_and_* the evaluation of default1 while the chained > form catches both exceptions only in the evaluation of E1. And putting > parens anywhere else doesn't do it either. > > Second, isn't an except expression like any other expression which can > be used anywhere an expression can be? So that means that writing > something that looks like a chained expression without parenthesis > _*will*_ be a valid expression and will have to have some > interpretation (depending on whether except is left associative or > right associative. +1. I now see that the semantics of expr1 except E1: (default1 except E2: default2) and (expr1 except E1: default1) except E2: default2 are not the same (in an earlier post I thought they were). And neither of them are the same as: Evaluate expr1 and see what exception it raises: If it raises E1, return default1 If it raises E2, return default2. (To spell it out, as much for my own benefit as anything else: The first catches E2 in the evaluation of default1. The second catches E2 in the evaluation of expr1 OR default1. The last catches E2 in the evaluation of expr1 only. ) I suggest the form without brackets should (be legal and) mean the last of these, because (1) It feels to me like the natural meaning (2) It can't conveniently be expressed otherwise without introducing additional syntax. Rob Cliffe > > Given that, I think it is better to define it as chained now rather > than leaving people to think it's chained and never being able to add > it in the future (since that will change the semantics of existing > code). This is analogous to handling a < b < c: it's a bug magnet in C > for inexperienced programmers but can't be changed without breaking > code. Starting from scratch in Python, it could be chained. > > --- Bruce > Learn how hackers think: http://j.mp/gruyere-security > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6606 - Release Date: 02/19/14 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rob.cliffe at btinternet.com Wed Feb 19 12:16:02 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Wed, 19 Feb 2014 11:16:02 +0000 Subject: [Python-ideas] except expression In-Reply-To: <53048B9C.4070504@btinternet.com> References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <53048B9C.4070504@btinternet.com> Message-ID: <53049272.3080803@btinternet.com> On 19/02/2014 10:46, Rob Cliffe wrote: > > On 19/02/2014 08:23, Bruce Leban wrote: >> >> On Tue, Feb 18, 2014 at 11:06 PM, Chris Angelico > > wrote: >> >> However, I'm removing two parts of the proposal: ... chaining of >> >> excepts (use parens, "(expr except Exception1: default1) except >> Exception2: default2"). >> >> >> There are two problems with trying to leave this out. >> >> First, you say "use parens" but no use of parenthesis can have the >> same effect. The alternative you give: >> (expr except E1: default1) except E2: default2 >> is not the same as the chained form. This catches E2 in both the >> evaluation of E1 *_and_* the evaluation of default1 while the chained >> form catches both exceptions only in the evaluation of E1. And >> putting parens anywhere else doesn't do it either. >> >> Second, isn't an except expression like any other expression which >> can be used anywhere an expression can be? So that means that writing >> something that looks like a chained expression without parenthesis >> _*will*_ be a valid expression and will have to have some >> interpretation (depending on whether except is left associative or >> right associative. > +1. > I now see that the semantics of > > expr1 except E1: (default1 except E2: default2) > > and > > (expr1 except E1: default1) except E2: default2 > > are not the same (in an earlier post I thought they were). > > And neither of them are the same as: > > Evaluate expr1 and see what exception it raises: > If it raises E1, return default1 > If it raises E2, return default2. > > (To spell it out, as much for my own benefit as anything else: > The first catches E2 in the evaluation of default1. > The second catches E2 in the evaluation of expr1 OR default1. > The last catches E2 in the evaluation of expr1 only. > ) > I suggest the form without brackets should (be legal and) mean the > last of these, because > (1) It feels to me like the natural meaning > (2) It can't conveniently be expressed otherwise without > introducing additional syntax. > > Rob Cliffe Also (3) When assigned to a variable it is equivalent to try: var = expr1 except E1: var = default1 except E2: var = default2 > >> >> Given that, I think it is better to define it as chained now rather >> than leaving people to think it's chained and never being able to add >> it in the future (since that will change the semantics of existing >> code). This is analogous to handling a < b < c: it's a bug magnet in >> C for inexperienced programmers but can't be changed without breaking >> code. Starting from scratch in Python, it could be chained. >> >> --- Bruce >> Learn how hackers think: http://j.mp/gruyere-security >> >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct:http://python.org/psf/codeofconduct/ >> >> >> No virus found in this message. >> Checked by AVG - www.avg.com >> Version: 2012.0.2247 / Virus Database: 3705/6606 - Release Date: 02/19/14 >> > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6606 - Release Date: 02/19/14 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Wed Feb 19 12:31:52 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 19 Feb 2014 11:31:52 +0000 Subject: [Python-ideas] except expression In-Reply-To: <20140219100150.GA3684@ando> References: <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <20140219081458.GD1458@ando> <20140219100150.GA3684@ando> Message-ID: On 19 February 2014 10:01, Steven D'Aprano wrote: > I don't think it needs a single word, especially since it's being > deferred for later. I'm partial to describing it as "multiple except > clauses", since that's exactly what it is. > > As far as I can tell, nobody refers to this as exception chaining: > > try: this > except A: a > except B: b > except C: c > ... > > Exception chaining is something very different, "raise A from B", and I > think it is harmful for comprehension to have "exception chaining" and > "chained excepts" mean radically different things. Agreed the terminology should be clarified. In expression terms, a < b < c is referred to as chained comparisons, so whichever way the term "chaining" is used will be surprising to someone... Paul From ncoghlan at gmail.com Wed Feb 19 13:17:05 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 19 Feb 2014 22:17:05 +1000 Subject: [Python-ideas] except expression In-Reply-To: References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> Message-ID: On 19 February 2014 17:53, Paul Moore wrote: > On 19 February 2014 07:34, Chris Angelico wrote: >> Have you a preferred keyword-based proposal? Does it use an existing >> keyword or create a new one? The floor is yours, Paul: sell me your >> keyword :) Bear in mind, I used to be in favour of the "expr except >> Exception pass default" form, so it shouldn't be too hard to push me >> back to there. > > Honestly? No, I don't. If I need an example of non-colon syntax I > choose "return" and consider "pass" because those are the 2 that seem > most reasonable, and I remember "return" fastest. But I can't really > argue strongly for them - whoever said it was right, *all* of the > keyword approaches are picking "something that's not awful" from what > we have. > > My list of new keywords is basically the same as yours, but I don't > think the construct is important enough to warrant a new keyword (if > the if-expression couldn't justify "then", then we don't stand a > chance!) > > So having struggled to find objections to the colon syntax, I've > reached a point where I think it's the best of the alternatives. "I > can't find a good enough objection" isn't exactly a ringing > endorsement, but it's the best I've got :-) Right, unless Guido manages to pull another PEP 308 style "I have come up with a clever idea nobody else thought of, and likely only the BDFL could sell to anyone else", I think the colon based version Chris has written up in the PEP currently counts as "least bad of the alternatives proposed, and sufficiently tolerable to be judged better than the assortment of relatively ad hoc workarounds we have at the moment". That said, I did suggest using the function return type annotation marker as a possibility, and that's not currently listed in the PEP: value = lst[2] except IndexError -> 'No value' I do slightly prefer that to the colon based version, although we may want to go with the generator expression approach of always requiring parentheses around it (with function call parentheses counting): value = (lst[2] except IndexError -> 'No value') On an unrelated tangent (as I forget which part of the thread it came up in), having to switch from the compiler checked and code completion friendly 'obj.attr' to the more opaque (from an automated code analysis point of view) 'getattr(obj, "attr", None)' is another symptom of this current limitation that should be listed in the motivation section of the PEP. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Wed Feb 19 13:28:59 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 19 Feb 2014 22:28:59 +1000 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> Message-ID: On 19 February 2014 18:39, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 7:23 PM, Bruce Leban wrote: >> Second, isn't an except expression like any other expression which can be >> used anywhere an expression can be? So that means that writing something >> that looks like a chained expression without parenthesis will be a valid >> expression and will have to have some interpretation (depending on whether >> except is left associative or right associative. > > Not quite; it could be made a SyntaxError to abut two except > expressions without parens. Failing that, the without-parens form > would always mean nesting (or what Steven calls chaining), and a > future development that puts a single try with multiple excepts would > have to be done with commas. > > But there's still one big question: What use-case have you for the > multi-catch form? (Maybe call it "continued"? "extended"??) I went > through the standard library and found pretty much nothing that even > tripped my checks, and those were sufficiently borderline that they > could just be left as statements. There's a different way of addressing that concern: applying the same syntactic restriction as is used for generator expressions, and *always* require the parentheses. If using the ":" delimited syntax, then the requirement would be unconditional, if using some other separator (like "return" or "->"), then the parentheses could be optional when the expression is the sole argument to a function call. If someone later makes the case for allowing multiple except clauses, then that's OK, since the parentheses to disambiguate are already guaranteed to be present, regardless of context. Either way, based on your survey of the stdlib, I agree with MAL's suggestion to also rule out embedded name binding. The example you give in the PEP of losing the distinction between whether an iterator yielded a new incremental value or terminated and returned a final value is another sign that allowing name binding is a potential bug magnet, because it conflates two different return channels into one. That's a very different thing from substituting in a known *default* value when the operation fails, and definitely not a practice we want to encourage. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From steve at pearwood.info Wed Feb 19 13:46:46 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 19 Feb 2014 23:46:46 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <20140219001048.GX4519@ando> Message-ID: <20140219124646.GB3684@ando> On Wed, Feb 19, 2014 at 01:04:54AM -0800, David Mertz wrote: > I don't know if this really amounts to a *strong* objection. To me, as > much as I try to like it reading this thread, the colon just continues to > feel wrong to me. Yes, of course I know the analogy with lambda, and I can > even see a certain analogy with dict literals. However, far more > compelling to me is making it look more like the ternary expression (which > it is really basically a special case of. I don't believe that it is a special case of if-expressions. The two operators are completely unrelated (although they sometimes get used for similar purposes). LBYL and EAFP have completely different semantics. Think "hammer and nail" and "screwdriver and screw" -- they can get used for the same thing, but you use them in different ways and they work according to different principles. I think you are being misled by the fact that both ternary operators have three arguments, and therefore there's really only a limited number of ways you can arrange them using infix notation, the most obvious being to place two symbols between the three operands. In C, we have cond ? true-value : false-value In Algol, we have ( cond | true-statements | false-statements ) In both of those forms, the first thing evaluated -- the condition -- is listed first. The proposed except ternary operator works similarly: expr except Exception fallback ^^^^ evaluate this first where might be spelled ":" or perhaps "return" or "import" *grin*. But in Python, the ternary if operator has the condition listed in the middle: true-statement if cond else false-condition ..................^^^^ evaluate this first This truly is an odd duck. It works, at least for English speakers, but it is very different from most ternary operators, which evaluate the left-most operand first, not the middle one. So if we were to make the except operator follow the lead of if, it would look something like this: exception except expr default .................^^^^ evaluate this first which is awful. So I reject your premise that we ought to make the except ternary operator look like the if ternary operator. > In terms of keywords to put in place of the colon, the "least bad" in my > mind is "return." Yes, of course, we don't actually return out of a > function call or remove a call stack element (well, unless we wind up doing > so in the implementation). But without fuzzing one's brain *too much* one > can think of the except expression as kind of like a function call, and > though of that way, 'return' makes sense. I've already talked about why I think that's an inappropriate term. So moving along: > The next "least bad" in my mind is 'else' because if preserved the parallel > with 'val if cond else fallback' most closely. So the else in a try...except statement runs when an exception does not occur, and the else in an except expression runs when an exception does occur. No offense intended, but this is how we get god-awful syntax like "for...else" :-) Great concept, lousy choice of keywords. In a for-loop, the "else" actually isn't an else at all. Like many people, I spent years convinced that for...else executed the "else" block if the for-loop *didn't* run, as in "run this for-loop, else run this block". In reality the "else" block unconditionally runs after the for-loop. (The only way to skip the else block is to break, return or raise, which jumps right out of the for....else statement.) It's more of a "then" rather than an "else". In this case, the except cause is not an "else" at all. We have: expr1 except Something else expr2 => evaluate an expression did an exception get raised? else evaluate another expression which implies that the right-most expression should be evaluated only if *no exception occurs*. Which would be pointless. -- Steven From rosuav at gmail.com Wed Feb 19 13:47:57 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 19 Feb 2014 23:47:57 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> Message-ID: On Wed, Feb 19, 2014 at 11:17 PM, Nick Coghlan wrote: > Right, unless Guido manages to pull another PEP 308 style "I have come > up with a clever idea nobody else thought of, and likely only the BDFL > could sell to anyone else", I think the colon based version Chris has > written up in the PEP currently counts as "least bad of the > alternatives proposed, and sufficiently tolerable to be judged better > than the assortment of relatively ad hoc workarounds we have at the > moment". > > That said, I did suggest using the function return type annotation > marker as a possibility, and that's not currently listed in the PEP: > > value = lst[2] except IndexError -> 'No value' I missed that in the flurry. Have added it now. > On an unrelated tangent (as I forget which part of the thread it came > up in), having to switch from the compiler checked and code completion > friendly 'obj.attr' to the more opaque (from an automated code > analysis point of view) 'getattr(obj, "attr", None)' is another > symptom of this current limitation that should be listed in the > motivation section of the PEP. Good point. Also added, though I put it into Rationale rather than Motivation. ChrisA From oscar.j.benjamin at gmail.com Wed Feb 19 13:57:37 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 19 Feb 2014 12:57:37 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> Message-ID: On 19 February 2014 07:06, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 5:10 PM, Chris Angelico wrote: > > Current PEP draft: > > https://raw2.github.com/Rosuav/ExceptExpr/master/pep-0463.txt It looks good to me. I think I most often use the ternary if/else as part of a larger expression such as a dict display. I can imagine that the same would apply here and it might be good to add an example of this. I don't know how common a pattern it is in other people's code but something like: header = { 'param1': foo, 'param2': bar, ... } try: header['paramN'] = load_from_file(baz, spam) except OSError: header['paramN'] = '' In this case your proposed syntax would make it possible to write the whole thing as a single expression. Without the syntax you'd probably need to write an additional function to get the same effect: def load_from_file_default_None(baz, spam): try: return load_from_file(baz, spam) except OSError: return None Oscar From elazarg at gmail.com Wed Feb 19 14:00:55 2014 From: elazarg at gmail.com (=?UTF-8?B?15DXnNei15bXqA==?=) Date: Wed, 19 Feb 2014 15:00:55 +0200 Subject: [Python-ideas] except expression In-Reply-To: References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> Message-ID: A bit OT: I think there should be a mention in the PEP of other languages with exception-handling expressions, like SML's - (hd []) handle Empty => 1; val it = 1 : int Which reminds Nick's proposal. I think the parens makes it look very clean in such simple examples. --- Elazar 2014-02-19 14:47 GMT+02:00 Chris Angelico : > On Wed, Feb 19, 2014 at 11:17 PM, Nick Coghlan wrote: > > Right, unless Guido manages to pull another PEP 308 style "I have come > > up with a clever idea nobody else thought of, and likely only the BDFL > > could sell to anyone else", I think the colon based version Chris has > > written up in the PEP currently counts as "least bad of the > > alternatives proposed, and sufficiently tolerable to be judged better > > than the assortment of relatively ad hoc workarounds we have at the > > moment". > > > > That said, I did suggest using the function return type annotation > > marker as a possibility, and that's not currently listed in the PEP: > > > > value = lst[2] except IndexError -> 'No value' > > I missed that in the flurry. Have added it now. > > > On an unrelated tangent (as I forget which part of the thread it came > > up in), having to switch from the compiler checked and code completion > > friendly 'obj.attr' to the more opaque (from an automated code > > analysis point of view) 'getattr(obj, "attr", None)' is another > > symptom of this current limitation that should be listed in the > > motivation section of the PEP. > > Good point. Also added, though I put it into Rationale rather than > Motivation. > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Wed Feb 19 14:01:34 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 19 Feb 2014 13:01:34 +0000 Subject: [Python-ideas] except expression In-Reply-To: <20140219124646.GB3684@ando> References: <20140219001048.GX4519@ando> <20140219124646.GB3684@ando> Message-ID: On 19 February 2014 12:46, Steven D'Aprano wrote: > But in Python, the ternary if operator has the condition listed in the > middle: > > true-statement if cond else false-condition > ..................^^^^ evaluate this first > > > This truly is an odd duck. It works, at least for English speakers, but > it is very different from most ternary operators, which evaluate the > left-most operand first, not the middle one. So if we were to make the > except operator follow the lead of if, it would look something like this: > > exception except expr default > .................^^^^ evaluate this first > > > which is awful. So I reject your premise that we ought to make the > except ternary operator look like the if ternary operator. I think this is pretty much a tangent by now, but your interpretation here isn't the only way of looking at things. Consider the following alternative way of looking at it TRUE-EXPR if COND else FALSE-EXPR Here, TRUE-EXPR is the "expected" result, qualified by the possibility that if COND isn't true then FALSE-EXPR is the result. Looking at the if expression in that way, the parallel with the exception expression is much clearer: EXPR except EXCEPTION return FALLBACK Here, EXPR is the "expected" result, but if you get EXCEPTION when evaluating it then use FALLBACK instead. (Feel free to replace "return" with colon or your favourite syntax alternative). I can't think what you mean by "most ternary operators" unless you're referring to ternary operators in languages other than Python, so I won't comment on that part of what you say. That may not be how *you* think of the if expression, but it's how I think of it (more accurately, it's how I learned to think of it as part of understanding why Guido chose that option). Paul From steve at pearwood.info Wed Feb 19 14:07:07 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 20 Feb 2014 00:07:07 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> Message-ID: <20140219130706.GC3684@ando> On Wed, Feb 19, 2014 at 10:17:05PM +1000, Nick Coghlan wrote: > That said, I did suggest using the function return type annotation > marker as a possibility, and that's not currently listed in the PEP: > > value = lst[2] except IndexError -> 'No value' I missed that too. I prefer that to nearly all of the keyword based suggestions so far. In order of most preferred to least: +1 : +0.9 -> +0.5 then -0.5 return -1 pass, else -> also passes the Tim Peters "grit on monitor" test. > I do slightly prefer that to the colon based version, although we may > want to go with the generator expression approach of always requiring > parentheses around it (with function call parentheses counting): > > value = (lst[2] except IndexError -> 'No value') I'd be happy with that solution. -- Steven From steve at pearwood.info Wed Feb 19 14:23:00 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 20 Feb 2014 00:23:00 +1100 Subject: [Python-ideas] Ternary conditional operator [was Re: except expression] In-Reply-To: References: <20140219001048.GX4519@ando> <20140219124646.GB3684@ando> Message-ID: <20140219132259.GD3684@ando> I agree with Paul that this is a tangent. Just to answer his questions: On Wed, Feb 19, 2014 at 01:01:34PM +0000, Paul Moore wrote: > On 19 February 2014 12:46, Steven D'Aprano wrote: > > But in Python, the ternary if operator has the condition listed in the > > middle: > > > > true-statement if cond else false-condition > > ..................^^^^ evaluate this first > > > > > > This truly is an odd duck. It works, at least for English speakers, but > > it is very different from most ternary operators, which evaluate the > > left-most operand first, not the middle one. So if we were to make the > > except operator follow the lead of if, it would look something like this: > > > > exception except expr default > > .................^^^^ evaluate this first > > > > > > which is awful. So I reject your premise that we ought to make the > > except ternary operator look like the if ternary operator. > > I think this is pretty much a tangent by now, but your interpretation > here isn't the only way of looking at things. Consider the following > alternative way of looking at it > > TRUE-EXPR if COND else FALSE-EXPR > > Here, TRUE-EXPR is the "expected" result, qualified by the possibility > that if COND isn't true then FALSE-EXPR is the result. Looking at the > if expression in that way, the parallel with the exception expression > is much clearer: > > EXPR except EXCEPTION return FALLBACK > > Here, EXPR is the "expected" result, but if you get EXCEPTION when > evaluating it then use FALLBACK instead. (Feel free to replace > "return" with colon or your favourite syntax alternative). Okay, that's reasonable. > I can't think what you mean by "most ternary operators" unless you're > referring to ternary operators in languages other than Python, so I > won't comment on that part of what you say. Yes. I suggested that Python's conditional operator "is very different from most ternary operators" in that the condition is in the middle rather than on the left. I don't mean that as a negative, I actually do think it works well for if/else, but it is certainly different from what other languages do. If you look at (say) this page: http://en.wikipedia.org/wiki/%3F: nearly all the conditional operators apart from Python take the form: condition SYMBOL true-expression SYMBOL false-expression usually with ? and : as the symbols. (I'm excluding those languages that use functional notation.) Coffeescript is a exception: SYMBOL condition SYMBOL true-expression SYMBOL false-expression Likewise, the BETWEEN ternary operator in SQL looks like value BETWEEN low AND high But as you say, this is a tangent. -- Steven From rosuav at gmail.com Wed Feb 19 14:30:16 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 00:30:16 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> Message-ID: On Wed, Feb 19, 2014 at 11:57 PM, Oscar Benjamin wrote: > I don't know how common a pattern it is in other people's code but > something like: > > header = { > 'param1': foo, > 'param2': bar, > ... > } > > try: > header['paramN'] = load_from_file(baz, spam) > except OSError: > header['paramN'] = '' That looks awesome! Do you have actual production code that looks like that, and if so, can I have a squiz at it? Because that's a strong use-case right there. What *isn't* a clear use-case is where you want the header to not exist at all if the exception's thrown. For that, it's better to use "try: assign; except OSError: pass", because there's no easy way to tell a dict to not set something. It might be nice if it could be done, but it can't. For that reason I've sometimes stipulated that a list or dict is allowed to have some kind of shim in it, resulting in code like this: lst = [ fn1 and load_from_file(fn1), fn2 and load_from_file(fn2), load_from_constant("spam spam spam", # ... etc ... ] But if you specifically want those empty strings anyway, then yes, except expressions would be perfect. ChrisA From rosuav at gmail.com Wed Feb 19 14:31:18 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 00:31:18 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> Message-ID: On Thu, Feb 20, 2014 at 12:00 AM, ????? wrote: > A bit OT: > I think there should be a mention in the PEP of other languages with > exception-handling expressions, like SML's > > - (hd []) handle Empty => 1; > val it = 1 : int I'm not familiar with SML. Want to write up a paragraph that I can insert directly into the PEP? ChrisA From alc at spika.net Wed Feb 19 14:34:53 2014 From: alc at spika.net (=?UTF-8?Q?Alejandro_L=C3=B3pez_Correa?=) Date: Wed, 19 Feb 2014 14:34:53 +0100 Subject: [Python-ideas] for-while statement Message-ID: Hello, I think adding an optional "WHILE" clause in "FOR" loops might be useful sometimes (shorter code and/or improved readability): for #VAR# in #SEQUENCE# while #WATCHDOG_EXPRESSION#: #CODE_BLOCK# Examples: keepRunning = True for i in range(100) while keepRunning: keepRunning = do_some_work( i ) found = False for candidate in sequence while not found: try: process( candidate ) found = True except InvalidCandidate: pass retryCount = 7 for i in range(1,1+retryCount) while resource.acquire() == FAIL: sleep( i**2 ) At the moment, I usually implement this either with ugly breaks: for i in range(100): if not do_some_work( i ): break found = False for candidate in sequence: try: process_candidate() except InvalidCandidate: pass else: found = True break Or with while loops and counters (or counting-like expressions): i = 1 while i <= retryCount and not resource.acquired: if resource.acquire() == FAIL: sleep( i**2 ) i += 1 Of course, actual code tends to be more complex, with "keepRunning" being modified in some branches of "IF" blocks, and there might be nested loops where the exit condition for the outer one is set in the inner loop. Compare these two examples: found = False for filePath in glob( '*.data' ): for line in open( filePath, 'rt' ): if line.startswith( '#' ): continue if handle( line ): found = True break if found: break found = False for filePath in glob( '*.data' ) while not found: for line in open( filePath, 'rt' ) while not found: if line.startswith( '#' ): continue if handle( line ): found = True -- Alejandro From steve at pearwood.info Wed Feb 19 14:39:16 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 20 Feb 2014 00:39:16 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> Message-ID: <20140219133916.GE3684@ando> On Wed, Feb 19, 2014 at 10:28:59PM +1000, Nick Coghlan wrote: > Either way, based on your survey of the stdlib, I agree with MAL's > suggestion to also rule out embedded name binding. The example you > give in the PEP of losing the distinction between whether an iterator > yielded a new incremental value or terminated and returned a final > value is another sign that allowing name binding is a potential bug > magnet, because it conflates two different return channels into one. Hmmm, I think that's a bit strong. If combining the iterator's yield value and the iterator's return value is a mistake, that's not the syntax that makes it a mistake, it's that the very idea of doing so is wrong. We wouldn't say that try...except statements are a bug magnet because we can write: try: result = next(it) except StopIteration as e: result = e.args[0] I had a use-case for naming the exception caught: you are working with some library that returns and expects 2-tuples of (success, value). So you might do this: try: return (True, expr) except SomeError as e: return (False, e.args[0]) which becomes: (True, expr) except SomeError as e: (False, e.args[0]) Chris suggested that there is an example of this in the imap module, but it hasn't hit the PEP yet. -- Steven From rosuav at gmail.com Wed Feb 19 14:43:37 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 00:43:37 +1100 Subject: [Python-ideas] Ternary conditional operator [was Re: except expression] In-Reply-To: <20140219132259.GD3684@ando> References: <20140219001048.GX4519@ando> <20140219124646.GB3684@ando> <20140219132259.GD3684@ando> Message-ID: On Thu, Feb 20, 2014 at 12:23 AM, Steven D'Aprano wrote: > If you look at (say) this page: > > http://en.wikipedia.org/wiki/%3F: > > nearly all the conditional operators apart from Python take the form: > > condition SYMBOL true-expression SYMBOL false-expression > > usually with ? and : as the symbols. Most of the reason for that is because they're all deriving from each other, and if you're going to borrow syntax from someone else, you should use the same little details. Making a ternary conditional operator that uses ? and : just as C does, but switches the true-expression and false-expression, would unnecessarily confuse people. So would fiddling with its associativity, not that that stopped PHP. C's ternary operator reads in the same order as a classic if statement: /* C */ if (cond) true_statement; else false_statement; value = cond ? true_expression : false_expression; # Python if cond: true_statement else: false_statement Perl's "shove the condition to the right hand of the page" notation reads backward: do_if_true if cond; It first evaluates cond, and then may or may not evaluate do_if_true. Python's ternary operator similarly reads in an odd order: true_expression if cond else false_expression evaluates from the middle out. But it's not the only thing that evaluates backward: >>> def foo(x): print("Calling foo:",x) return [0] >>> foo(2)[0] = foo(1) Calling foo: 1 Calling foo: 2 The RHS is evaluated before the LHS, yet this is not a problem to us. (Maybe that's because it almost never matters?) Is this something where people from a functional background approach the problem one way, and people from an imperative background approach it another way? Personally, I like my code to read more-or-less in the order it's written: top-to-bottom, left-to-right. Definitions before usage. But maybe that's because I grew up with a strongly imperative style. ChrisA From rosuav at gmail.com Wed Feb 19 14:47:07 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 00:47:07 +1100 Subject: [Python-ideas] except expression In-Reply-To: <20140219133916.GE3684@ando> References: <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <20140219133916.GE3684@ando> Message-ID: On Thu, Feb 20, 2014 at 12:39 AM, Steven D'Aprano wrote: > Chris suggested that there is an example of this in the imap module, but > it hasn't hit the PEP yet. Most of the examples are out in a separate file. I'm not sure which of them, if any, should be blessed with a mention in the PEP itself. https://github.com/Rosuav/ExceptExpr/blob/master/examples.py#L356 Part of the mess with that one is that it really needs an author's eye; I can show that this could be done cleanly with the new syntax, but I can't make it both clean *and* trivially provably performing the same job. It'd probably want some other way of rendering the exception to text. ChrisA From rosuav at gmail.com Wed Feb 19 14:52:22 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 00:52:22 +1100 Subject: [Python-ideas] except expression In-Reply-To: <20140219133916.GE3684@ando> References: <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <20140219133916.GE3684@ando> Message-ID: On Thu, Feb 20, 2014 at 12:39 AM, Steven D'Aprano wrote: > On Wed, Feb 19, 2014 at 10:28:59PM +1000, Nick Coghlan wrote: > >> Either way, based on your survey of the stdlib, I agree with MAL's >> suggestion to also rule out embedded name binding. The example you >> give in the PEP of losing the distinction between whether an iterator >> yielded a new incremental value or terminated and returned a final >> value is another sign that allowing name binding is a potential bug >> magnet, because it conflates two different return channels into one. > > Hmmm, I think that's a bit strong. If combining the iterator's yield > value and the iterator's return value is a mistake, that's not the > syntax that makes it a mistake, it's that the very idea of doing so is > wrong. And by the way, I'd say that it's wrong *in the general sense* to conflate yield/return, but such conflations happen all the time. We can use 'or' to turn any falsy value into some specific thing: def foo(widget, n=None): """Frob the widget n times, or once for each spam""" for i in range(n or len(widget.spam)): widget.frob() Nobody would ever suggest that 0, None, [], and "" should all be treated the same everywhere, but pass any of them to this function and it'll do the same thing. By moving the conflation as far outward as possible, we maintain the distinction as long as possible. At the point where the programmer knows that the two channels must mean the same thing, they can get merged. It's like merging stdout and stderr of a process prior to giving the text to a human. ChrisA From ncoghlan at gmail.com Wed Feb 19 15:03:39 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 20 Feb 2014 00:03:39 +1000 Subject: [Python-ideas] except expression In-Reply-To: References: <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <20140219133916.GE3684@ando> Message-ID: On 19 February 2014 23:52, Chris Angelico wrote: > On Thu, Feb 20, 2014 at 12:39 AM, Steven D'Aprano wrote: >> On Wed, Feb 19, 2014 at 10:28:59PM +1000, Nick Coghlan wrote: >> >>> Either way, based on your survey of the stdlib, I agree with MAL's >>> suggestion to also rule out embedded name binding. The example you >>> give in the PEP of losing the distinction between whether an iterator >>> yielded a new incremental value or terminated and returned a final >>> value is another sign that allowing name binding is a potential bug >>> magnet, because it conflates two different return channels into one. >> >> Hmmm, I think that's a bit strong. If combining the iterator's yield >> value and the iterator's return value is a mistake, that's not the >> syntax that makes it a mistake, it's that the very idea of doing so is >> wrong. > > And by the way, I'd say that it's wrong *in the general sense* to > conflate yield/return, but such conflations happen all the time. We > can use 'or' to turn any falsy value into some specific thing: > > def foo(widget, n=None): > """Frob the widget n times, or once for each spam""" > for i in range(n or len(widget.spam)): > widget.frob() > > Nobody would ever suggest that 0, None, [], and "" should all be > treated the same everywhere, but pass any of them to this function and > it'll do the same thing. > > By moving the conflation as far outward as possible, we maintain the > distinction as long as possible. At the point where the programmer > knows that the two channels must mean the same thing, they can get > merged. It's like merging stdout and stderr of a process prior to > giving the text to a human. Right, but people can still use the statement form or a context manager to help handle that. We don't need to introduce a quaternary expression with complex scoping issues for it. Since dropping the optional "as" clause drastically simplifies the proposal (no need for a hidden scope), and the use case for it isn't at all compelling, that puts it in the "don't bother" category for me. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From rosuav at gmail.com Wed Feb 19 15:05:23 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 01:05:23 +1100 Subject: [Python-ideas] for-while statement In-Reply-To: References: Message-ID: On Thu, Feb 20, 2014 at 12:34 AM, Alejandro L?pez Correa wrote: > I think adding an optional "WHILE" clause in "FOR" loops might be > useful sometimes (shorter code and/or improved readability): > > for #VAR# in #SEQUENCE# while #WATCHDOG_EXPRESSION#: > #CODE_BLOCK# Suggestion: Look at REXX's loops for some ideas and use-cases. http://www.kilowattsoftware.com/tutorial/rexx/do.htm REXX lets you combine zero or more loop conditions, including "WHILE condition", "FOR iteration_count", and so on. You may also want to consider a filtered iteration, with "IF condition" (which would be like putting "if not condition: continue" at the top of the loop). There are definite use-cases for all of these. The question is, which ones are worth blessing with syntax? ChrisA From rob.cliffe at btinternet.com Wed Feb 19 15:43:26 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Wed, 19 Feb 2014 14:43:26 +0000 Subject: [Python-ideas] except expression In-Reply-To: <20140219130706.GC3684@ando> References: <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> <20140219130706.GC3684@ando> Message-ID: <5304C30E.2080706@btinternet.com> On 19/02/2014 13:07, Steven D'Aprano wrote: > On Wed, Feb 19, 2014 at 10:17:05PM +1000, Nick Coghlan wrote: > >> That said, I did suggest using the function return type annotation >> marker as a possibility, and that's not currently listed in the PEP: >> >> value = lst[2] except IndexError -> 'No value' > I missed that too. I prefer that to nearly all of the keyword based > suggestions so far. In order of most preferred to least: > > +1 : > +0.9 -> > +0.5 then > -0.5 return > -1 pass, else > > > -> also passes the Tim Peters "grit on monitor" test. I pretty much agree. Rob Cliffe From rob.cliffe at btinternet.com Wed Feb 19 15:46:12 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Wed, 19 Feb 2014 14:46:12 +0000 Subject: [Python-ideas] for-while statement In-Reply-To: References: Message-ID: <5304C3B4.6080101@btinternet.com> On 19/02/2014 13:34, Alejandro L?pez Correa wrote: > Hello, > > I think adding an optional "WHILE" clause in "FOR" loops might be > useful sometimes (shorter code and/or improved readability): > > for #VAR# in #SEQUENCE# while #WATCHDOG_EXPRESSION#: > #CODE_BLOCK# It seems to me this would be the same as for #VAR# in #SEQUENCE#: if not (#WATCHDOG_EXPRESSION#): break #CODE_BLOCK# which doesn't really seem to me to justify adding new syntax. Rob Cliffe From oscar.j.benjamin at gmail.com Wed Feb 19 15:48:38 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 19 Feb 2014 14:48:38 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> Message-ID: On 19 February 2014 13:30, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 11:57 PM, Oscar Benjamin > wrote: >> I don't know how common a pattern it is in other people's code but >> something like: >> >> header = { >> 'param1': foo, >> 'param2': bar, >> ... >> } >> >> try: >> header['paramN'] = load_from_file(baz, spam) >> except OSError: >> header['paramN'] = '' > > That looks awesome! Do you have actual production code that looks like > that, and if so, can I have a squiz at it? Because that's a strong > use-case right there. I can't really find anything like that right now. I was just thinking that there are basically two situations where I like to use ternary if/else: as part of a larger expression/display or in a comprehension/generator expression. This proposal is similar but it's hard to find the cases that would be simplified by it as they will probably be written in a very different way right now. Oscar From oscar.j.benjamin at gmail.com Wed Feb 19 16:02:25 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 19 Feb 2014 15:02:25 +0000 Subject: [Python-ideas] for-while statement In-Reply-To: References: Message-ID: On 19 February 2014 13:34, Alejandro L?pez Correa wrote: > Hello, > > I think adding an optional "WHILE" clause in "FOR" loops might be > useful sometimes (shorter code and/or improved readability): > > for #VAR# in #SEQUENCE# while #WATCHDOG_EXPRESSION#: > #CODE_BLOCK# This idea comes up repeatedly. You might want to read some of the previous discussions about it: https://mail.python.org/pipermail/python-ideas/2013-January/018969.html https://mail.python.org/pipermail/python-ideas/2009-January/002466.html https://mail.python.org/pipermail/python-ideas/2013-June/021554.html Oscar From rosuav at gmail.com Wed Feb 19 16:06:51 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 02:06:51 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> Message-ID: On Thu, Feb 20, 2014 at 1:48 AM, Oscar Benjamin wrote: > This proposal is similar but it's > hard to find the cases that would be simplified by it as they will > probably be written in a very different way right now. If they're written the way you describe, with a "try" block that assigns to something and an "except" block that assigns to the exact same thing, my script will find them. You might want to disable (by commenting out) some of the handlers at the top; the ast.Expr one, in particular, tends to generate a lot of spam. https://github.com/Rosuav/ExceptExpr/blob/master/find_except_expr.py ChrisA From alc at spika.net Wed Feb 19 16:08:16 2014 From: alc at spika.net (=?UTF-8?Q?Alejandro_L=C3=B3pez_Correa?=) Date: Wed, 19 Feb 2014 16:08:16 +0100 Subject: [Python-ideas] for-while statement In-Reply-To: References: Message-ID: > There are definite use-cases for all of these. The question is, which > ones are worth blessing with syntax? > > ChrisA The addition of a while clause to for loops has the advantage that it does not require any new keyword, and adds some extra power to the limited for loops of python (compared to c/c++ for loops, for example) in a natural way IMHO. Of course it is not a must-have feature or something along these lines would have been already in place. For example, I miss sometimes constructs from c or pascal (IIRC) like "repeat STATEMENT until EXPRESSION" or "do STATEMENT while EXPRESSION" because the expression is evaluated after one iteration and that means vars in it do not need to be initialized before the loop. However, I understand adding such keywords as "repeat" or "do" to the language would break existing code that might use them as var names. 2014-02-19 15:46 GMT+01:00 Rob Cliffe : > It seems to me this would be the same as > > for #VAR# in #SEQUENCE#: > if not (#WATCHDOG_EXPRESSION#): break > #CODE_BLOCK# > > > which doesn't really seem to me to justify adding new syntax. > Rob Cliffe I was going to argue about higher readability and the case of nested loops, but then I've realised it might be unintuitive: sometimes it might seem natural that the first element in the sequence is consumed, sometimes the programmer might want the loop to not be entered at all if the expression is already false. This is of relevance in case of generators. # at least one element is extracted from a non-empty sequence for x in sequence while x > 0: pass # here it might be expected to not enter the loop at all if keepRunning is False, and thus leave the sequence untouched. for x in sequence while keepRunning: pass So forget about it. Sorry for the lost time. Alejandro From rosuav at gmail.com Wed Feb 19 16:16:54 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 02:16:54 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <20140219133916.GE3684@ando> Message-ID: On Thu, Feb 20, 2014 at 1:03 AM, Nick Coghlan wrote: > Since dropping the optional "as" clause drastically simplifies the > proposal (no need for a hidden scope), and the use case for it isn't > at all compelling, that puts it in the "don't bother" category for me. In the light of comments like this, I've deferred the 'as' clause, and also everything involving multiple except clauses. The proposal is now simply: expr except exception_list: default with straight-forward semantics. It can easily be extended in future. I've also added a whole bunch of examples, lifted from the separate examples.py. Note that the word "chain" no longer appears in the proposal; it's simply "multiple except clauses". Having a term for it is more useful when it's a part of the proposal; since it's deferred anyway, that's not a big deal. Latest version, as always, is here: https://raw2.github.com/Rosuav/ExceptExpr/master/pep-0463.txt Additional examples, with syntax highlighting: https://github.com/Rosuav/ExceptExpr/blob/master/examples.py Please let me know if I've made any glaring errors (even trivial ones like typos). Lots of editing has happened recently, and several times I discovered that I'd missed an 'as' somewhere or something like that. Chances are I've missed more. To everyone who's already submitted ideas: Thank you! Don't stop! :) ChrisA From oscar.j.benjamin at gmail.com Wed Feb 19 16:38:29 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 19 Feb 2014 15:38:29 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> Message-ID: On 19 February 2014 15:06, Chris Angelico wrote: > On Thu, Feb 20, 2014 at 1:48 AM, Oscar Benjamin > wrote: >> This proposal is similar but it's >> hard to find the cases that would be simplified by it as they will >> probably be written in a very different way right now. > > If they're written the way you describe, with a "try" block that > assigns to something and an "except" block that assigns to the exact > same thing, my script will find them. You might want to disable (by > commenting out) some of the handlers at the top; the ast.Expr one, in > particular, tends to generate a lot of spam. A search through cpython didn't turn up much. There are only two Subscript nodes found by this script and only this one seems relevant: http://hg.python.org/cpython/file/9cfb3d1cf0d1/Lib/sysconfig.py#l514 For whatever reason that code just creates a dict with a whole load of assignments anyway though. Personally I would have written that as a display so instead of this: _CONFIG_VARS = {} # Normalized versions of prefix and exec_prefix are handy to have; # in fact, these are the standard versions used most places in the # Distutils. _CONFIG_VARS['prefix'] = _PREFIX _CONFIG_VARS['exec_prefix'] = _EXEC_PREFIX _CONFIG_VARS['py_version'] = _PY_VERSION _CONFIG_VARS['py_version_short'] = _PY_VERSION_SHORT _CONFIG_VARS['py_version_nodot'] = _PY_VERSION[0] + _PY_VERSION[2] _CONFIG_VARS['installed_base'] = _BASE_PREFIX _CONFIG_VARS['base'] = _PREFIX _CONFIG_VARS['installed_platbase'] = _BASE_EXEC_PREFIX _CONFIG_VARS['platbase'] = _EXEC_PREFIX _CONFIG_VARS['projectbase'] = _PROJECT_BASE try: _CONFIG_VARS['abiflags'] = sys.abiflags except AttributeError: # sys.abiflags may not be defined on all platforms. _CONFIG_VARS['abiflags'] = '' You could do this: _CONFIG_VARS = { # Normalized versions of prefix and exec_prefix are handy to have; # in fact, these are the standard versions used most places in the # Distutils. 'prefix':_PREFIX, 'exec_prefix':_EXEC_PREFIX, 'py_version':_PY_VERSION, 'py_version_short':_PY_VERSION_SHORT, 'py_version_nodot':_PY_VERSION[0] + _PY_VERSION[2], 'installed_base':_BASE_PREFIX, 'base':_PREFIX, 'installed_platbase':_BASE_EXEC_PREFIX, 'platbase':_EXEC_PREFIX, 'projectbase':_PROJECT_BASE, # sys.abiflags may not be defined on all platforms. 'abiflags':sys.abiflags except AttributeError: '' } Oscar From rosuav at gmail.com Wed Feb 19 16:43:14 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 02:43:14 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> Message-ID: On Thu, Feb 20, 2014 at 2:38 AM, Oscar Benjamin wrote: > A search through cpython didn't turn up much. There are only two > Subscript nodes found by this script and only this one seems relevant: > http://hg.python.org/cpython/file/9cfb3d1cf0d1/Lib/sysconfig.py#l514 > Yep, I have that one already. In fact, that's one of the ones I pulled into the PEP itself, as it's quite a good example. Going a level further and putting everything into the braces is a logical extension of the improvement, imo. Have added a mention thereof to the PEP. ChrisA From mistersheik at gmail.com Wed Feb 19 16:45:45 2014 From: mistersheik at gmail.com (Neil Girdhar) Date: Wed, 19 Feb 2014 07:45:45 -0800 (PST) Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> Message-ID: <888fccbc-0267-40c5-a3a2-ecbcffb2d0ed@googlegroups.com> Totally agree with these sentiments. Another advantage is that this code is self-documenting. These "defaulting signatures" usually benefit from a comment to remind the reader what's going on. With the except clause, it's obvious. On Wednesday, February 12, 2014 7:14:45 PM UTC-5, Ram Rachum wrote: > > I'm happy you're for it. Maybe one day we'll see a Python 4 with no second > argument to dict.get, getattr and so many others... > > > On Thu, Feb 13, 2014 at 2:08 AM, Raymond Hettinger > > wrote: > >> >> On Feb 12, 2014, at 9:02 PM, Ram Rachum > >> wrote: >> >> Here's an idea that would help shortening code. Allow a ternary >> expression based on except, like so: >> >> first_entry = entries[0] except IndexError else None >> item = my_queue.get() except queue.Empty else None >> response_text = request('http://whatever.com').text except HttpError >> else "Can't access data" >> >> Aside from the fact that this would be a big grammar addition, a big >> problem here is the usage of the `else` keyword, that when used with except >> usually means "what would happen if there wasn't an exception" and here >> means the opposite. But I couldn't think of a nicer syntax. >> >> I realize that this is a big change and that most people would be opposed >> to this... But I guess I just wanted to share my idea :) >> >> >> I would like to see something like this come to fruition. >> We need a clean way to express the idea of "take an >> arbitrary, exception-raising function and give it a default >> argument". >> >> Hopefully, this would end the gradual but never-ending requests >> to bloat APIs with "default" arguments. For example, if your idea >> or some variant had been in place, the min() and max() functions >> likely wouldn't have grown complex signatures in Python 3.4. >> >> >> Raymond >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron3200 at gmail.com Wed Feb 19 17:17:41 2014 From: ron3200 at gmail.com (Ron Adam) Date: Wed, 19 Feb 2014 10:17:41 -0600 Subject: [Python-ideas] except expression In-Reply-To: References: <20140219001048.GX4519@ando> <20140219124646.GB3684@ando> Message-ID: On 02/19/2014 07:01 AM, Paul Moore wrote: > I think this is pretty much a tangent by now, but your interpretation > here isn't the only way of looking at things. Consider the following > alternative way of looking at it > > TRUE-EXPR if COND else FALSE-EXPR > > Here, TRUE-EXPR is the "expected" result, qualified by the possibility > that if COND isn't true then FALSE-EXPR is the result. Looking at the > if expression in that way, the parallel with the exception expression > is much clearer: > EXPR except EXCEPTION return FALLBACK > > Here, EXPR is the "expected" result, but if you get EXCEPTION when > evaluating it then use FALLBACK instead. (Feel free to replace > "return" with colon or your favourite syntax alternative). -1 on return. I expect that to exit the function where ever it is in the current scope. How about this? >>> def catch_else(exc, e1, e2): ... try: ... return e1() ... except exc: ... return e2() ... >>> catch_else(IndexError, lambda: [1, 2, 3][2], lambda: 0) 3 >>> catch_else(IndexError, lambda: [1, 2, 3][4], lambda: 0) 0 The only bad thing about that is having to spell out lambda for each of the arguments. *1 With a short literal form for a lambda that takes no arguments. value = catch_else(exc, \expr1, \expr2) #alternatives to '\'? The function "catch_else" could be a builtin or in functools depending on how useful it's believed to be. The shorter lambda expression literal... or expression quote as it's referred to in other languages would be useful on it's own, and more preferable than a full lambda for things like this. [*1] In discussion about how long it's taken for us to get away from incandescent lights, the expert brought up that there is a difference between acceptable and preferable. Cheers, Ron From g.rodola at gmail.com Wed Feb 19 17:52:19 2014 From: g.rodola at gmail.com (Giampaolo Rodola') Date: Wed, 19 Feb 2014 17:52:19 +0100 Subject: [Python-ideas] os.path.here() Message-ID: Hello python devs. One thing which appears to be pretty common in many scripts is figuring out module's parent directory path. I often find myself writing: HERE = os.path.abspath(os.path.dirname(__file__)) ...at the top of a module. Also, it is not rare to do further concatenations in order to declare where static files are located: CONF_FILE = os.path.abspath(os.path.join(HERE, 'app.conf')) CERT_FILE = os.path.abspath(os.path.join(HERE, 'cert.pem')) THIS_FILE = os.path.abspath(os.path.join(HERE, __file__)) A quick search within Python source code shows this is indeed a very common pattern: $ find . -name \*.py | xargs grep "os\.path" | grep __file__ Just some examples (there are *many*): http://hg.python.org/cpython/file/186f6f56f4bc/Lib/distutils/tests/__init__.py#l21 http://hg.python.org/cpython/file/186f6f56f4bc/PCbuild/build_tkinter.py#l11 http://hg.python.org/cpython/file/186f6f56f4bc/PCbuild/build_ssl.py#l68 http://hg.python.org/cpython/file/186f6f56f4bc/Tools/msi/uisample.py#l2 http://hg.python.org/cpython/file/186f6f56f4bc/Lib/distutils/tests/test_config_cmd.py#l30 http://hg.python.org/cpython/file/186f6f56f4bc/Lib/unittest/test/testmock/__main__.py#l7 http://hg.python.org/cpython/file/186f6f56f4bc/Lib/pydoc.py#l2463 I think there's space for improvements here so my proposal is to add a simple os.path.here() function working like this: >>> # assuming /home/giampaolo/app/run.py is the current module: >>> os.path.here() /home/giampaolo/app/ >>> os.path.here(__file__) /home/giampaolo/app/run.py >>> os.path.here('..') /home/giampaolo/ The implementation is pretty straightforward: def here(concat=None): """Return the absolute path of the parent directory where the script is defined. """ here = os.path.abspath(os.path.dirname(__file__)) if concat is not None: here = os.path.abspath(os.path.join(here, concat)) return here Thoughts? -- Giampaolo - http://grodola.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From oscar.j.benjamin at gmail.com Wed Feb 19 18:04:47 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 19 Feb 2014 17:04:47 +0000 Subject: [Python-ideas] os.path.here() In-Reply-To: References: Message-ID: On 19 February 2014 16:52, Giampaolo Rodola' wrote: > > The implementation is pretty straightforward: > > def here(concat=None): > """Return the absolute path of the parent directory where the > script is defined. > """ > here = os.path.abspath(os.path.dirname(__file__)) > if concat is not None: > here = os.path.abspath(os.path.join(here, concat)) > return here So if I do from os.path import here and get the above function what happens when I call it in another module? > Thoughts? This encourages writing code that makes assumptions that break when run from a zip file. I think that encouraging pkgutil.get_data() for loading resources that are stored adjacent to a module is better. Oscar From alexander.belopolsky at gmail.com Wed Feb 19 18:09:19 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 19 Feb 2014 12:09:19 -0500 Subject: [Python-ideas] except expression In-Reply-To: References: <20140219001048.GX4519@ando> <20140219124646.GB3684@ando> Message-ID: On Wed, Feb 19, 2014 at 11:17 AM, Ron Adam wrote: > value = catch_else(exc, \expr1, \expr2) #alternatives to '\'? ? .. ducks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.rodola at gmail.com Wed Feb 19 18:18:55 2014 From: g.rodola at gmail.com (Giampaolo Rodola') Date: Wed, 19 Feb 2014 18:18:55 +0100 Subject: [Python-ideas] os.path.here() In-Reply-To: References: Message-ID: On Wed, Feb 19, 2014 at 6:04 PM, Oscar Benjamin wrote: > On 19 February 2014 16:52, Giampaolo Rodola' wrote: > > > > The implementation is pretty straightforward: > > > > def here(concat=None): > > """Return the absolute path of the parent directory where the > > script is defined. > > """ > > here = os.path.abspath(os.path.dirname(__file__)) > > if concat is not None: > > here = os.path.abspath(os.path.join(here, concat)) > > return here > > So if I do from os.path import here and get the above function what > happens when I call it in another module? > Ouch! You're right, I naively didn't take that into account. =) I guess there are ways to inspect the caller's module name but I'm not gonna push for that. Sorry for the noise. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rymg19 at gmail.com Wed Feb 19 18:27:40 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Wed, 19 Feb 2014 11:27:40 -0600 Subject: [Python-ideas] os.path.here() In-Reply-To: References: Message-ID: That IS an option... ```python import inspect ... module_path = inspect.stack()[1][0].f_globals['__file__'] ``` On Wed, Feb 19, 2014 at 11:18 AM, Giampaolo Rodola' wrote: > > > > On Wed, Feb 19, 2014 at 6:04 PM, Oscar Benjamin < > oscar.j.benjamin at gmail.com> wrote: > >> On 19 February 2014 16:52, Giampaolo Rodola' wrote: >> > >> > The implementation is pretty straightforward: >> > >> > def here(concat=None): >> > """Return the absolute path of the parent directory where the >> > script is defined. >> > """ >> > here = os.path.abspath(os.path.dirname(__file__)) >> > if concat is not None: >> > here = os.path.abspath(os.path.join(here, concat)) >> > return here >> >> So if I do from os.path import here and get the above function what >> happens when I call it in another module? >> > > Ouch! You're right, I naively didn't take that into account. =) > I guess there are ways to inspect the caller's module name but I'm not > gonna push for that. > Sorry for the noise. > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From oscar.j.benjamin at gmail.com Wed Feb 19 18:35:25 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 19 Feb 2014 17:35:25 +0000 Subject: [Python-ideas] os.path.here() In-Reply-To: References: Message-ID: On 19 February 2014 17:18, Giampaolo Rodola' wrote: > > On Wed, Feb 19, 2014 at 6:04 PM, Oscar Benjamin > wrote: >> >> On 19 February 2014 16:52, Giampaolo Rodola' wrote: >> > >> > The implementation is pretty straightforward: >> > >> > def here(concat=None): >> > """Return the absolute path of the parent directory where the >> > script is defined. >> > """ >> > here = os.path.abspath(os.path.dirname(__file__)) >> > if concat is not None: >> > here = os.path.abspath(os.path.join(here, concat)) >> > return here >> >> So if I do from os.path import here and get the above function what >> happens when I call it in another module? > > Ouch! You're right, I naively didn't take that into account. =) > I guess there are ways to inspect the caller's module name but I'm not gonna > push for that. > Sorry for the noise. There are ways to do it. namedtuple does it here: http://hg.python.org/cpython/file/e6016fffc894/Lib/collections/__init__.py#l374 But I think that's considered to be an unfortunate mistake by some. Oscar From rob.cliffe at btinternet.com Wed Feb 19 19:56:52 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Wed, 19 Feb 2014 18:56:52 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FE969D.5000002@canterbury.ac.nz> <20140217052912.GJ4519@ando> <530290F0.7030506@canterbury.ac.nz> <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> Message-ID: <5304FE74.1060701@btinternet.com> On 18/02/2014 20:55, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 3:06 AM, Chris Angelico wrote: >> I'm currently working on finding a bunch of examples from the stdlib >> that could be translated. Will post them once I get the script sorted >> out. As it's currently 3AM, though, that means I need to fry up some >> bacon or something before I continue :) > Alright, results are in. > > Script: > https://github.com/Rosuav/ExceptExpr/blob/master/find_except_expr.py > Output: > https://github.com/Rosuav/ExceptExpr/blob/master/candidates.txt > Annotated examples: > https://github.com/Rosuav/ExceptExpr/blob/master/examples.py I've gone through a very small code base of my own to collect some statistics. Ignoring duplication where a section of code was copied from one program to another, there were approximately: 15 instances where the new syntax could naturally be used, roughly half were variable assignment, the rest were "return" statements. Of these I judged there were 9 where the new form was short enough or at least readable enough to be a clear improvement. The other 6 were long and complicated enough to be fairly indigestible using either the old or the new syntax. I also found, although I wasn't originally looking for them: - 2 instances where there was a temptation to abuse the new syntax: var = expr except Exception1: ShutDownProgram() - 8 instances where I wanted to do an operation and ignore certain exceptions, this sort of thing: try: dosomething() except ValueError: pass and found I wanted the syntax "dosomething() except ValueError: pass". Under the current proposal this could be achieved by "dosomething() except ValueError: None", but this does not read as naturally. - 13 instances where I wanted to do something that could _not_ be written as an expression and ignore errors, broken down as follows: 5 where I wanted to assign an expression value to a variable but only if evaluating the expression did not raise an error (if it did raise an error I wanted to leave the variable untouched) 1 where I wanted to return the value of an expression, but do nothing (NOT return) if evaluating the expression raised an error. 7 of the form "del x[y] except IndexError: pass" (so to speak) So is there a case for extending the syntax to allow expr except : pass where the expression is used stand-alone (i.e. its value is thrown away). "pass" could be semantically equivalent to "None" (re the interactive interpreter). Or even for allowing "except : pass" after other statements such as "del"? Overall there were 6 instances of an exception list more complicated than a single exception name. There were none of "except Exception1 as ...". So now I am willing to go along with dropping "as" from the PEP; it is something that could always be added later. Best wishes Rob Cliffe _ _ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Wed Feb 19 20:22:59 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 19 Feb 2014 11:22:59 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> Message-ID: <53050493.8080703@stoneleaf.us> On 02/18/2014 05:34 PM, Alexander Belopolsky wrote: > On Tue, Feb 18, 2014 at 7:10 PM, Steven D'Aprano wrote: >> Alexander wrote: >>> >>> I disagree. It is best to leave explicit selection of exceptions to catch >>> outside of the expressions. This is job for things like decimal or numpy >>> fp contexts. >> >> I don't understand what relevance fp contexts have here. >> [snip nice examples of fp context] Which is great for numpy et al, but useless for the rest of the Python world. -- ~Ethan~ From ethan at stoneleaf.us Wed Feb 19 21:25:12 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 19 Feb 2014 12:25:12 -0800 Subject: [Python-ideas] except expression In-Reply-To: <20140219130706.GC3684@ando> References: <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> <20140219130706.GC3684@ando> Message-ID: <53051328.40602@stoneleaf.us> On 02/19/2014 05:07 AM, Steven D'Aprano wrote: > On Wed, Feb 19, 2014 at 10:17:05PM +1000, Nick Coghlan wrote: > >> I do slightly prefer that to the colon based version, although we may >> want to go with the generator expression approach of always requiring >> parentheses around it (with function call parentheses counting): >> >> value = (lst[2] except IndexError -> 'No value') > > I'd be happy with that solution. Same here. -- ~Ethan~ From mertz at gnosis.cx Wed Feb 19 22:02:49 2014 From: mertz at gnosis.cx (David Mertz) Date: Wed, 19 Feb 2014 13:02:49 -0800 Subject: [Python-ideas] except expression In-Reply-To: <20140219124646.GB3684@ando> References: <20140219001048.GX4519@ando> <20140219124646.GB3684@ando> Message-ID: I'll accept Steven's criticism of the 'else' keyword as being terrible because the semantics is opposite of a try/except/else block. But I will push back on the similarity between an except expression and a ternary expression (as Python does it with the condition in the middle). I wasn't in love with the Python ternary with the condition in the middle when it was added. I'm still not sure I *love* it, since the C-style ternary with the condition at the front sort of feels more natural to me... well, it would if I knew how to spell it, which didn't seem to have a good answer within Python. But that is a long done deal, so we have what we have. However, *given* the condition-in-middle form, it promotes a different understanding of the ternary expression than C programmers have. In C, we think of a condition that is pretty much equally likely to be true or false, and then list the two "branches" after that. In Python (even though it's obviously formally exactly equivalent in power), we think of the "expected" value to assign, then the condition as an "exceptional" circumstance that makes us want something else. When I use the Python ternary, it is almost always similar to: x = normal_thing if isUnexceptional(normal_thing) else something_unusual That really does feel a whole lot like an exception in the middle there. Well, actually, I guess it's the negation of an exception. But the point is that what come right at the start is what we *expect* to usually be assigned, then we worry about details of what to do just-in-case. Thought of that way, it's VERY close to: x = normal_thing except UnusualException return something_unusual On Wed, Feb 19, 2014 at 4:46 AM, Steven D'Aprano wrote: > On Wed, Feb 19, 2014 at 01:04:54AM -0800, David Mertz wrote: > > I don't know if this really amounts to a *strong* objection. To me, as > > much as I try to like it reading this thread, the colon just continues to > > feel wrong to me. Yes, of course I know the analogy with lambda, and I > can > > even see a certain analogy with dict literals. However, far more > > compelling to me is making it look more like the ternary expression > (which > > it is really basically a special case of. > > I don't believe that it is a special case of if-expressions. The two > operators are completely unrelated (although they sometimes get used for > similar purposes). LBYL and EAFP have completely different semantics. > Think "hammer and nail" and "screwdriver and screw" -- they can get used > for the same thing, but you use them in different ways and they work > according to different principles. > > I think you are being misled by the fact that both ternary operators > have three arguments, and therefore there's really only a limited number > of ways you can arrange them using infix notation, the most obvious > being to place two symbols between the three operands. > > In C, we have cond ? true-value : false-value > > In Algol, we have ( cond | true-statements | false-statements ) > > In both of those forms, the first thing evaluated -- the condition -- > is listed first. The proposed except ternary operator works similarly: > > expr except Exception fallback > ^^^^ evaluate this first > > where might be spelled ":" or perhaps "return" or "import" > *grin*. > > > But in Python, the ternary if operator has the condition listed in the > middle: > > true-statement if cond else false-condition > ..................^^^^ evaluate this first > > > This truly is an odd duck. It works, at least for English speakers, but > it is very different from most ternary operators, which evaluate the > left-most operand first, not the middle one. So if we were to make the > except operator follow the lead of if, it would look something like this: > > exception except expr default > .................^^^^ evaluate this first > > > which is awful. So I reject your premise that we ought to make the > except ternary operator look like the if ternary operator. > > > > In terms of keywords to put in place of the colon, the "least bad" in my > > mind is "return." Yes, of course, we don't actually return out of a > > function call or remove a call stack element (well, unless we wind up > doing > > so in the implementation). But without fuzzing one's brain *too much* > one > > can think of the except expression as kind of like a function call, and > > though of that way, 'return' makes sense. > > I've already talked about why I think that's an inappropriate term. So > moving along: > > > > The next "least bad" in my mind is 'else' because if preserved the > parallel > > with 'val if cond else fallback' most closely. > > So the else in a try...except statement runs when an exception does not > occur, and the else in an except expression runs when an exception does > occur. No offense intended, but this is how we get god-awful syntax like > "for...else" :-) Great concept, lousy choice of keywords. > > In a for-loop, the "else" actually isn't an else at all. Like many > people, I spent years convinced that for...else executed the "else" > block if the for-loop *didn't* run, as in "run this for-loop, else run > this block". In reality the "else" block unconditionally runs after the > for-loop. (The only way to skip the else block is to break, return or > raise, which jumps right out of the for....else statement.) It's more of > a "then" rather than an "else". > > In this case, the except cause is not an "else" at all. We have: > > expr1 except Something else expr2 > > => evaluate an expression > did an exception get raised? > else evaluate another expression > > which implies that the right-most expression should be evaluated only if > *no exception occurs*. Which would be pointless. > > > > -- > Steven > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Wed Feb 19 22:02:33 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 19 Feb 2014 13:02:33 -0800 Subject: [Python-ideas] for-while statement In-Reply-To: References: Message-ID: <2B4DF97B-2BD2-4A51-ABD6-DE48B4922D82@yahoo.com> On Feb 19, 2014, at 5:34, Alejandro L?pez Correa wrote: > Hello, > > I think adding an optional "WHILE" clause in "FOR" loops might be > useful sometimes (shorter code and/or improved readability): > > for #VAR# in #SEQUENCE# while #WATCHDOG_EXPRESSION#: > #CODE_BLOCK# > > Examples: > > keepRunning = True > for i in range(100) while keepRunning: > keepRunning = do_some_work( i ) > > found = False > for candidate in sequence while not found: > try: > process( candidate ) > found = True > except InvalidCandidate: > pass > > retryCount = 7 > for i in range(1,1+retryCount) while resource.acquire() == FAIL: > sleep( i**2 ) > > > At the moment, I usually implement this either with ugly breaks: > > > for i in range(100): > if not do_some_work( i ): > break The only reason you think they're "ugly" is that you're thinking in C terms, not Python terms. This version is shorter, it's more explicit, and it has fewer places for you or after code maintainer to screw up and create bugs. (And yes, even in a trivial case like this, I've seen people write keepRunnig = ...) And this becomes even more apparent in the longer cases. You're relying on the fact that setting a flag _plus_ falling off the end of a loop (which isn't nearly as obvious or visible or explicit) equals break. > > found = False > for candidate in sequence: > try: > process_candidate() > except InvalidCandidate: > pass > else: > found = True > break Here, you don't need found at all. I suspect you're not using for...else because C doesn't have it? > > > Or with while loops and counters (or counting-like expressions): > > > i = 1 > while i <= retryCount and not resource.acquired: > if resource.acquire() == FAIL: > sleep( i**2 ) > i += 1 > > > Of course, actual code tends to be more complex, with "keepRunning" > being modified in some branches of "IF" blocks, and there might be > nested loops where the exit condition for the outer one is set in the > inner loop. Compare these two examples: > > found = False > for filePath in glob( '*.data' ): > for line in open( filePath, 'rt' ): > if line.startswith( '#' ): > continue > if handle( line ): > found = True > break > if found: > break > > found = False > for filePath in glob( '*.data' ) while not found: > for line in open( filePath, 'rt' ) while not found: > if line.startswith( '#' ): > continue > if handle( line ): > found = True > > -- Alejandro > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From abarnert at yahoo.com Wed Feb 19 22:26:07 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 19 Feb 2014 13:26:07 -0800 (PST) Subject: [Python-ideas] except expression In-Reply-To: <20140219081458.GD1458@ando> References: <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <20140219081458.GD1458@ando> Message-ID: <1392845167.26088.YahooMailNeo@web181003.mail.ne1.yahoo.com> From: Steven D'Aprano Sent: Wednesday, February 19, 2014 12:14 AM > On Wed, Feb 19, 2014 at 06:06:05PM +1100, Chris Angelico wrote: > > >> Despite the numbers saying that 'as' is almost completely > unnecessary >> to the proposal, I'm leaving it in for the moment, pending more data. >> However, I'm removing two parts of the proposal: bare except clauses >> (use "except BaseException:" if you really want that) and > chaining of >> excepts (use parens, "(expr except Exception1: default1) except >> Exception2: default2"). > > > I think it will be helpful to include a section with a few definitions, > because the term "chaining" is ambiguous. It could mean either: > > * Multiple except clauses in a single expression (that is, n-ary > ? operator, for n > ). Example: > > ? ? (expression except SpamError: spam, > ? ? ? ? ? ? ? ? except EggsError: eggs) > > > where the 5 operands here are > > ? ? - expression > ? ? - SpamError > ? ? - EggsError > ? ? - spam > ? ? - eggs > > > > Or chaining could mean wrapping one except-expression inside another, > such as: > > ? ? (expression except SpamError: spam) except EggsError: eggs > > which has two expressions each with three operands: > > ? ? - expression > ? ? - SpamError > ? ? - spam > > and > > ? ? - (expression except SpamError: spam) > ? ? - EggsError > ? ? - eggs > > > I think it is important to decide which one is chaining, and which one > is not, include it in the PEP as a definition, and stick with it. I > *strongly* suggest that chaining means the second case. That's what > chaining means when we talk about method chaining: > > obj.spam().eggs()? # call eggs method of the result returned by spam > > and exception chaining: > > raise ThisError from some_exception > > I don't think that we should prohibit passing one except-expression as > argument to another. > > The first case, the n-ary form, can be deferred for later. But I don't > think that's chaining. > > It is also important to note that this are not, in general, the same > thing! I think it's better to just avoid the word "chaining" entirely in the PEP, for the same reason it's worth avoiding the n-ary form in the syntax. The reason for banning the n-ary form is that it's ambiguous (to humans?obviously the parser could be trivially coded to pick one interpretation or the other) whether it's two excepts on a single try expression, or one except expression inside another.?To some people the first interpretation seems like the obvious way to read it, the only one consistent with the rest of Python?but there have been people in this discussion who disagree, and surely that will be even more of a problem with novices. The fact that they are not the same thing _in general_, but _often_ have the same effect anyway, just increases the potential for confusion. The term "chaining" is?ambiguous in the same way. To many people who see the first interpretation, "chaining" obviously means the second one. But there are obviously people who disagree?including the author of the PEP?and surely that will again be even more of a problem with novices. So, instead of introducing a potentially confusing term and then defining it, why not just avoid the term? Explain that the n-ary form with multiple except clauses is not allowed (for now); if you want that, _sometimes_ you can use an except expression inside another except expression, but sometimes you will have to rewrite your expression or use the statement form instead. The PEP could even give simple examples of when the substitution does and doesn't work. From masklinn at masklinn.net Wed Feb 19 23:07:29 2014 From: masklinn at masklinn.net (Masklinn) Date: Wed, 19 Feb 2014 23:07:29 +0100 Subject: [Python-ideas] for-while statement In-Reply-To: References: Message-ID: <879E09B7-1EDB-484C-9766-732CC9CBEF03@masklinn.net> On 2014-02-19, at 14:34 , Alejandro L?pez Correa wrote: > Hello, > > I think adding an optional "WHILE" clause in "FOR" loops might be > useful sometimes (shorter code and/or improved readability): It seems judicious application of itertools can do the job. > for #VAR# in #SEQUENCE# while #WATCHDOG_EXPRESSION#: > #CODE_BLOCK# > > Examples: > > keepRunning = True > for i in range(100) while keepRunning: > keepRunning = do_some_work( i ) > > found = False > for candidate in sequence while not found: > try: > process( candidate ) > found = True > except InvalidCandidate: > pass > > retryCount = 7 > for i in range(1,1+retryCount) while resource.acquire() == FAIL: > sleep( i**2 ) > > > At the moment, I usually implement this either with ugly breaks: > > > for i in range(100): > if not do_some_work( i ): > break for i in takewhile(do_some_work, range(100)): pass > > found = False > for candidate in sequence: > try: > process_candidate() > except InvalidCandidate: > pass > else: > found = True > break > for candidate in sequence: try: process_candidate() except InvalidCandidate: pass else: # found break else: # not found > > Or with while loops and counters (or counting-like expressions): > > > i = 1 > while i <= retryCount and not resource.acquired: > if resource.acquire() == FAIL: > sleep( i**2 ) > i += 1 > > > Of course, actual code tends to be more complex, with "keepRunning" > being modified in some branches of "IF" blocks, and there might be > nested loops where the exit condition for the outer one is set in the > inner loop. Compare these two examples: > > found = False > for filePath in glob( '*.data' ): > for line in open( filePath, 'rt' ): > if line.startswith( '#' ): > continue > if handle( line ): > found = True > break > if found: > break > found = False > for filePath in glob( '*.data' ) while not found: > for line in open( filePath, 'rt' ) while not found: > if line.startswith( '#' ): > continue > if handle( line ): > found = True > # itertools's missing piece flatmap = lambda fn, *it: chain.from_iterable(imap(fn, *it)) lines = flatmap(open, iglob('*.data'), repeat('rb') non_comments = ifilter(lambda line: not line.startswith('#'), lines) matches = ifilter(handle, non_comments) match = next(matches, None) Alternatively the filters could be replaced by matches =(line for line in lines if not line.startswith('#') if handle(line)) From ron3200 at gmail.com Wed Feb 19 23:13:56 2014 From: ron3200 at gmail.com (Ron Adam) Date: Wed, 19 Feb 2014 16:13:56 -0600 Subject: [Python-ideas] except expression In-Reply-To: References: <20140219001048.GX4519@ando> <20140219124646.GB3684@ando> Message-ID: On 02/19/2014 11:09 AM, Alexander Belopolsky wrote: > On Wed, Feb 19, 2014 at 11:17 AM, Ron Adam > > wrote: > > value = catch_else(exc, \expr1, \expr2) #alternatives to '\'? > > ? > > .. ducks. Hehe... So you noticed ? and \ are not that different? The one with both legs is the one that takes arguments. The '\' is more agreeable. ;-) From rosuav at gmail.com Wed Feb 19 23:59:31 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 09:59:31 +1100 Subject: [Python-ideas] except expression In-Reply-To: <1392845167.26088.YahooMailNeo@web181003.mail.ne1.yahoo.com> References: <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <20140219081458.GD1458@ando> <1392845167.26088.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: On Thu, Feb 20, 2014 at 8:26 AM, Andrew Barnert wrote: > I think it's better to just avoid the word "chaining" entirely in the PEP, for the same reason it's worth avoiding the n-ary form in the syntax. > Current version of the PEP lacks the word "chain" in any form. There's a section on "Multiple except clauses" (in the "Deferred sub-proposals" section), that's all. https://raw2.github.com/Rosuav/ExceptExpr/master/pep-0463.txt ChrisA From abarnert at yahoo.com Thu Feb 20 00:54:43 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 19 Feb 2014 15:54:43 -0800 (PST) Subject: [Python-ideas] except expression In-Reply-To: References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> Message-ID: <1392854083.25516.YahooMailNeo@web181002.mail.ne1.yahoo.com> From: Chris Angelico Sent: Wednesday, February 19, 2014 5:31 AM > On Thu, Feb 20, 2014 at 12:00 AM, ????? wrote: >> A bit OT: >> I think there should be a mention in the PEP of other languages with >> exception-handling expressions, like SML's >> >> ? ? - (hd []) handle Empty => 1; >> ? ? val it = 1 : int > > I'm not familiar with SML. Want to write up a paragraph that I can > insert directly into the PEP? Actually, any purely-functional languages where a function body is always an expression obviously have to do exception handling as an expression.?So, a survey may be helpful here. It's hard to map other languages to Python because most of them either allow statements inside expressions and take the value of the last statement (which must be an expression statement) as the value, or don't really have statements at all. Also, the different function-calling syntaxes can be very confusing. I'll try my best.?Wikipedia has a page on exception handling syntax (http://en.wikipedia.org/wiki/Exception_handling_syntax) that gives more details of tons of languages, and I'll try to link each one to a relevant docs or tutorial page. --- Ruby (http://www.skorks.com/2009/09/ruby-exceptions-and-exception-handling/) "begin?rescue?rescue?else?ensure?end" is an expression (potentially with statements inside it). It has the equivalent of an "as" clause, and the equivalent of bare except. And it uses no punctuation or keyword between the bare except/exception class/exception class with as clause and the value. (And?yes, it's ambiguous unless you understand Ruby's statement/expression rules.) ? ? x = begin computation() rescue MyException => e default(e) end; ? ? x = begin computation() rescue MyException default() end; ? ? x = begin computation() rescue default() end; ? ? x = begin computation() rescue MyException default() rescue OtherException other() end; In terms of this PEP: ? ? x = computation() except MyException as e default(e) ? ? x = computation() except MyException default(e) ? ? x = computation() except default(e) ? ? x = computation() except MyException default() except OtherException other() --- Erlang (http://erlang.org/doc/reference_manual/expressions.html#id79284) has a try expression that looks like this: ? ? x = try computation() catch MyException:e?-> default(e) end; ? ? x = try computation() catch MyException:e?-> default(e); OtherException:e -> other(e) end; The class and "as" name are mandatory, but you can use "_" for either. There's also an optional "when" guard on each, and a "throw" clause that you can catch, which I won't get into. To handle multiple exceptions, you just separate the clauses with semicolons, which I guess would map to commas in Python. So: ? ? x = try computation() except MyException as e -> default(e) ? ? x = try computation() except MyException as e -> default(e), OtherException as e->other_default(e) Erlang also has a "catch" expression, which, despite using the same keyword, is completely different, and you don't want to know about it. --- The ML family has two different ways of dealing with this, "handle" and "try"; the difference between the two is that "try" pattern-matches the exception, which gives you the effect of multiple except clauses and as clauses. In either form,?the handler clause is punctuated by "=>" in some dialects, "->" in others. To avoid confusion, I'll write the function calls in Python style. Here's SML?(http://www.cs.cmu.edu/~rwh/introsml/core/exceptions.htm)'s "handle": ? ? let x = computation() handle MyException => default();; Here's OCaml?(http://www2.lib.uchicago.edu/keith/ocaml-class/exceptions.html)'s "try": ? ? let x = try computation() with MyException explanation -> default(explanation);; ? ? let x = try computation() with? ? ? ? ? MyException(e) -> default(e)? ? ? ? | MyOtherException() -> other_default() ? ? ? | (e) -> fallback(e);; In terms of this PEP, these would be something like: ? ? x = computation() except MyException => default() ? ? x = try computation() except MyException e -> default() ? ? x = (try computation() ? ? ? ? ?except MyException as e -> default(e)? ? ? ? ? ?except MyOtherException -> other_default() ? ? ? ? ?except BaseException as e -> fallback(e)) Many ML-inspired but not-directly-related languages from academia mix things up, usually using more keywords and fewer symbols. So, the Oz (http://mozart.github.io/mozart-v1/doc-1.4.0/tutorial/node5.html) would map to Python as: ? ? x = try computation() catch MyException as e then default(e) --- Many Lisp-derived languages, like Clojure (http://clojure.org/special_forms#Special%20Forms--(try%20expr*%20catch-clause*%20finally-clause?)), implement?try/catch as special forms (if you don't know what that means, think function-like macros), so you write, effectively: ? ? try(computation(), catch(MyException, explanation, default(explanation))) ? ? try(computation(),? ? ? ? ? catch(MyException, explanation, default(explanation)), ? ? ? ? catch(MyOtherException, explanation, other_default(explanation))) In Common Lisp, this is done with a slightly clunkier "handler-case" macro (http://clhs.lisp.se/Body/m_hand_1.htm), but the basic idea is the same. --- The Lisp style is, surprisingly, used by some languages that don't have macros, like Lua, where?xpcall (http://www.gammon.com.au/scripts/doc.php?lua=xpcall) takes functions. Writing lambdas Python-style instead of Lua-style: ? ? x = xpcall(lambda: expression(), lambda e: default(e)) This actually returns (true, expression()) or (false, default(e)), but I think we can ignore that part. --- Haskell is actually similar to Lua here (except that it's all done with monads, of course): ? ? x = do catch(lambda: expression(), lambda e: default(e)) You can write a pattern matching expression within the function to decide what to do with it; catching and re-raising exceptions you don't want is cheap enough to be idiomatic. But Haskell infixing makes this nicer: ? ? x = do expression() `catch` lambda: default() ? ? x = do expression() `catch` lambda e: default(e) And that makes the parallel between the lambda colon and the except colon in the proposal much more obvious: ? ? x = expression() except Exception: default() ? ? x = expression() except Exception as e: default(e) --- Tcl (http://wiki.tcl.tk/902) has the other half of Lua's xpcall;?catch is a function which returns true if an exception was caught, false otherwise, and you get the value out in other ways. And it's all built around the?the implicit quote-and-exec that everything in Tcl is based on, making it even harder to describe in Python terms than Lisp macros, but something like: ? ? if {[ catch("computation()") "explanation"]} { default(explanation) } --- Smalltalk (http://smalltalk.gnu.org/wiki/exceptions) is also somewhat hard to map to Python. The basic version would be: ? ? x := computation() on:MyException do:default() ? but that's basically Smalltalk's passing-arguments-with-colons syntax, not its exception-handling syntax.? From rosuav at gmail.com Thu Feb 20 01:03:24 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 11:03:24 +1100 Subject: [Python-ideas] except expression In-Reply-To: <1392854083.25516.YahooMailNeo@web181002.mail.ne1.yahoo.com> References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> <1392854083.25516.YahooMailNeo@web181002.mail.ne1.yahoo.com> Message-ID: On Thu, Feb 20, 2014 at 10:54 AM, Andrew Barnert wrote: > Actually, any purely-functional languages where a function body is always an expression obviously have to do exception handling as an expression. So, a survey may be helpful here. > > It's hard to map other languages to Python because most of them either allow statements inside expressions and take the value of the last statement (which must be an expression statement) as the value, or don't really have statements at all. Also, the different function-calling syntaxes can be very confusing. I'll try my best. Wikipedia has a page on exception handling syntax (http://en.wikipedia.org/wiki/Exception_handling_syntax) that gives more details of tons of languages, and I'll try to link each one to a relevant docs or tutorial page. > Thanks for that! I can incorporate that directly. As a part of the PEP, it would be explicitly placed in the public domain, and I or anyone else who edits the PEP would be allowed to edit the text you've written. Just for the record, I want to confirm that you're okay with that :) ChrisA From yselivanov.ml at gmail.com Thu Feb 20 01:22:33 2014 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Wed, 19 Feb 2014 19:22:33 -0500 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> Message-ID: <53054AC9.5060100@gmail.com> On 2/15/2014, 11:04 PM, Chris Angelico wrote: > [snip] > Alternative Proposals > ===================== > > Discussion on python-ideas brought up the following syntax suggestions:: > value = expr except default if Exception [as e] > value = expr except default for Exception [as e] > value = expr except default from Exception [as e] > value = expr except Exception [as e] return default > value = expr except (Exception [as e]: default) > value = expr except Exception [as e] try default > value = expr except Exception [as e] continue with default > value = default except Exception [as e] else expr > value = try expr except Exception [as e]: default > value = expr except Exception [as e] pass default How about adding a new keyword? If we add new 'raises' keyword, then we can have the following new syntax: # set 'value' to 42 if len(a) < 10 # or else, return a[10] value = 42 if a[10] raises IndexError # set 'value' to 42 if len(a) < 10 # or else, return 'spam' value = 42 if a[10] raises IndexError else 'spam' # same as above but if a[10] raises anything value = 42 if a[10] raises # another example foo(None if dct['key'] raises) This syntax doesn't provide: - finally statement replacement; - handling multiple exceptions; - rerising exception; which I think is a good thing. No need to complicate this syntax. Pros: - easy to read (like English or pseudocode) - looks like existing 'expr if expr else expr' syntax Cons: - adds a new keyword Yury From ethan at stoneleaf.us Thu Feb 20 01:15:25 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 19 Feb 2014 16:15:25 -0800 Subject: [Python-ideas] except expression In-Reply-To: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> Message-ID: <5305491D.40301@stoneleaf.us> The three use-cases I find compelling are: # long form try: result = 1/x except ZeroDivisionError: result = NaN # previously defined try: os.unlink(some_file) except OSError: pass try: result = some_func(value1, value2) except SomeError: result = 42 which would be converted to (using Nick's notation): result = 1/x except ZeroDivisionError -> NaN os.unlink(some_file) except OSError -> None result = some_func(value1, value2) except SomeError -> 42 Clean, clear, concise. For the record, I could just as easily live with the colon instead of the arrow. -- ~Ethan~ From rosuav at gmail.com Thu Feb 20 01:56:50 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 11:56:50 +1100 Subject: [Python-ideas] except expression In-Reply-To: <53054AC9.5060100@gmail.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <53054AC9.5060100@gmail.com> Message-ID: On Thu, Feb 20, 2014 at 11:22 AM, Yury Selivanov wrote: > If we add new 'raises' keyword, then we can have the following > new syntax: Cost of adding a keyword is based on its usage. Searching the standard library for 'raises' comes up blank; same goes for 'then', though both 'use' and 'when' do come up (once and twelve times, respectively). I think it reasonably unlikely that 'raises' will be a problem. However, it's very similar to an existing keyword. The English form sounds a little archaic, but plausible: "Do this if spam raise something". > # set 'value' to 42 if len(a) < 10 > # or else, return a[10] > value = 42 if a[10] raises IndexError (Minor point: That should be 42 if len(a) <= 10.) > # set 'value' to 42 if len(a) < 10 > # or else, return 'spam' > value = 42 if a[10] raises IndexError else 'spam' Have you a use-case for this? It completely ignores the original expression value. > # same as above but if a[10] raises anything > value = 42 if a[10] raises I'm giving this an immediate common-law rejection, as the bare-except sub-proposal has already been rejected. > # another example > foo(None if dct['key'] raises) And what if it doesn't? Reads nicely for the exceptional case. Presumably it's the value of dct['key'] if it doesn't raise, but that's not obvious from how it reads. > This syntax doesn't provide: > - finally statement replacement; > - handling multiple exceptions; > - rerising exception; > which I think is a good thing. No need to complicate this > syntax. I'm fine with those absences. > Pros: > - easy to read (like English or pseudocode) > - looks like existing 'expr if expr else expr' syntax > > Cons: > - adds a new keyword That's not a killer, by any means. My main objection here is that it reads backwards. This is an issue with the "if/else" operator in Python, too. The actual evaluation order has to be: 1) Base expression 2) Exception list 3) Default expression I could accept switching 1 and 2, since the exception list will normally be constant anyway (some of the functional forms effectively do this, by having lambdas for the other two but not for the exception_list), but quite a few good use-cases depend on the default expression being calculated lazily. So if the base expression is evaluated before the default expression, it makes better sense to put base to the left of default. That way, the expression reads in the order that it should: >>> [print("1"), print("2")][print("3") or 0] 1 2 3 >>> print("2") if not print("1") else print("n/a") 1 2 There's no changing if/else, but I'm still inclined to support proposals that put the expressions in the "correct order". That's not a clincher on its own, but I want to see a compelling use-case that justifies the out-of-order layout. ChrisA From rosuav at gmail.com Thu Feb 20 02:00:09 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 12:00:09 +1100 Subject: [Python-ideas] except expression In-Reply-To: <5305491D.40301@stoneleaf.us> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> Message-ID: On Thu, Feb 20, 2014 at 11:15 AM, Ethan Furman wrote: > which would be converted to (using Nick's notation): > > result = 1/x except ZeroDivisionError -> NaN > > result = some_func(value1, value2) except SomeError -> 42 These two I strongly support (colon or arrow, either way). > os.unlink(some_file) except OSError -> None This one, not so much. You're using os.unlink(some_file) as a statement, ignoring its return value. Changing its return value to None in the case of an exception isn't exactly what you're trying to do. Yes, it does make for a one-liner, where the full form takes two: try: os.unlink(some_file) except OSError: pass but I'm not sure that it's the right way to do things. ChrisA From ncoghlan at gmail.com Thu Feb 20 02:09:59 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 20 Feb 2014 11:09:59 +1000 Subject: [Python-ideas] except expression In-Reply-To: <5305491D.40301@stoneleaf.us> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> Message-ID: On 20 February 2014 10:15, Ethan Furman wrote: > try: > os.unlink(some_file) > except OSError: > pass Note that Python 3.4+ (and previous versions once I get around to doing a new contextlib2 release) allows the statement case to be written as the one-liner: with suppress(OSError): os.unlink(some_file) I don't think this PEP needs to worry about that case, but Chris may want to explicitly reference contextlib.suppress as being preferred to "os.unlink(some_file) except OSError -> None" (or 0 or ... or whatever other dummy placeholder someone decided to use). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From yselivanov.ml at gmail.com Thu Feb 20 02:11:30 2014 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Wed, 19 Feb 2014 20:11:30 -0500 Subject: [Python-ideas] except expression In-Reply-To: References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <53054AC9.5060100@gmail.com> Message-ID: <53055642.3010801@gmail.com> On 2/19/2014, 7:56 PM, Chris Angelico wrote: > On Thu, Feb 20, 2014 at 11:22 AM, Yury Selivanov > wrote: >> If we add new 'raises' keyword, then we can have the following >> new syntax: > Cost of adding a keyword is based on its usage. Searching the standard > library for 'raises' comes up blank; same goes for 'then', though both > 'use' and 'when' do come up (once and twelve times, respectively). I > think it reasonably unlikely that 'raises' will be a problem. However, > it's very similar to an existing keyword. The English form sounds a > little archaic, but plausible: "Do this if spam raise something". > >> # set 'value' to 42 if len(a) < 10 >> # or else, return a[10] >> value = 42 if a[10] raises IndexError > (Minor point: That should be 42 if len(a) <= 10.) > >> # set 'value' to 42 if len(a) < 10 >> # or else, return 'spam' >> value = 42 if a[10] raises IndexError else 'spam' > Have you a use-case for this? It completely ignores the original > expression value. Something like result = 'failed' if command() raises else 'ok' > >> # same as above but if a[10] raises anything >> value = 42 if a[10] raises > I'm giving this an immediate common-law rejection, as the bare-except > sub-proposal has already been rejected. Since it is a new keyword, we can document that it can intercept only Exceptions, not BaseExceptions. > >> # another example >> foo(None if dct['key'] raises) > And what if it doesn't? Reads nicely for the exceptional case. > Presumably it's the value of dct['key'] if it doesn't raise, but > that's not obvious from how it reads. Without this you would need to write: foo(None if dct['key'] raises else dct[key]) Which is code duplication and a performance impact, since we'll have to evaluate expression twice. >> This syntax doesn't provide: >> - finally statement replacement; >> - handling multiple exceptions; >> - rerising exception; >> which I think is a good thing. No need to complicate this >> syntax. > I'm fine with those absences. > >> Pros: >> - easy to read (like English or pseudocode) >> - looks like existing 'expr if expr else expr' syntax >> >> Cons: >> - adds a new keyword > That's not a killer, by any means. > > My main objection here is that it reads backwards. Yes, that's understandable. Although, as you yourself noted, Python has 'expr if expr else expr' already. Yes, it is a bit awkward, but lots of people (including me) find it very readable and easy to understand. Same with my proposal. If follows the already established syntax and reads nicely. Could you please add to the PEP (in other proposals section?) Yury From rosuav at gmail.com Thu Feb 20 02:18:07 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 12:18:07 +1100 Subject: [Python-ideas] except expression In-Reply-To: <5305491D.40301@stoneleaf.us> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> Message-ID: On Thu, Feb 20, 2014 at 11:15 AM, Ethan Furman wrote: > result = 1/x except ZeroDivisionError -> NaN > > For the record, I could just as easily live with the colon instead of the > arrow. > Time to open up this branch of the discussion... colon or arrow? For the purposes of this debate, I'm comparing these two notations, and nothing else: result = 1/x except ZeroDivisionError -> NaN result = 1/x except ZeroDivisionError: NaN Advantages of the colon include: * It's already in use in most programs (how many Python programmers even know about the arrow?) * Parallel with the way lambda uses a colon to separate its arg list from a sub-expression * Parallel with statement form of "except X:" * Compactness (one character and a space to its right, vs two and spaces both sides) - a good thing, as this notation tends to want to be on one line Advantages of the arrow include -> * Splits nicely at the arrow: "1/x except ZeroDivisionError\n\t-> NaN" - doing this with a colon would look like introducing a suite, which would be extremely confusing * Unique syntax - can't be confused with starting a suite or initializing a dict * Comparable amount of visual separation between the three parts. Like with "if/else", where there's a whole word separating each part, the arrow gives a decent separator between the exception and the default. The colon binds the two closely together. * Distinguishes itself from the statement "except X:", which introduces more statements. Both proposals fit into my preferred layout order (meaning the evaluation order is strictly left to right). Neither adds new keywords to the language. Both put the word "except" immediately ahead of the exception list. Either would be plausible. ChrisA From rosuav at gmail.com Thu Feb 20 02:25:53 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 12:25:53 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> Message-ID: On Thu, Feb 20, 2014 at 12:09 PM, Nick Coghlan wrote: > On 20 February 2014 10:15, Ethan Furman wrote: >> try: >> os.unlink(some_file) >> except OSError: >> pass > > Note that Python 3.4+ (and previous versions once I get around to > doing a new contextlib2 release) allows the statement case to be > written as the one-liner: > > with suppress(OSError): os.unlink(some_file) > > I don't think this PEP needs to worry about that case, but Chris may > want to explicitly reference contextlib.suppress as being preferred to > "os.unlink(some_file) except OSError -> None" (or 0 or ... or whatever > other dummy placeholder someone decided to use). Thank you, yes. I'll add that message and move the whole idea to the rejected section. ChrisA From ethan at stoneleaf.us Thu Feb 20 02:05:43 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 19 Feb 2014 17:05:43 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> Message-ID: <530554E7.7060102@stoneleaf.us> On 02/19/2014 05:00 PM, Chris Angelico wrote: > On Thu, Feb 20, 2014 at 11:15 AM, Ethan Furman wrote: >> which would be converted to (using Nick's notation): >> >> result = 1/x except ZeroDivisionError -> NaN >> >> result = some_func(value1, value2) except SomeError -> 42 > > These two I strongly support (colon or arrow, either way). > >> os.unlink(some_file) except OSError -> None > > This one, not so much. You're using os.unlink(some_file) as a > statement, ignoring its return value. Changing its return value to > None in the case of an exception isn't exactly what you're trying to > do. Yes, it does make for a one-liner, where the full form takes two: > > try: os.unlink(some_file) > except OSError: pass > > but I'm not sure that it's the right way to do things. 1) It could be used inside a function call or assigned to a variable; 2) We're creating a new expression type -- it'll be valid anywhere an expression is valid, even all by itself. e.g. `1 + 2` is valid, even though nothing is done with the result. -- ~Ethan~ From ethan at stoneleaf.us Thu Feb 20 01:43:20 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 19 Feb 2014 16:43:20 -0800 Subject: [Python-ideas] except expression In-Reply-To: <53054AC9.5060100@gmail.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <53054AC9.5060100@gmail.com> Message-ID: <53054FA8.9070708@stoneleaf.us> On 02/19/2014 04:22 PM, Yury Selivanov wrote: > > If we add new 'raises' keyword, then we can have the following > new syntax: > > # set 'value' to 42 if len(a) < 10 > # or else, return a[10] > value = 42 if a[10] raises IndexError > > # set 'value' to 42 if len(a) < 10 > # or else, return 'spam' > value = 42 if a[10] raises IndexError else 'spam' > > # same as above but if a[10] raises anything > value = 42 if a[10] raises > > # another example > foo(None if dct['key'] raises) The big problems I see with this idea are that: - the emphasis is placed on the unusual rather than the normal result - it's not clear what the normal result should be For the second point, consider: value = 42 if a[10] raises There is nothing there to indicate that the value of a[10] is what should be used if no exception is raised. -1 -- ~Ethan~ From abarnert at yahoo.com Thu Feb 20 02:34:26 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 19 Feb 2014 17:34:26 -0800 (PST) Subject: [Python-ideas] except expression In-Reply-To: <530554E7.7060102@stoneleaf.us> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <530554E7.7060102@stoneleaf.us> Message-ID: <1392860066.60595.YahooMailNeo@web181006.mail.ne1.yahoo.com> From: Ethan Furman Sent: Wednesday, February 19, 2014 5:05 PM > Subject: Re: [Python-ideas] except expression > > On 02/19/2014 05:00 PM, Chris Angelico wrote: >> On Thu, Feb 20, 2014 at 11:15 AM, Ethan Furman > wrote: >>> which would be converted to (using Nick's notation): >>> >>> result = 1/x except ZeroDivisionError -> NaN >>> >>> result = some_func(value1, value2) except SomeError -> 42 >> >> These two I strongly support (colon or arrow, either way). >> >>> os.unlink(some_file) except OSError -> None >> >> This one, not so much. You're using os.unlink(some_file) as a >> statement, ignoring its return value. Changing its return value to >> None in the case of an exception isn't exactly what you're trying > to >> do. Yes, it does make for a one-liner, where the full form takes two: >> >> try: os.unlink(some_file) >> except OSError: pass >> >> but I'm not sure that it's the right way to do things. > > 1) It could be used inside a function call or assigned to a variable; Why would you use an expression that always evaluates to None (with or without an except handler that also always evaluates to None) in a function call or assignment, or really anywhere else but a simple expression statement? > 2) We're creating a new expression type -- it'll be valid anywhere an > expression is valid, even all by itself.? e.g. `1 > + 2` is valid, even though nothing is done with the result. Sure, it's valid, but it's also useless (except in the interactive interpreter, where something _is_ done with the result), so we don't need to add syntax to make it more powerful. Twice as powerful as useless is still useless. So, while there's no need to actually ban "os.unlink(some_file) except OSError: None", it doesn't make a good argument for the PEP. From rosuav at gmail.com Thu Feb 20 02:48:33 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 12:48:33 +1100 Subject: [Python-ideas] except expression In-Reply-To: <53055642.3010801@gmail.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <53054AC9.5060100@gmail.com> <53055642.3010801@gmail.com> Message-ID: On Thu, Feb 20, 2014 at 12:11 PM, Yury Selivanov wrote: > > On 2/19/2014, 7:56 PM, Chris Angelico wrote: >> >> On Thu, Feb 20, 2014 at 11:22 AM, Yury Selivanov >>> # set 'value' to 42 if len(a) < 10 >>> # or else, return 'spam' >>> value = 42 if a[10] raises IndexError else 'spam' >> >> Have you a use-case for this? It completely ignores the original >> expression value. > > Something like > result = 'failed' if command() raises else 'ok' Yep, I can see the theory of it. What I mean is, do you have actual production code that looks like this? Concrete examples are usually easier to argue based on. How would you currently spell it? try: command() result = 'ok' except SomeException: result = 'failed' Do you have code like that? Part of the question here is that, if the thing you're testing is going to have its return value ignored anyway, does it need to be an expression? I mean, why not do this: result = 'div by zero' if x += 1/y raises else 'no error' which would be some serious pain for the parser. But having an expression that deliberately ignores one of its subexpressions is a bit weird. We don't usually write this, unless we're going for obfuscation: value = (l.append("x"), l)[-1] Although I could imagine that a MacroPy trick be done to make that more useful, eg for method chaining: value = (l.append("x"), l)[-1].append("y") But for exception handling where you don't care about the value of the expression, there's already an obvious way to spell it, and that's the statement form. So I'm looking to see a compelling use-case from production code that shows the benefit of this notation. >>> # same as above but if a[10] raises anything >>> value = 42 if a[10] raises >> >> I'm giving this an immediate common-law rejection, as the bare-except >> sub-proposal has already been rejected. > > Since it is a new keyword, we can document that it > can intercept only Exceptions, not BaseExceptions. Yes, but overly-broad catching is a big problem even without catching "Exception..BaseException" (if I may use the git notation here - meaning "everything reachable from BaseException but not from Exception"). Earlier in this discussion we proposed having a bare except catch some specific set of "common exceptions". The results of that discussion are in the PEP's rejection section; one of the largest objections doesn't apply here (consistency with the statement form bare except), but the others do. >>> # another example >>> foo(None if dct['key'] raises) >> >> And what if it doesn't? Reads nicely for the exceptional case. >> Presumably it's the value of dct['key'] if it doesn't raise, but >> that's not obvious from how it reads. > > Without this you would need to write: > foo(None if dct['key'] raises else dct[key]) > > Which is code duplication and a performance impact, > since we'll have to evaluate expression twice. Right, not to mention a correctness error if the evaluation has side effects. But if you read it in English, there does need to be an "else" clause, otherwise you're left with the "And what if it doesn't?" that I started with. It reads very nicely for the exceptional case, but much less nicely for the normal case. I don't dispute the semantics though. >> My main objection here is that it reads backwards. > > Yes, that's understandable. > > Although, as you yourself noted, Python has > 'expr if expr else expr' already. Yes, it is a bit awkward, > but lots of people (including me) find it very readable and > easy to understand. Yes, but is that a good thing? The ternary if is justified because it reads well in English, but even so, it does trap people. I am listening to proposals that put the operands in the "wrong order", but the bar is higher - they have to justify that potential confusion. > Same with my proposal. If follows the already established > syntax and reads nicely. > > Could you please add to the PEP (in other proposals section?) It's in there. I split it into two parts: firstly, a nearly-equivalent proposal that uses the archaic "raise" (as that's already a keyword), and then a comment underneath about creating a new keyword. ChrisA From abarnert at yahoo.com Thu Feb 20 02:51:28 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 19 Feb 2014 17:51:28 -0800 (PST) Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> Message-ID: <1392861088.22517.YahooMailNeo@web181002.mail.ne1.yahoo.com> From: Chris Angelico Sent: Wednesday, February 19, 2014 5:18 PM > Subject: Re: [Python-ideas] except expression > > On Thu, Feb 20, 2014 at 11:15 AM, Ethan Furman wrote: >> result = 1/x except ZeroDivisionError -> NaN >> >> For the record, I could just as easily live with the colon instead of the >> arrow. >> > > Time to open up this branch of the discussion... colon or arrow? > > For the purposes of this debate, I'm comparing these two notations, > and nothing else: > > result = 1/x except ZeroDivisionError -> NaN > result = 1/x except ZeroDivisionError: NaN I hated the colon at first, but after seeing it in more simple cases like this one, I'm starting to warm to it. And I definitely like it better than the arrow. For reasons you mostly already elaborated: > Advantages of the colon include: > * It's already in use in most programs (how many Python programmers > even know about the arrow?) Yes. The colon "looks more like Python" here?maybe only because you don't see function annotations nearly as often as?compound statements, lambda expressions, dict displays, etc., but that's still compelling. Also, I've never really liked -> anywhere, but that may just be because I first used that in ML, and we were using a textbook that used -> but an implementation that used =>, so feel free to discount that too. > * Parallel with the way lambda uses a colon to separate its arg list > from a sub-expression I'm not sure how much this is worth. It loomed big in my head right after trying to write the most pythonic except-expression in Haskell and then translating it to Python. But that's equivalent to the fact that if you wanted to do this with a function instead of syntax you'd have to use a lambda, which is just as true of the if expression or a short-circuiting or or and expression, which don't need colons. > * Parallel with statement form of "except X:" I think this is the one that sells it to me. > * Compactness (one character and a space to its right, vs two and > spaces both sides) - a good thing, as this notation tends to want to > be on one line >? > Advantages of the arrow include -> > * Splits nicely at the arrow: "1/x except > ZeroDivisionError\n\t-> NaN" > - doing this with a colon would look like introducing a suite, which > would be extremely confusing > * Unique syntax - can't be confused with starting a suite or initializing a > dict > * Comparable amount of visual separation between the three parts. Like > with "if/else", where there's a whole word separating each part, > the > arrow gives a decent separator between the exception and the default. > The colon binds the two closely together. > * Distinguishes itself from the statement "except X:", which > introduces more statements. > > Both proposals fit into my preferred layout order (meaning the > evaluation order is strictly left to right). Neither adds new keywords > to the language. Both put the word "except" immediately ahead of the > exception list. Either would be plausible. > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > From rosuav at gmail.com Thu Feb 20 02:56:50 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 12:56:50 +1100 Subject: [Python-ideas] except expression In-Reply-To: <530554E7.7060102@stoneleaf.us> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <530554E7.7060102@stoneleaf.us> Message-ID: On Thu, Feb 20, 2014 at 12:05 PM, Ethan Furman wrote: > On 02/19/2014 05:00 PM, Chris Angelico wrote: >> >> On Thu, Feb 20, 2014 at 11:15 AM, Ethan Furman wrote: >>> os.unlink(some_file) except OSError -> None >> >> Yes, it does make for a one-liner, where the full form takes two: >> >> try: os.unlink(some_file) >> except OSError: pass >> >> but I'm not sure that it's the right way to do things. > > > 1) It could be used inside a function call or assigned to a variable; If you're using the return value, then sure. That's exactly what this is for. But as far as I know, os.unlink only ever returns None, so saying "and if it raises OSError, return None instead" doesn't make a lot of sense. > 2) We're creating a new expression type -- it'll be valid anywhere an > expression is valid, even all by itself. e.g. `1 + 2` is valid, even though > nothing is done with the result. Oh yes, I'm not going to *prevent* this sort of thing. It's no different from reworking your import statements into __import__ calls just so you can use ternary if to handle versions: tkinter = __import__("Tkinter" if sys.version_info.major==2 else "tkinter") Is it good code? I don't think so. Does anything stop you from doing it? Of course not. And if you really wanted to, you could probably turn your entire Python program into a single megantic (that's like gigantic only not so much - next one down is kilantic, and below that you just have antics - see also http://gatherer.wizards.com/Pages/Card/Details.aspx?multiverseid=370794) lambda, make everything into expressions, and do all assignments by declaring functions and calling them. But it wouldn't be Pythonic code. ChrisA From rob.cliffe at btinternet.com Thu Feb 20 03:49:14 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Thu, 20 Feb 2014 02:49:14 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> Message-ID: <53056D2A.3070905@btinternet.com> On 20/02/2014 01:09, Nick Coghlan wrote: > On 20 February 2014 10:15, Ethan Furman wrote: >> try: >> os.unlink(some_file) >> except OSError: >> pass > Note that Python 3.4+ (and previous versions once I get around to > doing a new contextlib2 release) allows the statement case to be > written as the one-liner: > > with suppress(OSError): os.unlink(some_file) > > I don't think this PEP needs to worry about that case, but Chris may > want to explicitly reference contextlib.suppress as being preferred to > "os.unlink(some_file) except OSError -> None" (or 0 or ... or whatever > other dummy placeholder someone decided to use). > > Cheers, > Nick. > Can I put in another plug for allowing: os.unlink(some_file) except OSError: pass Cons: It is anomalous. It only makes sense in an expression whose value is not used. Pros: It is very readable. It is a common use case. But maybe it should go, if anywhere, in a separate PEP. Thanks, Rob Cliffe From ethan at stoneleaf.us Thu Feb 20 03:29:43 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 19 Feb 2014 18:29:43 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> Message-ID: <53056897.9060106@stoneleaf.us> On 02/19/2014 05:25 PM, Chris Angelico wrote: > On Thu, Feb 20, 2014 at 12:09 PM, Nick Coghlan wrote: >> On 20 February 2014 10:15, Ethan Furman wrote: >>> try: >>> os.unlink(some_file) >>> except OSError: >>> pass >> >> Note that Python 3.4+ (and previous versions once I get around to >> doing a new contextlib2 release) allows the statement case to be >> written as the one-liner: >> >> with suppress(OSError): os.unlink(some_file) >> >> I don't think this PEP needs to worry about that case, but Chris may >> want to explicitly reference contextlib.suppress as being preferred to >> "os.unlink(some_file) except OSError -> None" (or 0 or ... or whatever >> other dummy placeholder someone decided to use). > > Thank you, yes. I'll add that message and move the whole idea to the > rejected section. Careful how you word that. "Rejected" makes it sound like it won't work, not that it will but is not the best way. -- ~Ethan~ From greg.ewing at canterbury.ac.nz Thu Feb 20 03:54:50 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 20 Feb 2014 15:54:50 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <20140219001048.GX4519@ando> <20140219124646.GB3684@ando> Message-ID: <53056E7A.40509@canterbury.ac.nz> On 20/02/14 02:01, Paul Moore wrote: > EXPR except EXCEPTION return FALLBACK > > Here, EXPR is the "expected" result, but if you get EXCEPTION when > evaluating it then use FALLBACK instead. I don't think there's any point in forcing things to be in the same order as the if-expression if it requires mangling English grammar. The reason we ended up with the oddly-ordered if-expression in the first place is that it *avoided* mangled English while staying within the existing set of keywords. To introduce another construct that mangles English and/or abuses keywords just to match the odd ordering of the if-expression would be a truly foolish consistency. -- Greg From greg.ewing at canterbury.ac.nz Thu Feb 20 03:58:28 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 20 Feb 2014 15:58:28 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> Message-ID: <53056F54.1030305@canterbury.ac.nz> Another idea: things[i] or else None if IndexError You can't claim that the exception is in the wrong place in relation to the 'except' keyword, because there is no 'except' keyword! -- Greg From stephen at xemacs.org Thu Feb 20 04:02:50 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 20 Feb 2014 12:02:50 +0900 Subject: [Python-ideas] except expression In-Reply-To: <53054AC9.5060100@gmail.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <53054AC9.5060100@gmail.com> Message-ID: <878ut6wko5.fsf@uwakimon.sk.tsukuba.ac.jp> Yury Selivanov writes: > value = 42 if a[10] raises IndexError "If EXPR raises EXCEPTION" is often interpreted to mean "EXPR may raise EXCEPTION" following the usage in Java and other languages where exceptions are part of function signatures. Maybe it's just me, but this ambiguity would be annoying every time I see this syntax. > > # set 'value' to 42 if len(a) < 10 > # or else, return 'spam' > value = 42 if a[10] raises IndexError else 'spam' > > # same as above but if a[10] raises anything > value = 42 if a[10] raises Please, no bare excepts. They're bad enough in the statement form where the colon immediately follows and it's painfully obvious they're bare. > # another example > foo(None if dct['key'] raises) And what's the value if dct['key'] quietly succeeds? I suppose it's dct['key'], but that is not at all what the English interpretation would be! At least in my dialect, the English interpretation would correspond to try: dct['key'] foo() except: foo(None) From stephen at xemacs.org Thu Feb 20 04:13:00 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 20 Feb 2014 12:13:00 +0900 Subject: [Python-ideas] except expression In-Reply-To: <53056897.9060106@stoneleaf.us> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53056897.9060106@stoneleaf.us> Message-ID: <8761oawk77.fsf@uwakimon.sk.tsukuba.ac.jp> Ethan Furman writes: > > Thank you, yes. I'll add that message and move the whole idea to the > > rejected section. > > Careful how you word that. "Rejected" makes it sound like it won't > work, not that it will but is not the best way. Nope. "Rejected" means a decision was made. It doesn't imply anything about the reason for the decision. If you think it does, that's your problem. Chris's usage was perfectly fine in the context of python-dev practice. From greg.ewing at canterbury.ac.nz Thu Feb 20 04:13:56 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 20 Feb 2014 16:13:56 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> Message-ID: <530572F4.9020902@canterbury.ac.nz> On 19/02/14 22:04, David Mertz wrote: > I don't know if this really amounts to a *strong* objection. To me, as much > as I try to like it reading this thread, the colon just continues to feel > wrong to me. Yes, of course I know the analogy with lambda, and I can even > see a certain analogy with dict literals. However, far more compelling to me > is making it look more like the ternary expression (which it is really > basically a special case of. The only colon-less version I've seen so far that I could live with would be things[i] except None if IndexError Attempts to use commas instead of keywords are visually confusing, and abuses of keywords like "except ... return" and "except ... pass" just look horrible to me. Yes, it's inconsistent with the way things are ordered in the try statement, but I don't think that's necessarily a fatal flaw. It reads like English, so it's fairly obvious to anyone reading it what the intended meaning is, and if you find yourself writing things[i] except IndexError if None and you have even half your wits about you, then you'll notice that something doesn't make sense when you get to the 'if'. Also if you try to write something like things[i] except IndexError else None you'll find out it's wrong pretty quickly via a SyntaxError. One other objection might be that if you want an 'as' clause it would have to be things[i] except f(e) if IndexError as e which puts the binding of e after its use. But that's not unprecedented -- comprehensions do this too. -- Greg From greg.ewing at canterbury.ac.nz Thu Feb 20 04:14:59 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 20 Feb 2014 16:14:59 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <20140219001048.GX4519@ando> <20140219124646.GB3684@ando> Message-ID: <53057333.5090707@canterbury.ac.nz> On 20/02/14 11:13, Ron Adam wrote: > > On 02/19/2014 11:09 AM, Alexander Belopolsky wrote: > >> ? >> >> .. ducks. > > Hehe... So you noticed ? and \ are not that different? > > The one with both legs is the one that takes arguments. So '\' is a duck standing on one leg? -- Greg From yselivanov.ml at gmail.com Thu Feb 20 04:55:58 2014 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Wed, 19 Feb 2014 22:55:58 -0500 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> Message-ID: <53057CCE.2050505@gmail.com> On 2/19/2014, 8:18 PM, Chris Angelico wrote: > Advantages of the arrow include -> I would refrain from introducing a new operator here, especially '->'. This one may later be used for communication between channels/queues or other async stuff (like in golang, although they use '<-'.) Yury From rosuav at gmail.com Thu Feb 20 05:09:34 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 15:09:34 +1100 Subject: [Python-ideas] except expression In-Reply-To: <53056897.9060106@stoneleaf.us> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53056897.9060106@stoneleaf.us> Message-ID: On Thu, Feb 20, 2014 at 1:29 PM, Ethan Furman wrote: >> Thank you, yes. I'll add that message and move the whole idea to the >> rejected section. > > > Careful how you word that. "Rejected" makes it sound like it won't work, > not that it will but is not the best way. > What I'm rejecting is the proposal for: statement except Expression: pass Based on the current proposal, that won't work. It makes reasonable sense, but we can't do everything. As Rob says, that may be better done separately; it's really an alternative short-hand for the statement form, not an expression form. This will work, but is not recommended: expression except Expression: None That's not part of the rejected section. ChrisA From rosuav at gmail.com Thu Feb 20 05:14:13 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 15:14:13 +1100 Subject: [Python-ideas] except expression In-Reply-To: <53056F54.1030305@canterbury.ac.nz> References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <53056F54.1030305@canterbury.ac.nz> Message-ID: On Thu, Feb 20, 2014 at 1:58 PM, Greg Ewing wrote: > Another idea: > > things[i] or else None if IndexError > > You can't claim that the exception is in the wrong place > in relation to the 'except' keyword, because there is no > 'except' keyword! Added to the Alternative Proposals section for completeness, but I don't particularly like this. It uses three keywords which currently all occur in expression contexts; whether the parser can handle it or not, I suspect there'll be a lot of odd cases where a human has trouble reading it. ChrisA From rosuav at gmail.com Thu Feb 20 05:16:51 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 15:16:51 +1100 Subject: [Python-ideas] except expression In-Reply-To: <8761oawk77.fsf@uwakimon.sk.tsukuba.ac.jp> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53056897.9060106@stoneleaf.us> <8761oawk77.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Thu, Feb 20, 2014 at 2:13 PM, Stephen J. Turnbull wrote: > Ethan Furman writes: > > > > Thank you, yes. I'll add that message and move the whole idea to the > > > rejected section. > > > > Careful how you word that. "Rejected" makes it sound like it won't > > work, not that it will but is not the best way. > > Nope. "Rejected" means a decision was made. It doesn't imply anything > about the reason for the decision. If you think it does, that's your > problem. Chris's usage was perfectly fine in the context of python-dev > practice. And by the way, reasons for decisions are often going into the PEP itself; when I make a quick comment on list about a rejection, I won't repeat everything that I've said there, so you'd need to check the draft PEP for the reasoning. ChrisA From greg.ewing at canterbury.ac.nz Thu Feb 20 05:17:23 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 20 Feb 2014 17:17:23 +1300 Subject: [Python-ideas] except expression In-Reply-To: <5305491D.40301@stoneleaf.us> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> Message-ID: <530581D3.7@canterbury.ac.nz> Ethan Furman wrote: > result = 1/x except ZeroDivisionError -> NaN > > os.unlink(some_file) except OSError -> None > > result = some_func(value1, value2) except SomeError -> 42 I would have a hard time getting used to this usage of '->'. Nothing else in Python uses it in an expresson like this. It just seems very arbitrary. Here's another idea: result = 1/x except {ZeroDivisionError: NaN} You can read the part in {...} as a dictionary (although it needn't be implemented that way) so it doesn't use colons in any way they aren't being used already. -- Greg From rosuav at gmail.com Thu Feb 20 05:18:25 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 15:18:25 +1100 Subject: [Python-ideas] except expression In-Reply-To: <530572F4.9020902@canterbury.ac.nz> References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> <530572F4.9020902@canterbury.ac.nz> Message-ID: On Thu, Feb 20, 2014 at 2:13 PM, Greg Ewing wrote: > things[i] except IndexError if None > > and you have even half your wits about you, then you'll > notice that something doesn't make sense when you get to > the 'if'. Actually, you have to keep going to see if you hit an 'else', because "IndexError if None else something_else" is the same as "something_else". ChrisA From greg.ewing at canterbury.ac.nz Thu Feb 20 05:23:27 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 20 Feb 2014 17:23:27 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> <530572F4.9020902@canterbury.ac.nz> Message-ID: <5305833F.2010203@canterbury.ac.nz> Chris Angelico wrote: > On Thu, Feb 20, 2014 at 2:13 PM, Greg Ewing wrote: > >> things[i] except IndexError if None >> >>and you have even half your wits about you, then you'll >>notice that something doesn't make sense when you get to >>the 'if'. > > Actually, you have to keep going to see if you hit an 'else', because > "IndexError if None else something_else" is the same as > "something_else". For sanity, parentheses should probably be required for that interpretation: things[i] except (value if cond else other_value) if IndexError -- Greg From rosuav at gmail.com Thu Feb 20 05:24:03 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 15:24:03 +1100 Subject: [Python-ideas] except expression In-Reply-To: <53057CCE.2050505@gmail.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53057CCE.2050505@gmail.com> Message-ID: On Thu, Feb 20, 2014 at 2:55 PM, Yury Selivanov wrote: > On 2/19/2014, 8:18 PM, Chris Angelico wrote: >> >> Advantages of the arrow include -> > > I would refrain from introducing a new operator here, > especially '->'. > > This one may later be used for communication between > channels/queues or other async stuff (like in golang, > although they use '<-'.) Technically, -> does exist in the language: http://www.python.org/dev/peps/pep-3107/ I don't know if it's still available for use as an operator, but I'd be extremely cautious about using <- as an operator; it could be binary less-than followed by unary minus: >>> 1 <- 2 False Demanding space around binary operators is the job of style guides, but creating <- as an operator would mean first moving that demand into the language (at least in some places). ChrisA From rosuav at gmail.com Thu Feb 20 05:33:00 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 15:33:00 +1100 Subject: [Python-ideas] except expression In-Reply-To: <530581D3.7@canterbury.ac.nz> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <530581D3.7@canterbury.ac.nz> Message-ID: On Thu, Feb 20, 2014 at 3:17 PM, Greg Ewing wrote: > Here's another idea: > > result = 1/x except {ZeroDivisionError: NaN} > > You can read the part in {...} as a dictionary > (although it needn't be implemented that way) > so it doesn't use colons in any way they aren't > being used already. If it even might be implemented that way, the default clause(s) would have to be eagerly evaluated. I'd like to avoid that, for several reasons: 1) In a try/except statement, the suites are parsed "try first, then except". 2) Fixing this difference with dict.get() is a valuable improvement - see the main Proposal section of the PEP. 3) Performance would be unnecessarily impacted by evaluating all the default values. 4) It makes good sense for a conditionally-executed section to be allowed to depend on its condition. Just as you can write "1/x if x else NaN", confident that it won't evaluate "1/x" unless x, you should be able to write "1/x except ZeroDivisionError: ask_user_to_choose(NaN, 0, Inf)" and depend on the user being asked only when the division by zero happens. Restricting this would force people to go back to the statement form, while leaving the code looking like it should be converted. Conversely, if you retain lazy evaluation while using a very dict-like notation will confuse people enormously. This would be... a dictionary that maps exception types to unevaluated expressions? Why can't I use one of those somewhere else!!! ChrisA From yselivanov.ml at gmail.com Thu Feb 20 05:37:00 2014 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Wed, 19 Feb 2014 23:37:00 -0500 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53057CCE.2050505@gmail.com> Message-ID: <5305866C.4010306@gmail.com> On 2/19/2014, 11:24 PM, Chris Angelico wrote: > On Thu, Feb 20, 2014 at 2:55 PM, Yury Selivanov wrote: >> On 2/19/2014, 8:18 PM, Chris Angelico wrote: >>> Advantages of the arrow include -> >> I would refrain from introducing a new operator here, >> especially '->'. >> >> This one may later be used for communication between >> channels/queues or other async stuff (like in golang, >> although they use '<-'.) > Technically, -> does exist in the language: I know, and this is another reason to not to use '->' for the except expression. > > http://www.python.org/dev/peps/pep-3107/ > > I don't know if it's still available for use as an operator, but I'd > be extremely cautious about using <- as an operator; it could be > binary less-than followed by unary minus: Sorry, I can't find where I suggested to use '<-' in Python. Yury From rosuav at gmail.com Thu Feb 20 05:41:32 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 15:41:32 +1100 Subject: [Python-ideas] except expression In-Reply-To: <5305833F.2010203@canterbury.ac.nz> References: <5303789A.7040406@mrabarnett.plus.com> <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> <530572F4.9020902@canterbury.ac.nz> <5305833F.2010203@canterbury.ac.nz> Message-ID: On Thu, Feb 20, 2014 at 3:23 PM, Greg Ewing wrote: > Chris Angelico wrote: >> >> On Thu, Feb 20, 2014 at 2:13 PM, Greg Ewing >> wrote: >> >>> things[i] except IndexError if None >>> >>> and you have even half your wits about you, then you'll >>> notice that something doesn't make sense when you get to >>> the 'if'. >> >> >> Actually, you have to keep going to see if you hit an 'else', because >> "IndexError if None else something_else" is the same as >> "something_else". > > > For sanity, parentheses should probably be required > for that interpretation: > > things[i] except (value if cond else other_value) if IndexError Unless you require them all the time, parsing's going to have to go all the way to the end of the expression before being sure of its interpretation. I'm not sure that's a good thing. Again, it may or may not be a problem for a computer lexer, but a human would have to remember to look for the else, just in case. Of course, I don't often expect people to be often writing stuff like this, but it is technically legal: things[i] except None if IndexError if issubclass(things, list) else KeyError Not to mention that this case is better handled by catching LookupError - but not everyone knows about that. (The normal rule against catching more than you expect isn't likely to be a problem here. But I could imagine someone specifically avoiding LookupError in case the other sort of error gets thrown somehow.) Yes, this should be parenthesized; but operator precedence rules mean that this must have one of two meanings: (things[i] except None if IndexError) if issubclass(things, list) else KeyError things[i] except None if (IndexError if issubclass(things, list) else KeyError) Would you know, without looking up a precedence table, which this is? It's clear which one the programmer intended, but would you know whether the parser treats it the same way? Yet there must be an order to them. ChrisA From rosuav at gmail.com Thu Feb 20 05:53:14 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 15:53:14 +1100 Subject: [Python-ideas] except expression In-Reply-To: <5305866C.4010306@gmail.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53057CCE.2050505@gmail.com> <5305866C.4010306@gmail.com> Message-ID: On Thu, Feb 20, 2014 at 3:37 PM, Yury Selivanov wrote: > On 2/19/2014, 11:24 PM, Chris Angelico wrote: >> >> On Thu, Feb 20, 2014 at 2:55 PM, Yury Selivanov >> wrote: >>> >>> On 2/19/2014, 8:18 PM, Chris Angelico wrote: >>>> >>>> Advantages of the arrow include -> >>> >>> I would refrain from introducing a new operator here, >>> especially '->'. >>> >>> This one may later be used for communication between >>> channels/queues or other async stuff (like in golang, >>> although they use '<-'.) >> >> Technically, -> does exist in the language: > > I know, and this is another reason to not to use > '->' for the except expression. I'd call that a medium-weak negative. If there were a current competing proposal, then that would be a strong incentive to leave that character stream alone. Reserving it for a hypothetical future proposal is only a weak objection. It's unlikely that this would warp any other protocol excessively, but it's impossible to warp this one around a nebulous "maybe". It is a small argument in favour of the colon, though, simply because that doesn't have this problem. But a small argument. >> I don't know if it's still available for use as an operator, but I'd >> be extremely cautious about using <- as an operator; it could be >> binary less-than followed by unary minus: > > Sorry, I can't find where I suggested to use '<-' in Python. If -> got snaffled by exception handling, then async comms could just say "oh, let's use the other then". But it would be problematic to. ChrisA From ron3200 at gmail.com Thu Feb 20 08:52:02 2014 From: ron3200 at gmail.com (Ron Adam) Date: Thu, 20 Feb 2014 01:52:02 -0600 Subject: [Python-ideas] except expression In-Reply-To: <53056F54.1030305@canterbury.ac.nz> References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <53056F54.1030305@canterbury.ac.nz> Message-ID: On 02/19/2014 08:58 PM, Greg Ewing wrote: > Another idea: > > things[i] or else None if IndexError > > You can't claim that the exception is in the wrong place > in relation to the 'except' keyword, because there is no > 'except' keyword! By adding a __bool__ attribue to exceptions that always returns False, you can get some of that, but it has the same issues that logic operators have when a normally False item is also valid data. To get around that we would need a differently based logic system where the caught exception(s) are the only False values. Exception logic with False as an exception. e1 and e2 or e3 Translate to this... try: e1 e2 except False: e3 Then with IndexError. (except IndexError: things[i] or None) Cheers, Ron From pconnell at gmail.com Thu Feb 20 09:21:53 2014 From: pconnell at gmail.com (Phil Connell) Date: Thu, 20 Feb 2014 08:21:53 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <53054AC9.5060100@gmail.com> <53055642.3010801@gmail.com> Message-ID: <20140220082153.GB26200@phconnel-ws.cisco.com> On Thu, Feb 20, 2014 at 12:48:33PM +1100, Chris Angelico wrote: > On Thu, Feb 20, 2014 at 12:11 PM, Yury Selivanov > wrote: > > > > On 2/19/2014, 7:56 PM, Chris Angelico wrote: > >> > >> On Thu, Feb 20, 2014 at 11:22 AM, Yury Selivanov > >>> value = 42 if a[10] raises > >> My main objection here is that it reads backwards. > > > > Yes, that's understandable. > > > > Although, as you yourself noted, Python has > > 'expr if expr else expr' already. Yes, it is a bit awkward, > > but lots of people (including me) find it very readable and > > easy to understand. > > Yes, but is that a good thing? The ternary if is justified because it > reads well in English, but even so, it does trap people. I am > listening to proposals that put the operands in the "wrong order", but > the bar is higher - they have to justify that potential confusion. Moreover the analogy with the if expression is completely bogus: value = if raises is the *opposite* way round to value = if else In the if expression, the mainline value comes first, the exceptional value comes last. Yury's proposal has it the other way round, exceptional value first. From rosuav at gmail.com Thu Feb 20 09:36:28 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Feb 2014 19:36:28 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <53056F54.1030305@canterbury.ac.nz> Message-ID: On Thu, Feb 20, 2014 at 6:52 PM, Ron Adam wrote: > > > On 02/19/2014 08:58 PM, Greg Ewing wrote: >> >> Another idea: >> >> things[i] or else None if IndexError >> >> You can't claim that the exception is in the wrong place >> in relation to the 'except' keyword, because there is no >> 'except' keyword! > > > > By adding a __bool__ attribue to exceptions that always returns False, you > can get some of that, but it has the same issues that logic operators have > when a normally False item is also valid data. > > To get around that we would need a differently based logic system where the > caught exception(s) are the only False values. > > Exception logic with False as an exception. > > e1 and e2 or e3 > > Translate to this... > > try: > e1 > e2 > except False: > e3 > > > Then with IndexError. > > (except IndexError: things[i] or None) I've read your post a few times and am still a bit lost. How would the above be evaluated? ChrisA From python at mrabarnett.plus.com Thu Feb 20 14:06:41 2014 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 20 Feb 2014 13:06:41 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> Message-ID: <5305FDE1.3020109@mrabarnett.plus.com> On 2014-02-20 01:18, Chris Angelico wrote: > On Thu, Feb 20, 2014 at 11:15 AM, Ethan Furman wrote: >> result = 1/x except ZeroDivisionError -> NaN >> >> For the record, I could just as easily live with the colon instead of the >> arrow. >> > > Time to open up this branch of the discussion... colon or arrow? > [snip] I definitely prefer the colon. From denis.spir at gmail.com Thu Feb 20 14:10:37 2014 From: denis.spir at gmail.com (spir) Date: Thu, 20 Feb 2014 14:10:37 +0100 Subject: [Python-ideas] code segments Message-ID: <5305FECD.5090509@gmail.com> I often find myself needing to wrap little pieces of code into a whole "object" (programming element or value), sometimes with a name. I feel the same kind of lack as an FP programmer having to code in a language without function "objects"; and indeed the situation is very similar. The cases where i need that in fact follow a schema, let's call it "event". In games, an event is usually something that happens in a particular situation: there is a logic like situation --> happening, an event is in fact the binding of 2 elements. We find something similar with automation (which also is event-driven) sequences: a tree of stages which are reached under a condition and each command a given action: condition --> action. More generally, one may find a kind of cause --> effect schema. An event can be conceptualised as a {cause effect} pair. But what are the pair's elements? And how to encode them in a program? monster'appears : Event{ cause : and monster.is'hidden (= character.pos (33 99)) effect : monster.move 37 101 monster.groar 'fiercefully monster.is'hidden :: false character.scream } The cause is conceptually a logical expression; but it can be arbitrarily complex --thus may require multiple statements if only for readability, also for debugging or other "meta" needs. Otherwise, it is like a proper function, with an output (here a logical value) and no effect. But it does not take any input! instead, any pieces of data it uses are present in the surrounding scope: they _must_ be there, if i may say by logical necessity. The effect is an action, like a procedure that changes the world and computes no product. Similarly, it takes no input but finds its data in the outer scope. Thus, we have 2 kinds of input-less procedures. Otherwise, the notion is somewhat like Ruby blocks I guess. This is also similar to the recently proposed "inline" functions on python-ideas (reason why cc to this list). There may also be a relation to dynamic scoping, since such code segments in fact appear to take their input from the caller's scope: but it is not a _caller_, instead a kind of surrounding block and scope, like in case of inlining. I think _this_ kind of semantics, similar to inlining, is the actual value of dynamic scoping, and what we may miss with static scoping only and ordinary procedures only. We may need a way to have "reified" blocks of code executed in the surrounding scope as if they were explicitely written there, or inlined (or C-like macros). Actual functions/procedures would be overkill, since there is no stack frame (or similar), thus all the typical "prologue" and "epilogue" of procedure calls is unneeded. We just need to jump there, wherever the code lies in memory, and come back. Or, inline ? la C, or like C macros, but this in my view does not match the semantics. What I'm considering is code segment "objects" like procedures but without: 1. input variables, 2. local scope, 3. call/return [0], three related points in fact. Even more generally, one may note ordinary "functions", in mainstream langs and FP alike, combine a number of properties, the ones listed above plus: 0. more basically, general notion of a block of code as programming element 4. in addition, possible output product, for a function-like code segment (maybe I miss some) For the present notion of code segments, all i need is 0, plus 4 in some cases. [2] I'm not sure at all this is a good idea, however --but I like it and would regularly use it if available. d [0] With register save, new stack frame construction, param passing (prologue), etc, and undoing of all that. [1] A static location would not be thread safe; the only other possibility I can think of is dynamic allocation on the heap: overkill. A real alternative is _not_ passing the product back, but noting that at a low level, there are no functions, instead only actions that write somewhere. Eg: x := DIV y z actually means: DIV x (y z) (and in fact at an even lower level x is y, so we get "DIV x z") Similarly, when part of a higher expression we need temp storage location, eg: a := SUM b (DIV c d) requires: DIV temp (c d) SUM a (b temp) Also: IF (EQUAL a b) THEN do-that gives: EQUAL temp (a b) IF temp THEN do-that Function-like code segments would take the place of explicit expressions here. For instance: a := segment-name a := SUM b segment-name if segment-name then do-that Thus, a rewriting rule can get read of their output altogether, I guess. [2] Detail: in a low-level language, it may be easy to do that: a pair of goto; but the instruction for jumping back must accept a _variable_ target, and the the forward one may also be variable at times. In assembly, it's jumping to given addresses, or even just setting the PC register (program counter); there is no issue of variable targets at this level, they're variable adresses in general, meaning pointers. The only issue is scope restriction on such jumps (whether and how to possibly jump into other memory segments). A final point is the output of function-like code segments: we may need a single, temporary stack slot for it [1]. All these issues are solved for ordinary procedures. From elazarg at gmail.com Thu Feb 20 14:11:59 2014 From: elazarg at gmail.com (=?UTF-8?B?15DXnNei15bXqA==?=) Date: Thu, 20 Feb 2014 15:11:59 +0200 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: <20140218230110.GT4519@ando> References: <20140218230110.GT4519@ando> Message-ID: 2014-02-19 1:01 GMT+02:00 Steven D'Aprano : > > On Tue, Feb 18, 2014 at 04:25:28PM -0600, Ryan Gonzalez wrote: > > > In Python 2, you'd do this: > > > > next((x for x in mylist if x)) > > That works fine in Python 3 too. > The problem with this approach, which I personally ran into a couple of days ago, is that raising StopIteration in the case of empty `mylist` is *not* what you want, in general. "first" assumes non-exhausted iterator; raising StopIteration is easily caught in the closest `for` loop, and you end up failing silently. But Errors should never pass silently. This is a case of an "almost working" solution, similar to the and-or "trenary" conditional operator. I think it's horrible. Non-advanced Python programmer may not be able to find such a bug. An implementation of first() should raise some other exception than StopIteration. Elazar -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron3200 at gmail.com Thu Feb 20 14:23:34 2014 From: ron3200 at gmail.com (Ron Adam) Date: Thu, 20 Feb 2014 07:23:34 -0600 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <53056F54.1030305@canterbury.ac.nz> Message-ID: On 02/20/2014 02:36 AM, Chris Angelico wrote: >> >By adding a __bool__ attribue to exceptions that always returns False, you >> >can get some of that, but it has the same issues that logic operators have >> >when a normally False item is also valid data. >> > >> >To get around that we would need a differently based logic system where the >> >caught exception(s) are the only False values. >> > >> >Exception logic with False as an exception. >> > >> > e1 and e2 or e3 >> > >> >Translate to this... >> > >> > try: >> > e1 >> > e2 >> > except False: >> > e3 >> > >> > >> >Then with IndexError. >> > >> > (except IndexError: things[i] or None) > I've read your post a few times and am still a bit lost. How would the > above be evaluated? Yes, I was just showing how the logic would work, not the exact executable code. Here's a more complete examples. It still needs the parser to translate the 'or's and 'and's in the except expression to the correct calls. def _except_or(self, e1, e2): try: _ = e1() except exc: _ = e2() return _ def _except_and(e1, e2): e1() return e2() Then this ... value = (except exc: e1 and e2 or e3) Would translate to: value = _except_or(exc, lambda: _except_and(lambda: e1, lambda: e2), lambda: e3) and this ... value = (except IndexError: things[i] or None) would be: value = _except_or(exc, lambda: things[i], lambda: None) The expression doesn't need to be just too terms, it could be something more complex. (except KeyError: d1[key] or d2[key] or d3[key] or None) Would give the first dictionary lookup that doesn't raise KeyError or None. Because the exception paths and data paths don't overlap, they could be dictionaries containing exception instances and it would still work. Does that help? cheers, Ron From rosuav at gmail.com Thu Feb 20 14:31:39 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 21 Feb 2014 00:31:39 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <53056F54.1030305@canterbury.ac.nz> Message-ID: On Fri, Feb 21, 2014 at 12:23 AM, Ron Adam wrote: > The expression doesn't need to be just too terms, it could be something more > complex. > > (except KeyError: d1[key] or d2[key] or d3[key] or None) > > > Would give the first dictionary lookup that doesn't raise KeyError or None. > > Because the exception paths and data paths don't overlap, they could be > dictionaries containing exception instances and it would still work. Okay. I think I follow. The way to spell that in the current proposal is: d1[key] except KeyError: (d2[key] except KeyError: (d3[key] except KeyError: None)) which is rather more verbose. On the other hand, the syntax you have requires magic around the 'or' keyword. ChrisA From ron3200 at gmail.com Thu Feb 20 14:37:43 2014 From: ron3200 at gmail.com (Ron Adam) Date: Thu, 20 Feb 2014 07:37:43 -0600 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <53056F54.1030305@canterbury.ac.nz> Message-ID: On 02/20/2014 07:23 AM, Ron Adam wrote: > and this ... > > value = (except IndexError: things[i] or None) > > would be: > > value = _except_or(exc, lambda: things[i], lambda: None) Should've been... value = _except_or(IndexError, lambda: things[i], lambda: None) Cheers, Ron From rob.cliffe at btinternet.com Thu Feb 20 14:38:04 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Thu, 20 Feb 2014 13:38:04 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> Message-ID: <5306053C.8010507@btinternet.com> On 20/02/2014 01:18, Chris Angelico wrote: > On Thu, Feb 20, 2014 at 11:15 AM, Ethan Furman wrote: >> result = 1/x except ZeroDivisionError -> NaN >> >> For the record, I could just as easily live with the colon instead of the >> arrow. >> > Time to open up this branch of the discussion... colon or arrow? Colon, please. More compact, reads naturally as it's part of the English language, consistent with "except" statements. Rob Cliffe. From elazarg at gmail.com Thu Feb 20 14:41:45 2014 From: elazarg at gmail.com (=?UTF-8?B?15DXnNei15bXqA==?=) Date: Thu, 20 Feb 2014 15:41:45 +0200 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <53056F54.1030305@canterbury.ac.nz> Message-ID: 2014-02-20 15:31 GMT+02:00 Chris Angelico : > > On Fri, Feb 21, 2014 at 12:23 AM, Ron Adam wrote: > > The expression doesn't need to be just too terms, it could be something more > > complex. > > > > (except KeyError: d1[key] or d2[key] or d3[key] or None) > > > > > > Would give the first dictionary lookup that doesn't raise KeyError or None. > > > > Because the exception paths and data paths don't overlap, they could be > > dictionaries containing exception instances and it would still work. > > Okay. I think I follow. The way to spell that in the current proposal is: > > d1[key] except KeyError: (d2[key] except KeyError: (d3[key] except > KeyError: None)) > > which is rather more verbose. On the other hand, the syntax you have > requires magic around the 'or' keyword. > Perhaps it should be a colon-seperated list: (except KeyError: d1[key] : d2[key] : d3[key] : None) Or maybe a semicolon. (except KeyError: d1[key] ; d2[key] ; d3[key] ; None) --- Elazar -------------- next part -------------- An HTML attachment was scrubbed... URL: From mal at egenix.com Thu Feb 20 14:45:56 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 20 Feb 2014 14:45:56 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> Message-ID: <53060714.9050404@egenix.com> On 20.02.2014 02:18, Chris Angelico wrote: > On Thu, Feb 20, 2014 at 11:15 AM, Ethan Furman wrote: >> result = 1/x except ZeroDivisionError -> NaN >> >> For the record, I could just as easily live with the colon instead of the >> arrow. >> > > Time to open up this branch of the discussion... colon or arrow? > > For the purposes of this debate, I'm comparing these two notations, > and nothing else: > > result = 1/x except ZeroDivisionError -> NaN > result = 1/x except ZeroDivisionError: NaN I'm -1 on both of them. The colon should stay reserved for starting new blocks of statements. The arrow is used for return type annotations, which is a completely different concept than returning values. I also find it disturbing that people are actually considering to use this expression form as a way to do quick&dirty suppression of exceptions. The intended use case should really be limited to providing default values for functions or methods that are expected to return a value. Abusing the fact that procedures in Python return None (simply because we don't have procedures and need to use functions instead) to make use of except expressions would make code less readable. x = data[1] except IndexError return None # is readable f = open('x.txt', 'r') except IOError return None # is probably not a good idea os.remove('/') except IOError return None # is really bad style The purpose of such except expressions should be to work around small corner cases, not to address important exceptional cases in ones applications. Sometimes I wish we had expression objects in Python to wrap expressions without evaluating them - sort of like lambdas without arguments but with a nicer syntax. These could then be used to implement a default() builtin. Here's a version using lambdas as example: def default(expr, value=None, *exceptions): try: expr() except exceptions: return value x = default(lambda: 1/0, None, ZeroDivisionError) print (x) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 20 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From ron3200 at gmail.com Thu Feb 20 15:13:02 2014 From: ron3200 at gmail.com (Ron Adam) Date: Thu, 20 Feb 2014 08:13:02 -0600 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <53056F54.1030305@canterbury.ac.nz> Message-ID: On 02/20/2014 07:31 AM, Chris Angelico wrote: > On Fri, Feb 21, 2014 at 12:23 AM, Ron Adam wrote: >> >The expression doesn't need to be just too terms, it could be something more >> >complex. >> > >> > (except KeyError: d1[key] or d2[key] or d3[key] or None) >> > >> > >> >Would give the first dictionary lookup that doesn't raise KeyError or None. >> > >> >Because the exception paths and data paths don't overlap, they could be >> >dictionaries containing exception instances and it would still work. > Okay. I think I follow. The way to spell that in the current proposal is: > > d1[key] except KeyError: (d2[key] except KeyError: (d3[key] except > KeyError: None)) > > which is rather more verbose. On the other hand, the syntax you have > requires magic around the 'or' keyword. > > ChrisA Correct. And it would need to be able to differentiate the normal use of 'or' and the except 'or'. Probably with braces in some way. Ron From rob.cliffe at btinternet.com Thu Feb 20 15:46:29 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Thu, 20 Feb 2014 14:46:29 +0000 Subject: [Python-ideas] except expression In-Reply-To: <53060714.9050404@egenix.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53060714.9050404@egenix.com> Message-ID: <53061545.8000103@btinternet.com> On 20/02/2014 13:45, M.-A. Lemburg wrote: > On 20.02.2014 02:18, Chris Angelico wrote: >> On Thu, Feb 20, 2014 at 11:15 AM, Ethan Furman wrote: >>> result = 1/x except ZeroDivisionError -> NaN >>> >>> For the record, I could just as easily live with the colon instead of the >>> arrow. >>> >> Time to open up this branch of the discussion... colon or arrow? >> >> For the purposes of this debate, I'm comparing these two notations, >> and nothing else: >> >> result = 1/x except ZeroDivisionError -> NaN >> result = 1/x except ZeroDivisionError: NaN > I'm -1 on both of them. I'm afraid answering your post will mean repeating stuff said earlier in this thread, but here goes: > The colon should stay reserved for starting new blocks of statements. It isn't reserved for that - it is already used for slices, dictionary literals and lambdas. > The arrow is used for return type annotations, which is a completely > different concept than returning values. (I'm not so keen on the arrow myself but) with a limited set of characters, some of them have to perform more than one role, as colons, dots, parentheses and braces (to name but a few) already do. > > I also find it disturbing that people are actually considering > to use this expression form as a way to do quick&dirty suppression > of exceptions. This is a relatively common construction (as a survey of my own small codebase indicates). Quick, yes. How dirty it is depends entirely on context, (and how readable it can be made) and is up to the programmer's judgment. > > The intended use case should really be limited to providing default > values for functions or methods that are expected to return a value. That's your opinion. Why, if it is useful in other ways? [1] > > Abusing the fact that procedures in Python return None (simply > because we don't have procedures and need to use functions instead) > to make use of except expressions would make code less readable. I don't quite see the relevance. None of your examples rely on a function returning None. Could you give an example? > > x = data[1] except IndexError return None # is readable Agreed. > f = open('x.txt', 'r') except IOError return None # is probably not a good idea I personally think this could be spelt more readably (with a colon or "then"). But however it's done, it's a meaningful and useful construct. > os.remove('/') except IOError return None # is really bad style Do you mean because it's not very readable (I agree) or because it's necessarily a bad thing to do (I disagree, particularly if we chose a less drastic example)? > > The purpose of such except expressions should be to work around > small corner cases, not to address important exceptional cases > in ones applications. See [1] above. > > > Sometimes I wish we had expression objects in Python to wrap > expressions without evaluating them - sort of like lambdas > without arguments but with a nicer syntax. These could then > be used to implement a default() builtin. > > Here's a version using lambdas as example: > > def default(expr, value=None, *exceptions): > try: > expr() > except exceptions: > return value > > x = default(lambda: 1/0, None, ZeroDivisionError) > > print (x) > This is interesting. Expression objects might be useful in various ways. But it seems a rather circuitous and obscure way to provide a default value after an exception, and harder to extend: x = default(lambda: d1[key], default(lambda:d2[key], None, KeyError), KeyError) # And I hope I've spelt it right! Rob Cliffe From abarnert at yahoo.com Thu Feb 20 15:49:15 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 20 Feb 2014 06:49:15 -0800 Subject: [Python-ideas] except expression In-Reply-To: <53060714.9050404@egenix.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53060714.9050404@egenix.com> Message-ID: <81F4D48F-F758-4095-8657-0E581D5D77CA@yahoo.com> On Feb 20, 2014, at 5:45, "M.-A. Lemburg" wrote: > On 20.02.2014 02:18, Chris Angelico wrote: >> On Thu, Feb 20, 2014 at 11:15 AM, Ethan Furman wrote: >>> result = 1/x except ZeroDivisionError -> NaN >>> >>> For the record, I could just as easily live with the colon instead of the >>> arrow. >> >> Time to open up this branch of the discussion... colon or arrow? >> >> For the purposes of this debate, I'm comparing these two notations, >> and nothing else: >> >> result = 1/x except ZeroDivisionError -> NaN >> result = 1/x except ZeroDivisionError: NaN > > I'm -1 on both of them. > > The colon should stay reserved for starting new blocks of statements. So you don't like the colon in lambdas, dict displays, or slices? [snip] > I also find it disturbing that people are actually considering > to use this expression form as a way to do quick&dirty suppression > of exceptions. Agreed. Especially after Nick Coghlan demonstrated that we already have a more readable and more concise way to do it without abusing anything: with suppress(IOError): os.remove('/') Who sees that and says, "I like that it's one line, but if only it could be a longer line, with more keywords, and misleadingly imply a useful value"? [snip] > Sometimes I wish we had expression objects in Python to wrap > expressions without evaluating them - sort of like lambdas > without arguments but with a nicer syntax. These could then > be used to implement a default() builtin. There's no "sort of" about it; you want lambda with a nicer syntax. I sympathize with that. Why do you think Haskell gets away with catch being a function when other functional languages don't? Maybe it's this: catch \-> expensive_call \-e> (dangerous_default e) vs. this: catch(function() { expensive_call() }, function(e) { dangerous_default(e) } That being said, I don't know that it would have the same benefits in Python. In a language that already encourages defining functions all over the place like JavaScript or OCaml, the verbose syntax is painful. But Python isn't like that. Haskell also has a one-character compose operator, which I love in Haskell and would love in JS, but I don't miss it in Python. (We don't even have a compose function in the stdlib.) Python lets you use functional style when it's the obvious way to express something, but doesn't force it on you when it isn't. From abarnert at yahoo.com Thu Feb 20 15:55:16 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 20 Feb 2014 06:55:16 -0800 Subject: [Python-ideas] code segments In-Reply-To: <5305FECD.5090509@gmail.com> References: <5305FECD.5090509@gmail.com> Message-ID: <83EF4A3D-A08C-4C4D-9405-B7F173483F2C@yahoo.com> Is there any sense at all in which what you're asking for is different from a simple macro or quasi-quote? Have you installed MacroPy and played with it, as suggested to you on the previous thread? Also, you keep talking about function prologues and epilogues even after it was explained to you that (a) Python functions don't have them, and (b) the cost of a few stack manipulations is a small percentage of the cost of a function call so this would be a silly thing to worry about anyway. Do you really not get how Python is different from C? Sent from a random iPhone On Feb 20, 2014, at 5:10, spir wrote: > I often find myself needing to wrap little pieces of code into a whole "object" (programming element or value), sometimes with a name. I feel the same kind of lack as an FP programmer having to code in a language without function "objects"; and indeed the situation is very similar. > > The cases where i need that in fact follow a schema, let's call it "event". In games, an event is usually something that happens in a particular situation: there is a logic like situation --> happening, an event is in fact the binding of 2 elements. We find something similar with automation (which also is event-driven) sequences: a tree of stages which are reached under a condition and each command a given action: condition --> action. More generally, one may find a kind of cause --> effect schema. An event can be conceptualised as a {cause effect} pair. But what are the pair's elements? And how to encode them in a program? > > monster'appears : Event{ > cause : and monster.is'hidden (= character.pos (33 99)) > effect : > monster.move 37 101 > monster.groar 'fiercefully > monster.is'hidden :: false > character.scream > } > > The cause is conceptually a logical expression; but it can be arbitrarily complex --thus may require multiple statements if only for readability, also for debugging or other "meta" needs. Otherwise, it is like a proper function, with an output (here a logical value) and no effect. But it does not take any input! instead, any pieces of data it uses are present in the surrounding scope: they _must_ be there, if i may say by logical necessity. > The effect is an action, like a procedure that changes the world and computes no product. Similarly, it takes no input but finds its data in the outer scope. > > Thus, we have 2 kinds of input-less procedures. Otherwise, the notion is somewhat like Ruby blocks I guess. This is also similar to the recently proposed "inline" functions on python-ideas (reason why cc to this list). > > There may also be a relation to dynamic scoping, since such code segments in fact appear to take their input from the caller's scope: but it is not a _caller_, instead a kind of surrounding block and scope, like in case of inlining. I think _this_ kind of semantics, similar to inlining, is the actual value of dynamic scoping, and what we may miss with static scoping only and ordinary procedures only. We may need a way to have "reified" blocks of code executed in the surrounding scope as if they were explicitely written there, or inlined (or C-like macros). > > Actual functions/procedures would be overkill, since there is no stack frame (or similar), thus all the typical "prologue" and "epilogue" of procedure calls is unneeded. We just need to jump there, wherever the code lies in memory, and come back. Or, inline ? la C, or like C macros, but this in my view does not match the semantics. > > What I'm considering is code segment "objects" like procedures but without: > 1. input variables, > 2. local scope, > 3. call/return [0], > three related points in fact. > > Even more generally, one may note ordinary "functions", in mainstream langs and FP alike, combine a number of properties, the ones listed above plus: > 0. more basically, general notion of a block of code as programming element > 4. in addition, possible output product, for a function-like code segment > (maybe I miss some) > For the present notion of code segments, all i need is 0, plus 4 in some cases. [2] > > I'm not sure at all this is a good idea, however --but I like it and would regularly use it if available. > > d > > [0] With register save, new stack frame construction, param passing (prologue), etc, and undoing of all that. > > [1] A static location would not be thread safe; the only other possibility I can think of is dynamic allocation on the heap: overkill. > > A real alternative is _not_ passing the product back, but noting that at a low level, there are no functions, instead only actions that write somewhere. Eg: > x := DIV y z > actually means: > DIV x (y z) > (and in fact at an even lower level x is y, so we get "DIV x z") > Similarly, when part of a higher expression we need temp storage location, eg: > a := SUM b (DIV c d) > requires: > DIV temp (c d) > SUM a (b temp) > Also: > IF (EQUAL a b) THEN do-that > gives: > EQUAL temp (a b) > IF temp THEN do-that > Function-like code segments would take the place of explicit expressions here. For instance: > a := segment-name > a := SUM b segment-name > if segment-name then do-that > Thus, a rewriting rule can get read of their output altogether, I guess. > > [2] Detail: in a low-level language, it may be easy to do that: a pair of goto; but the instruction for jumping back must accept a _variable_ target, and the the forward one may also be variable at times. In assembly, it's jumping to given addresses, or even just setting the PC register (program counter); there is no issue of variable targets at this level, they're variable adresses in general, meaning pointers. The only issue is scope restriction on such jumps (whether and how to possibly jump into other memory segments). A final point is the output of function-like code segments: we may need a single, temporary stack slot for it [1]. All these issues are solved for ordinary procedures. > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From mal at egenix.com Thu Feb 20 16:22:27 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 20 Feb 2014 16:22:27 +0100 Subject: [Python-ideas] except expression In-Reply-To: <53061545.8000103@btinternet.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53060714.9050404@egenix.com> <53061545.8000103@btinternet.com> Message-ID: <53061DB3.9020902@egenix.com> On 20.02.2014 15:46, Rob Cliffe wrote: > > On 20/02/2014 13:45, M.-A. Lemburg wrote: >> On 20.02.2014 02:18, Chris Angelico wrote: >>> On Thu, Feb 20, 2014 at 11:15 AM, Ethan Furman wrote: >>>> result = 1/x except ZeroDivisionError -> NaN >>>> >>>> For the record, I could just as easily live with the colon instead of the >>>> arrow. >>>> >>> Time to open up this branch of the discussion... colon or arrow? >>> >>> For the purposes of this debate, I'm comparing these two notations, >>> and nothing else: >>> >>> result = 1/x except ZeroDivisionError -> NaN >>> result = 1/x except ZeroDivisionError: NaN >> I'm -1 on both of them. > I'm afraid answering your post will mean repeating stuff said earlier in this thread, but here goes: >> The colon should stay reserved for starting new blocks of statements. > It isn't reserved for that - it is already used for slices, dictionary literals and lambdas. >> The arrow is used for return type annotations, which is a completely >> different concept than returning values. > (I'm not so keen on the arrow myself but) with a limited set of characters, some of them have to > perform more than one role, as colons, dots, parentheses and braces (to name but a few) already do. >> >> I also find it disturbing that people are actually considering >> to use this expression form as a way to do quick&dirty suppression >> of exceptions. > This is a relatively common construction (as a survey of my own small codebase indicates). Quick, > yes. How dirty it is depends entirely on context, (and how readable it can be made) and is up to > the programmer's judgment. >> >> The intended use case should really be limited to providing default >> values for functions or methods that are expected to return a value. > That's your opinion. Why, if it is useful in other ways? [1] Sure, it's my opinion :-) That's what this mailing list is all about: tossing around ideas. As I've mentioned before, I think people are putting too many features into this expression style and just want to express this concern. The thread started out with the objective to find a solution for the common default argument problem - which is a good thing. The current discussion is getting somewhat out of focus. >> Abusing the fact that procedures in Python return None (simply >> because we don't have procedures and need to use functions instead) >> to make use of except expressions would make code less readable. > I don't quite see the relevance. None of your examples rely on a function returning None. Could > you give an example? I gave an example below and Nick gave another one. >> x = data[1] except IndexError return None # is readable > Agreed. >> f = open('x.txt', 'r') except IOError return None # is probably not a good idea > I personally think this could be spelt more readably (with a colon or "then"). But however it's > done, it's a meaningful and useful construct. I usually consider an IOError to be too serious to hide away in a single line. >> os.remove('/') except IOError return None # is really bad style > Do you mean because it's not very readable (I agree) or because it's necessarily a bad thing to do > (I disagree, particularly if we chose a less drastic example)? The above is the procedure example I was talking about above. I find this an even worse style than the open() error, since there's absolutely no need to use an expression for this - the os.remove() will never return a value, so you don't need an expression. >> The purpose of such except expressions should be to work around >> small corner cases, not to address important exceptional cases >> in ones applications. > See [1] above. >> >> >> Sometimes I wish we had expression objects in Python to wrap >> expressions without evaluating them - sort of like lambdas >> without arguments but with a nicer syntax. These could then >> be used to implement a default() builtin. >> >> Here's a version using lambdas as example: >> >> def default(expr, value=None, *exceptions): >> try: >> expr() >> except exceptions: >> return value >> >> x = default(lambda: 1/0, None, ZeroDivisionError) >> >> print (x) >> > This is interesting. Expression objects might be useful in various ways. But it seems a rather > circuitous and obscure way to provide a default value after an exception, and harder to extend: > x = default(lambda: d1[key], default(lambda:d2[key], None, KeyError), KeyError) # And I hope I've > spelt it right! Well, like I said: the lambda notation doesn't make this look very nice, but the builtin function approach certainly is a lot more flexible than having to introduce a new syntax. BTW: The indexing example that started this thread is so common that I put a special function into mxTools to address it (back in 1998): get(object,index[,default]) Returns object[index], or, if that fails, default. and this works for all indexing compatible objects. You can find more such tools here: http://www.egenix.com/products/python/mxBase/mxTools/ (a lot of those things are no longer needed, since Python has grown built-in support for them over the years) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 20 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From alexander.belopolsky at gmail.com Thu Feb 20 16:23:21 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 20 Feb 2014 10:23:21 -0500 Subject: [Python-ideas] except expression In-Reply-To: <53061545.8000103@btinternet.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53060714.9050404@egenix.com> <53061545.8000103@btinternet.com> Message-ID: On Thu, Feb 20, 2014 at 9:46 AM, Rob Cliffe wrote: > > >> I also find it disturbing that people are actually considering >> to use this expression form as a way to do quick&dirty suppression >> of exceptions. >> > This is a relatively common construction (as a survey of my own small > codebase indicates). Quick, yes. How dirty it is depends entirely on > context, (and how readable it can be made) and is up to the programmer's > judgment. Isn't it what contextlib.suppress() [1] was invented for? Do we need yet another way to express the same? http://docs.python.org/3.4/library/contextlib.html#contextlib.suppress -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Thu Feb 20 16:24:12 2014 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 20 Feb 2014 15:24:12 +0000 Subject: [Python-ideas] except expression In-Reply-To: <53061545.8000103@btinternet.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53060714.9050404@egenix.com> <53061545.8000103@btinternet.com> Message-ID: <53061E1C.8070301@mrabarnett.plus.com> On 2014-02-20 14:46, Rob Cliffe wrote: > > On 20/02/2014 13:45, M.-A. Lemburg wrote: >> On 20.02.2014 02:18, Chris Angelico wrote: >>> On Thu, Feb 20, 2014 at 11:15 AM, Ethan Furman wrote: >>>> result = 1/x except ZeroDivisionError -> NaN >>>> >>>> For the record, I could just as easily live with the colon instead of the >>>> arrow. >>>> >>> Time to open up this branch of the discussion... colon or arrow? >>> >>> For the purposes of this debate, I'm comparing these two notations, >>> and nothing else: >>> >>> result = 1/x except ZeroDivisionError -> NaN >>> result = 1/x except ZeroDivisionError: NaN >> I'm -1 on both of them. > I'm afraid answering your post will mean repeating stuff said earlier in > this thread, but here goes: >> The colon should stay reserved for starting new blocks of statements. > It isn't reserved for that - it is already used for slices, dictionary > literals and lambdas. It should be said that slices and dict literals do "enclose" the colon, the first with in[...] and the second within {...}. @Marc-Andre: Would it be better if the expression containing the except clause were enclosed in (...), e.g. "result = (1/x except ZeroDivisionError: NaN)" rather than "result = 1/x except ZeroDivisionError: NaN"? [snip] From kaiser.yann at gmail.com Thu Feb 20 16:24:31 2014 From: kaiser.yann at gmail.com (Yann Kaiser) Date: Thu, 20 Feb 2014 16:24:31 +0100 Subject: [Python-ideas] except expression In-Reply-To: <81F4D48F-F758-4095-8657-0E581D5D77CA@yahoo.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53060714.9050404@egenix.com> <81F4D48F-F758-4095-8657-0E581D5D77CA@yahoo.com> Message-ID: I too am uncomfortable with re-purposing except ...: from introducing a block to introducing either a block or an expression. "Keyword then colon" is a great indicator for when you should start a new indent, especially with regards to learners. Lambda undermines this enough as it is, and dicts/slices don't have a keyword preceding them. I think we aren't going deep enough into the rabbit hole. Why restrain ourselves to trying to cram try-except-finally into an expression, losing the finally part, when we already have a tool right under our eyes, as Nick Coghlan pointed out, which leverages try-except-finally entirely in a concise fashion, the with statement. If context managers are amended to support a return value(maybe even transforming one?), a with-expression could look like: last = L.pop() with default(None, IndexError) x = expr with produce_default(expensive_op, AnException) contents = f.read() with open('filename') as f d = Decimal(1) / Decimal(7) with Context(prec=5) The syntax is cleaner, including when easing the original problem, and is capable of so much more. I seem to be lacking modesty today, because it seems like this is worth scrapping except-expression for it. (Sorry for the incorrect placement of my reply, I just subscribed.) On 20 February 2014 15:49, Andrew Barnert wrote: > On Feb 20, 2014, at 5:45, "M.-A. Lemburg" wrote: > >> On 20.02.2014 02:18, Chris Angelico wrote: >>> On Thu, Feb 20, 2014 at 11:15 AM, Ethan Furman wrote: >>>> result = 1/x except ZeroDivisionError -> NaN >>>> >>>> For the record, I could just as easily live with the colon instead of the >>>> arrow. >>> >>> Time to open up this branch of the discussion... colon or arrow? >>> >>> For the purposes of this debate, I'm comparing these two notations, >>> and nothing else: >>> >>> result = 1/x except ZeroDivisionError -> NaN >>> result = 1/x except ZeroDivisionError: NaN >> >> I'm -1 on both of them. >> >> The colon should stay reserved for starting new blocks of statements. > > So you don't like the colon in lambdas, dict displays, or slices? > > [snip] > >> I also find it disturbing that people are actually considering >> to use this expression form as a way to do quick&dirty suppression >> of exceptions. > > Agreed. Especially after Nick Coghlan demonstrated that we already have a more readable and more concise way to do it without abusing anything: > > with suppress(IOError): os.remove('/') > > Who sees that and says, "I like that it's one line, but if only it could be a longer line, with more keywords, and misleadingly imply a useful value"? > > [snip] > >> Sometimes I wish we had expression objects in Python to wrap >> expressions without evaluating them - sort of like lambdas >> without arguments but with a nicer syntax. These could then >> be used to implement a default() builtin. > > There's no "sort of" about it; you want lambda with a nicer syntax. > > I sympathize with that. Why do you think Haskell gets away with catch being a function when other functional languages don't? Maybe it's this: > > catch \-> expensive_call \-e> (dangerous_default e) > > vs. this: > > catch(function() { expensive_call() }, function(e) { dangerous_default(e) } > > That being said, I don't know that it would have the same benefits in Python. In a language that already encourages defining functions all over the place like JavaScript or OCaml, the verbose syntax is painful. But Python isn't like that. Haskell also has a one-character compose operator, which I love in Haskell and would love in JS, but I don't miss it in Python. (We don't even have a compose function in the stdlib.) Python lets you use functional style when it's the obvious way to express something, but doesn't force it on you when it isn't. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From ron3200 at gmail.com Thu Feb 20 16:26:43 2014 From: ron3200 at gmail.com (Ron Adam) Date: Thu, 20 Feb 2014 09:26:43 -0600 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <53056F54.1030305@canterbury.ac.nz> Message-ID: On 02/20/2014 07:41 AM, ????? wrote: > > 2014-02-20 15:31 GMT+02:00 Chris Angelico > >: > > > > On Fri, Feb 21, 2014 at 12:23 AM, Ron Adam > > wrote: > > > The expression doesn't need to be just too terms, it could be > something more > > > complex. > > > > > > (except KeyError: d1[key] or d2[key] or d3[key] or None) > > > > > > > > > Would give the first dictionary lookup that doesn't raise KeyError or > None. > > > > > > Because the exception paths and data paths don't overlap, they could be > > > dictionaries containing exception instances and it would still work. > > > > Okay. I think I follow. The way to spell that in the current proposal is: > > > > d1[key] except KeyError: (d2[key] except KeyError: (d3[key] except > > KeyError: None)) > > > > which is rather more verbose. On the other hand, the syntax you have > > requires magic around the 'or' keyword. > > > Perhaps it should be a colon-seperated list: > > (except KeyError: d1[key] : d2[key] : d3[key] : None) > > Or maybe a semicolon. > > (except KeyError: d1[key] ; d2[key] ; d3[key] ; None) The semicolon corresponds to "and statement" in it's normal use and has no return value. How about re-using the binary operators... (except KeyError try d1[key] | d2[key] | d3[key] | None) It's not uncommon for symbols to mean different things in different places. %, +, * There are probably others I'm not thinking of at the moment. I really like this actually and it makes the syntax even more concise. :-) Cheers, Ron From mal at egenix.com Thu Feb 20 16:35:54 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 20 Feb 2014 16:35:54 +0100 Subject: [Python-ideas] except expression In-Reply-To: <81F4D48F-F758-4095-8657-0E581D5D77CA@yahoo.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53060714.9050404@egenix.com> <81F4D48F-F758-4095-8657-0E581D5D77CA@yahoo.com> Message-ID: <530620DA.10201@egenix.com> On 20.02.2014 15:49, Andrew Barnert wrote: > On Feb 20, 2014, at 5:45, "M.-A. Lemburg" wrote: > >> On 20.02.2014 02:18, Chris Angelico wrote: >>> On Thu, Feb 20, 2014 at 11:15 AM, Ethan Furman wrote: >>>> result = 1/x except ZeroDivisionError -> NaN >>>> >>>> For the record, I could just as easily live with the colon instead of the >>>> arrow. >>> >>> Time to open up this branch of the discussion... colon or arrow? >>> >>> For the purposes of this debate, I'm comparing these two notations, >>> and nothing else: >>> >>> result = 1/x except ZeroDivisionError -> NaN >>> result = 1/x except ZeroDivisionError: NaN >> >> I'm -1 on both of them. >> >> The colon should stay reserved for starting new blocks of statements. > > So you don't like the colon in lambdas, dict displays, or slices? Ok, that sentence was oversimplified :-) The proposed syntax looks too much like the regular try-except syntax which uses the colon to signal the start of a new block of statements. > [snip] > >> I also find it disturbing that people are actually considering >> to use this expression form as a way to do quick&dirty suppression >> of exceptions. > > Agreed. Especially after Nick Coghlan demonstrated that we already have a more readable and more concise way to do it without abusing anything: > > with suppress(IOError): os.remove('/') > > Who sees that and says, "I like that it's one line, but if only it could be a longer line, with more keywords, and misleadingly imply a useful value"? > > [snip] > >> Sometimes I wish we had expression objects in Python to wrap >> expressions without evaluating them - sort of like lambdas >> without arguments but with a nicer syntax. These could then >> be used to implement a default() builtin. > > There's no "sort of" about it; you want lambda with a nicer syntax. .. and without the function call overhead :-) > I sympathize with that. Why do you think Haskell gets away with catch being a function when other functional languages don't? Maybe it's this: > > catch \-> expensive_call \-e> (dangerous_default e) > > vs. this: > > catch(function() { expensive_call() }, function(e) { dangerous_default(e) } > > That being said, I don't know that it would have the same benefits in Python. In a language that already encourages defining functions all over the place like JavaScript or OCaml, the verbose syntax is painful. But Python isn't like that. Haskell also has a one-character compose operator, which I love in Haskell and would love in JS, but I don't miss it in Python. (We don't even have a compose function in the stdlib.) Python lets you use functional style when it's the obvious way to express something, but doesn't force it on you when it isn't. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 20 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From rosuav at gmail.com Thu Feb 20 16:53:05 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 21 Feb 2014 02:53:05 +1100 Subject: [Python-ideas] except expression In-Reply-To: <53060714.9050404@egenix.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53060714.9050404@egenix.com> Message-ID: On Fri, Feb 21, 2014 at 12:45 AM, M.-A. Lemburg wrote: > I also find it disturbing that people are actually considering > to use this expression form as a way to do quick&dirty suppression > of exceptions. > > The intended use case should really be limited to providing default > values for functions or methods that are expected to return a value. > > Abusing the fact that procedures in Python return None (simply > because we don't have procedures and need to use functions instead) > to make use of except expressions would make code less readable. I absolutely agree here. Nothing in the proposal is _ever_ advocating that. None of the examples is even showing that. The fact that you happen to be able to is just an effect of simple and straight-forward rules. > x = data[1] except IndexError return None # is readable That's the intent. > f = open('x.txt', 'r') except IOError return None # is probably not a good idea Well, that'd be okay if the rest of your code is like this: if f: data = f.read(...) For instance, you might have a rule that a config file will be read if it exists, but otherwise you query the console. So you might do: foo = f.readline() if f else input("Enter the foobinator: ") > os.remove('/') except IOError return None # is really bad style Yes. This one I don't like, and style guides should condemn it. Or just leave it up to common sense: Don't be stupid. :) ChrisA From oscar.j.benjamin at gmail.com Thu Feb 20 16:54:11 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 20 Feb 2014 15:54:11 +0000 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: <20140218230110.GT4519@ando> Message-ID: On 20 February 2014 13:11, ????? wrote: > 2014-02-19 1:01 GMT+02:00 Steven D'Aprano : >> On Tue, Feb 18, 2014 at 04:25:28PM -0600, Ryan Gonzalez wrote: >> >> > In Python 2, you'd do this: >> > >> > next((x for x in mylist if x)) >> >> That works fine in Python 3 too. >> > > The problem with this approach, which I personally ran into a couple of days > ago, is that raising StopIteration in the case of empty `mylist` is *not* > what you want, in general. I ran into this problem once some time ago and it took a long time to track down the bug. Since then a bare next with no default and no try/except StopIteration sticks out like a sore thumb every time I see it. Bare next() calls should be discouraged. In the situations where they are justified I think that it deserves a code comment at the least: x = next(iterator) # Propagate StopIteration Oscar From rosuav at gmail.com Thu Feb 20 16:58:38 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 21 Feb 2014 02:58:38 +1100 Subject: [Python-ideas] except expression In-Reply-To: <53061DB3.9020902@egenix.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53060714.9050404@egenix.com> <53061545.8000103@btinternet.com> <53061DB3.9020902@egenix.com> Message-ID: On Fri, Feb 21, 2014 at 2:22 AM, M.-A. Lemburg wrote: >>> os.remove('/') except IOError return None # is really bad style >> Do you mean because it's not very readable (I agree) or because it's necessarily a bad thing to do >> (I disagree, particularly if we chose a less drastic example)? > > The above is the procedure example I was talking about above. > > I find this an even worse style than the open() error, since > there's absolutely no need to use an expression for this - the > os.remove() will never return a value, so you don't need > an expression. And that's why I, too, decry this as a bad use of the feature. It's like writing: os.remove(fn) or None which implies (a) that os.remove() might return something other than None, and (b) that the value of the expression is important. Both implications mislead the reader. The insertion of a single line break will do it. Let it stand that: try: os.remove(fn) except OSError: pass and there you are, out of your difficulty at once! (Or, as mentioned, contextlib.suppress.) ChrisA From tjreedy at udel.edu Thu Feb 20 17:05:43 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 20 Feb 2014 11:05:43 -0500 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: <20140218230110.GT4519@ando> Message-ID: On 2/20/2014 8:11 AM, ????? wrote: > > > > 2014-02-19 1:01 GMT+02:00 Steven D'Aprano > >: > > > > On Tue, Feb 18, 2014 at 04:25:28PM -0600, Ryan Gonzalez wrote: > > > > > In Python 2, you'd do this: > > > > > > next((x for x in mylist if x)) > > > > That works fine in Python 3 too. > > > > The problem with this approach, which I personally ran into a couple of > days ago, is that raising StopIteration in the case of empty `mylist` is > *not* what you want, in general. "first" assumes non-exhausted iterator; > raising StopIteration is easily caught in the closest `for` loop, and > you end up failing silently. But > > Errors should never pass silently. > > This is a case of an "almost working" solution, similar to the and-or > "trenary" conditional operator. I think it's horrible. Non-advanced > Python programmer may not be able to find such a bug. > > An implementation of first() should raise some other exception than > StopIteration. #untested __missing = object() def first(iterable, default=__missing): for o in interable: if o: return o else: if default is not __missing: return default else: raise ValueError("iterable has no true value and there is no default") -- Terry Jan Reedy From abarnert at yahoo.com Thu Feb 20 17:05:03 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 20 Feb 2014 08:05:03 -0800 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53060714.9050404@egenix.com> <81F4D48F-F758-4095-8657-0E581D5D77CA@yahoo.com> Message-ID: On Feb 20, 2014, at 7:24, Yann Kaiser wrote: > I too am uncomfortable with re-purposing except ...: from introducing > a block to introducing either a block or an expression. "Keyword then > colon" is a great indicator for when you should start a new indent, > especially with regards to learners. Lambda undermines this enough as > it is, and dicts/slices don't have a keyword preceding them. > > I think we aren't going deep enough into the rabbit hole. Why restrain > ourselves to trying to cram try-except-finally into an expression, > losing the finally part, when we already have a tool right under our > eyes, as Nick Coghlan pointed out, which leverages try-except-finally > entirely in a concise fashion, the with statement. > > If context managers are amended to support a return value(maybe even > transforming one?), a with-expression could look like: This is a really interesting idea. Or maybe two connected but somewhat separate ones. Either way, I think this is worth fleshing out and making a new post out of. I suspect a lot of people will miss it buried in the middle of this gigantic thread. > last = L.pop() with default(None, IndexError) > x = expr with produce_default(expensive_op, AnException) > contents = f.read() with open('filename') as f > d = Decimal(1) / Decimal(7) with Context(prec=5) > > The syntax is cleaner, including when easing the original problem, and > is capable of so much more. I seem to be lacking modesty today, > because it seems like this is worth scrapping except-expression for > it. > > (Sorry for the incorrect placement of my reply, I just subscribed.) > > On 20 February 2014 15:49, Andrew Barnert wrote: >> On Feb 20, 2014, at 5:45, "M.-A. Lemburg" wrote: >> >>> On 20.02.2014 02:18, Chris Angelico wrote: >>>> On Thu, Feb 20, 2014 at 11:15 AM, Ethan Furman wrote: >>>>> result = 1/x except ZeroDivisionError -> NaN >>>>> >>>>> For the record, I could just as easily live with the colon instead of the >>>>> arrow. >>>> >>>> Time to open up this branch of the discussion... colon or arrow? >>>> >>>> For the purposes of this debate, I'm comparing these two notations, >>>> and nothing else: >>>> >>>> result = 1/x except ZeroDivisionError -> NaN >>>> result = 1/x except ZeroDivisionError: NaN >>> >>> I'm -1 on both of them. >>> >>> The colon should stay reserved for starting new blocks of statements. >> >> So you don't like the colon in lambdas, dict displays, or slices? >> >> [snip] >> >>> I also find it disturbing that people are actually considering >>> to use this expression form as a way to do quick&dirty suppression >>> of exceptions. >> >> Agreed. Especially after Nick Coghlan demonstrated that we already have a more readable and more concise way to do it without abusing anything: >> >> with suppress(IOError): os.remove('/') >> >> Who sees that and says, "I like that it's one line, but if only it could be a longer line, with more keywords, and misleadingly imply a useful value"? >> >> [snip] >> >>> Sometimes I wish we had expression objects in Python to wrap >>> expressions without evaluating them - sort of like lambdas >>> without arguments but with a nicer syntax. These could then >>> be used to implement a default() builtin. >> >> There's no "sort of" about it; you want lambda with a nicer syntax. >> >> I sympathize with that. Why do you think Haskell gets away with catch being a function when other functional languages don't? Maybe it's this: >> >> catch \-> expensive_call \-e> (dangerous_default e) >> >> vs. this: >> >> catch(function() { expensive_call() }, function(e) { dangerous_default(e) } >> >> That being said, I don't know that it would have the same benefits in Python. In a language that already encourages defining functions all over the place like JavaScript or OCaml, the verbose syntax is painful. But Python isn't like that. Haskell also has a one-character compose operator, which I love in Haskell and would love in JS, but I don't miss it in Python. (We don't even have a compose function in the stdlib.) Python lets you use functional style when it's the obvious way to express something, but doesn't force it on you when it isn't. >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ From rosuav at gmail.com Thu Feb 20 17:13:19 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 21 Feb 2014 03:13:19 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53060714.9050404@egenix.com> <81F4D48F-F758-4095-8657-0E581D5D77CA@yahoo.com> Message-ID: On Fri, Feb 21, 2014 at 2:24 AM, Yann Kaiser wrote: > I too am uncomfortable with re-purposing except ...: from introducing > a block to introducing either a block or an expression. "Keyword then > colon" is a great indicator for when you should start a new indent, > especially with regards to learners. Lambda undermines this enough as > it is, and dicts/slices don't have a keyword preceding them. "Keyword then colon at the end of a line" is still a good indicator for indenting. If it's not the end of a line, you can't start indenting anyway. Proper layout of an expression-except that goes across multiple lines is still open for debate, but the most common cases will fit onto a single line anyway. The same applies to lambda and dict expressions, the colon won't end the line. Incidentally, quite a few of Python's control structures don't actually have "keyword then colon" anyway: for NAME in EXPR: if EXPR: while EXPR: except EXPR: except EXPR as NAME: So looking for a syntax-highlighted bit followed by a colon will pick up only a subset of cases anyway. > I think we aren't going deep enough into the rabbit hole. Why restrain > ourselves to trying to cram try-except-finally into an expression, > losing the finally part, when we already have a tool right under our > eyes, as Nick Coghlan pointed out, which leverages try-except-finally > entirely in a concise fashion, the with statement. > > If context managers are amended to support a return value(maybe even > transforming one?), a with-expression could look like: > > last = L.pop() with default(None, IndexError) > x = expr with produce_default(expensive_op, AnException) > contents = f.read() with open('filename') as f > d = Decimal(1) / Decimal(7) with Context(prec=5) > > The syntax is cleaner, including when easing the original problem, and > is capable of so much more. I seem to be lacking modesty today, > because it seems like this is worth scrapping except-expression for > it. That smells like a very different proposal :) If you want to champion that one, knock together some examples (contrived or concrete) and start suggesting an expression-with construct, which could then, as you say, subsume this proposal. Be aware, though, that it's going to need some fancy magic to handle lazy evaluation of the default. Compare these constructs: # Basic notation, works fine # Note: Don't assume anything about what x is. # x could literally be any object in the system. try: x = cache[key] except LookupError: # This computation is expensive, might require # network traffic even. x = compute_and_cache(key) # Currently legal but eagerly calculates default # Also depends on cache being an actual dict x = cache.get(key, compute_and_cache(key)) # Proposed except-expression x = cache[key] except LookupError: compute_and_cache(key) # with-statement with exception_yields_default(LookupError, lambda: compute_and_cache(key)): x = cache[key] # If it fails, get the value from the context manager somehow It's plausible, but you'd have to have some system for ensuring lazy calculation. You'll find a vaguely effective helper-function example in the PEP; it's a bit ugly and uses lambdas everywhere, but it would be properly lazy. Here's another example of where lazy evaluation is important: try: opt = config[key] except LookupError: opt = input("Enter "+key+": ") opt = config[key] except LookupError: input("Enter "+key+": ") How would you do this with an expression form of with? Make sure it looks clean and evaluates lazily. ChrisA From tjreedy at udel.edu Thu Feb 20 17:12:50 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 20 Feb 2014 11:12:50 -0500 Subject: [Python-ideas] code segments In-Reply-To: <5305FECD.5090509@gmail.com> References: <5305FECD.5090509@gmail.com> Message-ID: On 2/20/2014 8:10 AM, spir wrote: > I often find myself needing to wrap little pieces of code into a whole > "object" (programming element or value), sometimes with a name. I feel > the same kind of lack as an FP programmer having to code in a language > without function "objects"; and indeed the situation is very similar. > > The cases where i need that in fact follow a schema, let's call it > "event". In games, an event is usually something that happens in a > particular situation: there is a logic like situation --> happening, an > event is in fact the binding of 2 elements. We find something similar > with automation (which also is event-driven) sequences: a tree of stages > which are reached under a condition and each command a given action: > condition --> action. More generally, one may find a kind of cause --> > effect schema. An event can be conceptualised as a {cause effect} pair. > But what are the pair's elements? And how to encode them in a program? > > monster'appears : Event{ > cause : and monster.is'hidden (= character.pos (33 99)) > effect : > monster.move 37 101 > monster.groar 'fiercefully > monster.is'hidden :: false > character.scream > } > > The cause is conceptually a logical expression; but it can be > arbitrarily complex --thus may require multiple statements if only for > readability, also for debugging or other "meta" needs. Otherwise, it is > like a proper function, with an output (here a logical value) and no > effect. But it does not take any input! instead, any pieces of data it > uses are present in the surrounding scope: they _must_ be there, if i > may say by logical necessity. > The effect is an action, like a procedure that changes the world and > computes no product. Similarly, it takes no input but finds its data in > the outer scope. > > Thus, we have 2 kinds of input-less procedures. Otherwise, the notion is > somewhat like Ruby blocks I guess. This is also similar to the recently > proposed "inline" functions on python-ideas (reason why cc to this list). > > There may also be a relation to dynamic scoping, since such code > segments in fact appear to take their input from the caller's scope: but > it is not a _caller_, instead a kind of surrounding block and scope, > like in case of inlining. I think _this_ kind of semantics, similar to > inlining, is the actual value of dynamic scoping, and what we may miss > with static scoping only and ordinary procedures only. We may need a way > to have "reified" blocks of code executed in the surrounding scope as if > they were explicitely written there, or inlined (or C-like macros). This sounds exactly like exec(codestring, globals(), locals()). It is exactly as safe as explicit putting the codestring right there in the code, without quotes. -- Terry Jan Reedy From oscar.j.benjamin at gmail.com Thu Feb 20 17:14:17 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 20 Feb 2014 16:14:17 +0000 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: <20140218230110.GT4519@ando> Message-ID: On 20 February 2014 16:05, Terry Reedy wrote: >> >> An implementation of first() should raise some other exception than >> StopIteration. > > > #untested > __missing = object() > def first(iterable, default=__missing): > for o in interable: > if o: > return o > else: > if default is not __missing: > return default > else: > raise ValueError("iterable has no true value and there is no default") It's easy enough to do if you know that bare next is a bad thing. More-itertools does it the way I would but has a long comment wondering whether it should actually raise StopIteration: https://github.com/erikrose/more-itertools/blob/master/more_itertools/more.py#L37 The thing is just that bare next is not something that's widely recognised as being dangerous. I've seen examples of this kind of bug in samples from many Python aficionados (including at least one from you Terry). Oscar From rosuav at gmail.com Thu Feb 20 17:17:53 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 21 Feb 2014 03:17:53 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <53056F54.1030305@canterbury.ac.nz> Message-ID: On Fri, Feb 21, 2014 at 2:26 AM, Ron Adam wrote: > How about re-using the binary operators... > > (except KeyError try d1[key] | d2[key] | d3[key] | None) > > > It's not uncommon for symbols to mean different things in different places. > > %, +, * > > There are probably others I'm not thinking of at the moment. > > I really like this actually and it makes the syntax even more concise. :-) That works really nicely when you can control the types returned. I've written stuff that abuses magic methods (one of the most fun was producing something that actually stored an indeterminate value in a variable!), but it depends on writing the classes yourself. Can't be used in the general sense. But if you _can_ control it, all you need is to do this: value = d1[key] | d2[key] | d3[key] | None and then define d1[key] to return a special object which, when or'd with something else, tries to look itself up, and if it fails, returns the second object. Actually, here's an even cleaner way. Let's suppose those are dictionary-like objects that you fully control. Just do this: value = (d1 | d2 | d3)[key] and have the "cache | cache" return a helper object that will try one and then the other. Could be beautifully clean... but isn't a replacement for except-expressions. ChrisA From rosuav at gmail.com Thu Feb 20 17:22:18 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 21 Feb 2014 03:22:18 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53060714.9050404@egenix.com> <81F4D48F-F758-4095-8657-0E581D5D77CA@yahoo.com> Message-ID: On Fri, Feb 21, 2014 at 3:05 AM, Andrew Barnert wrote: > I suspect a lot of people will miss it buried in the middle of this gigantic thread. It is, a bit :) Gmail breaks the thread automatically when it hits 100 emails... and it's done that four times so far (in the space of a week, mark you, so that's 60 emails a day average). I'd like to get back to the topic at hand, if we can, so I'll ask this question again. Which do you prefer, out of these? value = expr except Exception -> default value = expr except Exception: default Reasons are preferred over mere preferences, but the latter have value too. And hating both is acceptable; so far, none of the keyword-only options has snowballed in support, but "expr except Exception pass default" and "expr except Exception then default" are probably the front runners in the "no new keywords" and "one new keyword" categories. ChrisA From rosuav at gmail.com Thu Feb 20 17:34:13 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 21 Feb 2014 03:34:13 +1100 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: <20140218230110.GT4519@ando> Message-ID: On Fri, Feb 21, 2014 at 3:14 AM, Oscar Benjamin wrote: > More-itertools does it the way I would but has a long comment > wondering whether it should actually raise StopIteration: > https://github.com/erikrose/more-itertools/blob/master/more_itertools/more.py#L37 Has that been subsumed by next(iter(x),default) ? ChrisA From rob.cliffe at btinternet.com Thu Feb 20 17:42:48 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Thu, 20 Feb 2014 16:42:48 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53060714.9050404@egenix.com> <53061545.8000103@btinternet.com> Message-ID: <53063088.5010709@btinternet.com> On 20/02/2014 15:23, Alexander Belopolsky wrote: > > On Thu, Feb 20, 2014 at 9:46 AM, Rob Cliffe > wrote: > > > > I also find it disturbing that people are actually considering > to use this expression form as a way to do quick&dirty suppression > of exceptions. > > This is a relatively common construction (as a survey of my own > small codebase indicates). Quick, yes. How dirty it is depends > entirely on context, (and how readable it can be made) and is up > to the programmer's judgment. > > > Isn't it what contextlib.suppress() [1] was invented for? Do we need > yet another way to express the same? Well maybe. I have never learned about context managers and know nothing about them. And without some other incentive, rather than make the effort, I will stick with the tried and trusted "try ... except ... finally" workhorse which is built into the language and which I do understand. > > http://docs.python.org/3.4/library/contextlib.html#contextlib.suppress > > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6609 - Release Date: 02/20/14 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From oscar.j.benjamin at gmail.com Thu Feb 20 18:00:21 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 20 Feb 2014 17:00:21 +0000 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: <20140218230110.GT4519@ando> Message-ID: On 20 February 2014 16:34, Chris Angelico wrote: > On Fri, Feb 21, 2014 at 3:14 AM, Oscar Benjamin > wrote: >> More-itertools does it the way I would but has a long comment >> wondering whether it should actually raise StopIteration: >> https://github.com/erikrose/more-itertools/blob/master/more_itertools/more.py#L37 > > Has that been subsumed by next(iter(x),default) ? If the default argument is provided then yes it's not giving much over next/iter. The effect that I more often want is that an empty iterable raises an exception. next() with no default already does that but it's the wrong kind of exception and can't safely be allowed to propagate. In that case the alternative is try: obj = next(iter(iterable)) except StopIteration: raise ValueError (which is exactly what more-itertools first does). Oscar From rymg19 at gmail.com Thu Feb 20 18:10:58 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Thu, 20 Feb 2014 11:10:58 -0600 Subject: [Python-ideas] Raise exception if (not) true Message-ID: In Python, you'll constantly see code like this: ```python if x != y: raise ValueError('x != y!!') ``` or: ```python if not isinstance(x,SomeType): raise TypeError('x is not SomeType!') ``` Assertion help a bit: ```python assert isinstance(x,SomeType), 'x is not SomeType!' ``` Notice I said "a bit". If optimizations are on, they're disabled. In addition, the only type of error thrown is an AssertionError. I propose a `raise_if` function. If the given condition is True, an exception is raised. So, the above examples would be shortened to: ```python raise_if(x!=y, ValueError, 'x != y!!') raise_if(not isinstance(x,SomeType),TypeError, 'x is not SomeType!') ``` There could also be a raise_if_not function that does the opposite: ```python raise_if_not(isinstance(x,SomeType), TypeError, 'x is not SomeType!') ``` Thoughts? -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Thu Feb 20 18:29:20 2014 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 20 Feb 2014 17:29:20 +0000 Subject: [Python-ideas] Raise exception if (not) true In-Reply-To: References: Message-ID: <53063B70.7090608@mrabarnett.plus.com> On 2014-02-20 17:10, Ryan Gonzalez wrote: > In Python, you'll constantly see code like this: > > ```python > if x != y: > raise ValueError('x != y!!') > ``` > > or: > > ```python > if not isinstance(x,SomeType): > raise TypeError('x is not SomeType!') > ``` > > Assertion help a bit: > > ```python > assert isinstance(x,SomeType), 'x is not SomeType!' > ``` > > Notice I said "a bit". If optimizations are on, they're disabled. In > addition, the only type of error thrown is an AssertionError. > > I propose a `raise_if` function. If the given condition is True, an > exception is raised. So, the above examples would be shortened to: > > ```python > > raise_if(x!=y, ValueError, 'x != y!!') > raise_if(not isinstance(x,SomeType),TypeError, 'x is not SomeType!') > > ``` > > There could also be a raise_if_not function that does the opposite: > > ```python > raise_if_not(isinstance(x,SomeType), TypeError, 'x is not SomeType!') > ``` > > Thoughts? > So: raise_if_not(isinstance(x, SomeType), TypeError, 'x is not SomeType!') is equivalent to: if not isinstance(x, SomeType): raise TypeError('x is not SomeType!') ? It doesn't improve the language much, IMHO! :-) From ron3200 at gmail.com Thu Feb 20 18:31:09 2014 From: ron3200 at gmail.com (Ron Adam) Date: Thu, 20 Feb 2014 11:31:09 -0600 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <53056F54.1030305@canterbury.ac.nz> Message-ID: On 02/20/2014 10:17 AM, Chris Angelico wrote: > On Fri, Feb 21, 2014 at 2:26 AM, Ron Adam wrote: >> >How about re-using the binary operators... >> > >> > (except KeyError try d1[key] | d2[key] | d3[key] | None) >> > >> > >> >It's not uncommon for symbols to mean different things in different places. >> > >> > %, +, * >> > >> >There are probably others I'm not thinking of at the moment. >> > >> >I really like this actually and it makes the syntax even more concise.:-) > That works really nicely when you can control the types returned. I've > written stuff that abuses magic methods (one of the most fun was > producing something that actually stored an indeterminate value in a > variable!), but it depends on writing the classes yourself. Can't be > used in the general sense. > > But if you_can_ control it, all yit was so long agoou need is to do this: > > value = d1[key] | d2[key] | d3[key] | None It needs the syntax part to read correctly, otherwise you would think it's a binary or. And you would still have the issue of early evaluation, so it still needs some special help. The syntax helps to make the meaning clear. value = (except IndexError try items[n] | None) I think with highlighting, this would read very nice and the meaning be unambiguous after learning the use of the binary operators in this context. I don't think that would take long as they aren't frequently used symbols. > and then define d1[key] to return a special object which, when or'd > with something else, tries to look itself up, and if it fails, returns > the second object. Actually, here's an even cleaner way. Let's suppose > those are dictionary-like objects that you fully control. Just do > this: > > value = (d1 | d2 | d3)[key] > > and have the "cache | cache" return a helper object that will try one > and then the other. Could be beautifully clean... but isn't a > replacement for except-expressions. I've done something similar with magic methods to implement formula evaluation. With each formula object returning a simplified formula object. (I can't remember the details it was so long ago, but it was a hack in any case.) Just looked at the int and byte types. Are there magic methods for the binary operators? I don't see them, or I'm overlooking them. There are byte codes, BINARY_OR, etc... those wouldn't be used in this case. I don't think this would be hard to implement in byte code. The "&" operation is just executing the two instructions in sequence. The "|", is a standard exception block. The "!" is a raise exception after the expression. The rest is just were to put those in relation to each other, which is handled by the AST. Nothing very special or difficult to implement. Getting the syntax right is the only tricky part, and that may only be because I don't have much experience doing that. Cheers, Ron From ron3200 at gmail.com Thu Feb 20 18:39:40 2014 From: ron3200 at gmail.com (Ron Adam) Date: Thu, 20 Feb 2014 11:39:40 -0600 Subject: [Python-ideas] except expression In-Reply-To: References: <530307C0.7080409@canterbury.ac.nz> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> <20140219032533.GA1458@ando> <53056F54.1030305@canterbury.ac.nz> Message-ID: On 02/20/2014 11:31 AM, Ron Adam wrote: > > I don't think this would be hard to implement in byte code. The "&" > operation is just executing the two instructions in sequence. The "|", is > a standard exception block. The "!" is a raise exception after the > expression. The rest is just were to put those in relation to each other, > which is handled by the AST. Nothing very special or difficult to > implement. Getting the syntax right is the only tricky part, and that may > only be because I don't have much experience doing that. The '!' may be a bit more than that if it's also suppresses an exception... (except exc try ! e1 & e2) # same as '|' ? Maybe it's not needed. Cheers, Ron From pconnell at gmail.com Thu Feb 20 18:41:27 2014 From: pconnell at gmail.com (Phil Connell) Date: Thu, 20 Feb 2014 17:41:27 +0000 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> Message-ID: <20140220174127.GA12116@phconnel-ws.cisco.com> On Thu, Feb 20, 2014 at 12:18:07PM +1100, Chris Angelico wrote: > On Thu, Feb 20, 2014 at 11:15 AM, Ethan Furman wrote: > > result = 1/x except ZeroDivisionError -> NaN > > > > For the record, I could just as easily live with the colon instead of the > > arrow. > > > > Time to open up this branch of the discussion... colon or arrow? > > For the purposes of this debate, I'm comparing these two notations, > and nothing else: > > result = 1/x except ZeroDivisionError -> NaN > result = 1/x except ZeroDivisionError: NaN There's a nice example that just came up in some code I was looking at. Paraphrasing: if some_cond: ... elif not might_be_none.foo: ... else: ... There's a bug in the elif expression, namely 'might_be_none' might be ..None :) The current spelling to fix this is: elif not getattr(might_be_none, "foo", None): ... I'm was a fan of the colon version, but it's *really* ugly in this case: elif not might_be_none.foo except AttributeError: None: ... ': None:' is just awful. In this case, -> looks much better IMO: elif not might_be_none.foo except AttributeError -> None: ... Cheers, Phil From abarnert at yahoo.com Thu Feb 20 19:18:24 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 20 Feb 2014 10:18:24 -0800 Subject: [Python-ideas] Raise exception if (not) true In-Reply-To: <53063B70.7090608@mrabarnett.plus.com> References: <53063B70.7090608@mrabarnett.plus.com> Message-ID: <0C50A937-530E-4491-A0F7-27B1DC354A10@yahoo.com> On Feb 20, 2014, at 9:29, MRAB wrote: > On 2014-02-20 17:10, Ryan Gonzalez wrote: >> In Python, you'll constantly see code like this: >> >> ```python >> if x != y: >> raise ValueError('x != y!!') >> ``` >> >> or: >> >> ```python >> if not isinstance(x,SomeType): >> raise TypeError('x is not SomeType!') >> ``` >> >> Assertion help a bit: >> >> ```python >> assert isinstance(x,SomeType), 'x is not SomeType!' >> ``` >> >> Notice I said "a bit". If optimizations are on, they're disabled. In >> addition, the only type of error thrown is an AssertionError. >> >> I propose a `raise_if` function. If the given condition is True, an >> exception is raised. So, the above examples would be shortened to: >> >> ```python >> >> raise_if(x!=y, ValueError, 'x != y!!') >> raise_if(not isinstance(x,SomeType),TypeError, 'x is not SomeType!') >> >> ``` >> >> There could also be a raise_if_not function that does the opposite: >> >> ```python >> raise_if_not(isinstance(x,SomeType), TypeError, 'x is not SomeType!') >> ``` >> >> Thoughts? > So: > > raise_if_not(isinstance(x, SomeType), TypeError, 'x is not SomeType!') > > is equivalent to: > > if not isinstance(x, SomeType): raise TypeError('x is not SomeType!') > > ? > > It doesn't improve the language much, IMHO! :-) And if you really want it in your project, it's a trivial one-liner, so just write it and use it. From markus at unterwaditzer.net Thu Feb 20 19:21:04 2014 From: markus at unterwaditzer.net (Markus Unterwaditzer) Date: Thu, 20 Feb 2014 19:21:04 +0100 Subject: [Python-ideas] Raise exception if (not) true In-Reply-To: References: Message-ID: On 2014-02-20 18:10, Ryan Gonzalez wrote: > I propose a `raise_if` function. If the given condition is True, an > exception is raised. So, the above examples would be shortened to: > > ```python > > raise_if(x!=y, ValueError, 'x != y!!') > raise_if(not isinstance(x,SomeType),TypeError, 'x is not SomeType!') > > ``` > > There could also be a raise_if_not function that does the opposite: > > ```python > raise_if_not(isinstance(x,SomeType), TypeError, 'x is not SomeType!') > ``` > > Thoughts? I don't see what is wrong with using `if`-statements. They're IMO more readable and obviously they have more usecases. -- Markus From ethan at stoneleaf.us Thu Feb 20 19:23:30 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 20 Feb 2014 10:23:30 -0800 Subject: [Python-ideas] except expression In-Reply-To: <53055642.3010801@gmail.com> References: <52FCAB93.40504@gmail.com> <52FD574E.5020403@canterbury.ac.nz> <52FE0D05.8070900@btinternet.com> <52FE969D.5000002@canterbury.ac.nz> <20140215181136.GH4281@ando> <52FFF59B.4090301@canterbury.ac.nz> <530007EF.6030007@mrabarnett.plus.com> <20140216033502.GA4519@ando> <53054AC9.5060100@gmail.com> <53055642.3010801@gmail.com> Message-ID: <53064822.6050308@stoneleaf.us> On 02/19/2014 05:11 PM, Yury Selivanov wrote: > > Same with my proposal. If follows the already established > syntax and reads nicely. I agree it reads nicely, at least in the case of an exception being raised. But it's entirely unclear what the result should be if no exception is raised. -- ~Ethan~ From mertz at gnosis.cx Thu Feb 20 19:32:44 2014 From: mertz at gnosis.cx (David Mertz) Date: Thu, 20 Feb 2014 10:32:44 -0800 Subject: [Python-ideas] except expression In-Reply-To: <53060714.9050404@egenix.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53060714.9050404@egenix.com> Message-ID: On Thu, Feb 20, 2014 at 5:45 AM, M.-A. Lemburg wrote: > x = data[1] except IndexError return None # is readable > f = open('x.txt', 'r') except IOError return None # is probably not a good > idea > os.remove('/') except IOError return None # is really bad style > I agree with the comments, but I think the middle one doesn't present the usage that really *would* be useful: # Fallback to other file-like object f = open('x.txt') except IOError return io.StringIO('') g = open('y.txt') except IOError return urlopen('http://the.remote/y.txt') # Sometimes we want the content of files txt = open('x.txt').read() except IOError return '' I completely agree that something one calls entirely for its side effect, not its value, is a terrible use for except expressions. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kaiser.yann at gmail.com Thu Feb 20 19:50:09 2014 From: kaiser.yann at gmail.com (Yann Kaiser) Date: Thu, 20 Feb 2014 19:50:09 +0100 Subject: [Python-ideas] with expression Message-ID: As an alternative to the recently-proposed "except expression", I suggested this in its thread. I was recommended to post this separately, as, while it is related, it is different enough from the original idea. The general idea is to extend the context manager protocol so that it can produce a return value in alternative to simply letting or denying an exception from propagating, and introduce an inline form of "with". The motivation behind is, in all honesty, that I find the suggested except-expressions from PEP 463 to be too verbose to inline, and that context managers have a much broader reach. The motivation from the aforementioned PEP(inline evaluation to a "default" expression upon catching an exception) also applies here. It could look a bit like this: contexted_expr with context_manager as c Once more, the rationale from PEP 463 also applies here, in that we can shift "easy defaulting" from being a potential concern for the callee to being always available to the caller through new syntax. Likewise, currently in-lining the use of a context manager can be done, but only through manually encapsulating what would be the context-ed code through lambdas or eval. A full example could be as such: class Default(object): """Context manager that returns a given default value when an exception is caught.""" def __init__(self, value, *exception_types): self.value = value self.exception_types = exception_types or BaseException def __enter__(self): pass def __exit__(self, typ, val, tb): if typ and issubclass(typ, self.exception_types): return True, self.value lst = [1, 2] # found is assigned 2 found = lst[1] with Default(0, IndexError) # not found is assigned 0 not_found = lst[2] with Default(0, IndexError) The different interpretation of __exit__'s return value is probably something that needs to be discussed. In this form, it is potentially backwards-incompatible, so a preferable alternative would be a different special method, perhaps: def __return__(self, typ, val, tb, ret): if typ is None: return False, ret * 3 elif isinstance(typ, IndexError): return True, 10 The alternatively-named special method would take priority over __exit__ and take over its augmented function: If no exception has occurred, typ, val, and tb are given None, as with the regular __exit__ incarnation, but ret, or whatever is the fourth positional parameter, is supplied with what "expr" evaluated to in "expr with cmgr". When an exception has occurred or propagated, typ, val and tb are set to the appropriate values(Since exceptions now keep their traceback as an attribute, maybe only supply the exception object?), and ret is given None. If the return value of this special method is None, the exception or return value is propagated as is. If it is a sequence, the first element is used like the return value of __exit__ would be, and the second element is used to (re-)place the return value of the whole with expression. When multiple context managers are being chained, the return value/exception is forwarded much like it is with twisted's Deferreds. In the use case of providing a default value, if the default value is the product of an expensive operation, an alternative context manager can be designed to compute the value only when needed, for instance: fa = factorials[n] with SetDefault(factorials, n, lambda: math.factorial(n)) Other examples using existing context managers: contents = f.read() with open('file') as f with open('file') as f: contents = f.read() d = Decimal(1) / Decimal(7) with Context(prec=5) with Context(prec=5): d = Decimal(1) / Decimal(7) I think that's all I can think of so far. Sorry as this might be a little too detailed to start with, so I will remind you there is no offense in rethinking any of what I posted here. -yk From elazarg at gmail.com Thu Feb 20 19:54:11 2014 From: elazarg at gmail.com (=?UTF-8?B?15DXnNei15bXqA==?=) Date: Thu, 20 Feb 2014 20:54:11 +0200 Subject: [Python-ideas] with expression In-Reply-To: References: Message-ID: What I'd personally like to see is a combined for-with, something like x = [x.split() for x with in open(thisfile)] and for x with in open(somefile): print(x) A with expression might help too, --- Elazar 2014-02-20 20:50 GMT+02:00 Yann Kaiser : > As an alternative to the recently-proposed "except expression", I > suggested this in its thread. I was recommended to post this > separately, as, while it is related, it is different enough from the > original idea. > > The general idea is to extend the context manager protocol so that it > can produce a return value in alternative to simply letting or denying > an exception from propagating, and introduce an inline form of "with". > > The motivation behind is, in all honesty, that I find the suggested > except-expressions from PEP 463 to be too verbose to inline, and that > context managers have a much broader reach. The motivation from the > aforementioned PEP(inline evaluation to a "default" expression upon > catching an exception) also applies here. > > It could look a bit like this: > > contexted_expr with context_manager as c > > Once more, the rationale from PEP 463 also applies here, in that we > can shift "easy defaulting" from being a potential concern for the > callee to being always available to the caller through new syntax. > Likewise, currently in-lining the use of a context manager can be > done, but only through manually encapsulating what would be the > context-ed code through lambdas or eval. > > A full example could be as such: > > class Default(object): > """Context manager that returns a given default value when an > exception > is caught.""" > > def __init__(self, value, *exception_types): > self.value = value > self.exception_types = exception_types or BaseException > > def __enter__(self): > pass > > def __exit__(self, typ, val, tb): > if typ and issubclass(typ, self.exception_types): > return True, self.value > > > lst = [1, 2] > # found is assigned 2 > found = lst[1] with Default(0, IndexError) > # not found is assigned 0 > not_found = lst[2] with Default(0, IndexError) > > The different interpretation of __exit__'s return value is probably > something that needs to be discussed. In this form, it is potentially > backwards-incompatible, so a preferable alternative would be a > different special method, perhaps: > > def __return__(self, typ, val, tb, ret): > if typ is None: > return False, ret * 3 > elif isinstance(typ, IndexError): > return True, 10 > > The alternatively-named special method would take priority over > __exit__ and take over its augmented function: > If no exception has occurred, typ, val, and tb are given None, as with > the regular __exit__ incarnation, but ret, or whatever is the fourth > positional parameter, is supplied with what "expr" evaluated to in > "expr with cmgr". When an exception has occurred or propagated, typ, > val and tb are set to the appropriate values(Since exceptions now keep > their traceback as an attribute, maybe only supply the exception > object?), and ret is given None. > If the return value of this special method is None, the exception or > return value is propagated as is. If it is a sequence, the first > element is used like the return value of __exit__ would be, and the > second element is used to (re-)place the return value of the whole > with expression. When multiple context managers are being chained, the > return value/exception is forwarded much like it is with twisted's > Deferreds. > > In the use case of providing a default value, if the default value is > the product of an expensive operation, an alternative context manager > can be designed to compute the value only when needed, for instance: > > fa = factorials[n] with SetDefault(factorials, n, lambda: > math.factorial(n)) > > Other examples using existing context managers: > > contents = f.read() with open('file') as f > > with open('file') as f: > contents = f.read() > > > > d = Decimal(1) / Decimal(7) with Context(prec=5) > > with Context(prec=5): > d = Decimal(1) / Decimal(7) > > I think that's all I can think of so far. Sorry as this might be a > little too detailed to start with, so I will remind you there is no > offense in rethinking any of what I posted here. > > -yk > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Thu Feb 20 20:54:51 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 20 Feb 2014 11:54:51 -0800 Subject: [Python-ideas] with expression In-Reply-To: References: Message-ID: <7D1FAEAF-A909-433C-8EF7-F5614DD2AFFA@yahoo.com> On Feb 20, 2014, at 10:50, Yann Kaiser wrote: > As an alternative to the recently-proposed "except expression", I > suggested this in its thread. I was recommended to post this > separately, as, while it is related, it is different enough from the > original idea. > > The general idea is to extend the context manager protocol so that it > can produce a return value in alternative to simply letting or denying > an exception from propagating, and introduce an inline form of "with". > > The motivation behind is, in all honesty, that I find the suggested > except-expressions from PEP 463 to be too verbose to inline, and that > context managers have a much broader reach. The motivation from the > aforementioned PEP(inline evaluation to a "default" expression upon > catching an exception) also applies here. > > It could look a bit like this: > > contexted_expr with context_manager as c > > Once more, the rationale from PEP 463 also applies here, in that we > can shift "easy defaulting" from being a potential concern for the > callee to being always available to the caller through new syntax. > Likewise, currently in-lining the use of a context manager can be > done, but only through manually encapsulating what would be the > context-ed code through lambdas or eval. > > A full example could be as such: > > class Default(object): > """Context manager that returns a given default value when an exception > is caught.""" > > def __init__(self, value, *exception_types): > self.value = value > self.exception_types = exception_types or BaseException > > def __enter__(self): > pass > > def __exit__(self, typ, val, tb): > if typ and issubclass(typ, self.exception_types): > return True, self.value > > > lst = [1, 2] > # found is assigned 2 > found = lst[1] with Default(0, IndexError) > # not found is assigned 0 > not_found = lst[2] with Default(0, IndexError) This case is a nice example. > The different interpretation of __exit__'s return value is probably > something that needs to be discussed. In this form, it is potentially > backwards-incompatible, so a preferable alternative would be a > different special method, perhaps: > > def __return__(self, typ, val, tb, ret): > if typ is None: > return False, ret * 3 > elif isinstance(typ, IndexError): > return True, 10 > > The alternatively-named special method would take priority over > __exit__ and take over its augmented function: > If no exception has occurred, typ, val, and tb are given None, as with > the regular __exit__ incarnation, but ret, or whatever is the fourth > positional parameter, is supplied with what "expr" evaluated to in > "expr with cmgr". When an exception has occurred or propagated, typ, > val and tb are set to the appropriate values(Since exceptions now keep > their traceback as an attribute, maybe only supply the exception > object?), and ret is given None. > If the return value of this special method is None, the exception or > return value is propagated as is. If it is a sequence, the first > element is used like the return value of __exit__ would be, and the > second element is used to (re-)place the return value of the whole > with expression. When multiple context managers are being chained, the > return value/exception is forwarded much like it is with twisted's > Deferreds. > > In the use case of providing a default value, if the default value is > the product of an expensive operation, an alternative context manager > can be designed to compute the value only when needed, for instance: > > fa = factorials[n] with SetDefault(factorials, n, lambda: math.factorial(n)) This one is less useful. That SetDefault has to repeat the factorials and n references, and it makes that fact explicit to the reader, and writes the same thing in two different ways. But, more importantly, a simple "setdefault" function that did the same thing as your SetDefault class without the context management would be easier to write, and both nicer and easier to use: fa = setdefault(factorials, n, lambda: math.factorial(n)) > Other examples using existing context managers: > > contents = f.read() with open('file') as f This probably buys you more in a context where a statement doesn't work just as well, like a function parameter, or a lambda callback: self.read = Button('Read', command=lambda: dostuff(f) with open(path) as f) But either way, it's something I've wanted before. > with open('file') as f: > contents = f.read() > > > > d = Decimal(1) / Decimal(7) with Context(prec=5) > > with Context(prec=5): > d = Decimal(1) / Decimal(7) > > I think that's all I can think of so far. Sorry as this might be a > little too detailed to start with, so I will remind you there is no > offense in rethinking any of what I posted here. > > -yk > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From abarnert at yahoo.com Thu Feb 20 21:02:56 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 20 Feb 2014 12:02:56 -0800 Subject: [Python-ideas] with expression In-Reply-To: References: Message-ID: <37E24F70-9B27-4FF4-BBDF-A48B14800BBC@yahoo.com> On Feb 20, 2014, at 10:54, ????? wrote: > What I'd personally like to see is a combined for-with, something like > > x = [x.split() for x with in open(thisfile)] > > and > > for x with in open(somefile): > print(x) This has two prepositions in a row, attempting to share the same object. That isn't readable in English, or any other human language. Trying to parse it makes my brain hurt. Also, it obviously only works for objects which are iterable, and are also context managers whose context is self--which basically means file-like objects only. This is why we have the "as" clause in with statements (and in the proposed with expression). If you reorganize it to make more sense: x = [x.split() for x in f with open(thisfile) as f] ... then it's exactly the same thing I proposed last year, which was shot down for good reasons. > > A with expression might help too, > > --- > Elazar > > > 2014-02-20 20:50 GMT+02:00 Yann Kaiser : >> As an alternative to the recently-proposed "except expression", I >> suggested this in its thread. I was recommended to post this >> separately, as, while it is related, it is different enough from the >> original idea. >> >> The general idea is to extend the context manager protocol so that it >> can produce a return value in alternative to simply letting or denying >> an exception from propagating, and introduce an inline form of "with". >> >> The motivation behind is, in all honesty, that I find the suggested >> except-expressions from PEP 463 to be too verbose to inline, and that >> context managers have a much broader reach. The motivation from the >> aforementioned PEP(inline evaluation to a "default" expression upon >> catching an exception) also applies here. >> >> It could look a bit like this: >> >> contexted_expr with context_manager as c >> >> Once more, the rationale from PEP 463 also applies here, in that we >> can shift "easy defaulting" from being a potential concern for the >> callee to being always available to the caller through new syntax. >> Likewise, currently in-lining the use of a context manager can be >> done, but only through manually encapsulating what would be the >> context-ed code through lambdas or eval. >> >> A full example could be as such: >> >> class Default(object): >> """Context manager that returns a given default value when an exception >> is caught.""" >> >> def __init__(self, value, *exception_types): >> self.value = value >> self.exception_types = exception_types or BaseException >> >> def __enter__(self): >> pass >> >> def __exit__(self, typ, val, tb): >> if typ and issubclass(typ, self.exception_types): >> return True, self.value >> >> >> lst = [1, 2] >> # found is assigned 2 >> found = lst[1] with Default(0, IndexError) >> # not found is assigned 0 >> not_found = lst[2] with Default(0, IndexError) >> >> The different interpretation of __exit__'s return value is probably >> something that needs to be discussed. In this form, it is potentially >> backwards-incompatible, so a preferable alternative would be a >> different special method, perhaps: >> >> def __return__(self, typ, val, tb, ret): >> if typ is None: >> return False, ret * 3 >> elif isinstance(typ, IndexError): >> return True, 10 >> >> The alternatively-named special method would take priority over >> __exit__ and take over its augmented function: >> If no exception has occurred, typ, val, and tb are given None, as with >> the regular __exit__ incarnation, but ret, or whatever is the fourth >> positional parameter, is supplied with what "expr" evaluated to in >> "expr with cmgr". When an exception has occurred or propagated, typ, >> val and tb are set to the appropriate values(Since exceptions now keep >> their traceback as an attribute, maybe only supply the exception >> object?), and ret is given None. >> If the return value of this special method is None, the exception or >> return value is propagated as is. If it is a sequence, the first >> element is used like the return value of __exit__ would be, and the >> second element is used to (re-)place the return value of the whole >> with expression. When multiple context managers are being chained, the >> return value/exception is forwarded much like it is with twisted's >> Deferreds. >> >> In the use case of providing a default value, if the default value is >> the product of an expensive operation, an alternative context manager >> can be designed to compute the value only when needed, for instance: >> >> fa = factorials[n] with SetDefault(factorials, n, lambda: math.factorial(n)) >> >> Other examples using existing context managers: >> >> contents = f.read() with open('file') as f >> >> with open('file') as f: >> contents = f.read() >> >> >> >> d = Decimal(1) / Decimal(7) with Context(prec=5) >> >> with Context(prec=5): >> d = Decimal(1) / Decimal(7) >> >> I think that's all I can think of so far. Sorry as this might be a >> little too detailed to start with, so I will remind you there is no >> offense in rethinking any of what I posted here. >> >> -yk >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From zuo at chopin.edu.pl Thu Feb 20 21:39:15 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 20 Feb 2014 21:39:15 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <5302FC50.8060109@canterbury.ac.nz> <530307C0.7080409@canterbury.ac.nz> <530356CD.3060506@mrabarnett.plus.com> <20140218232706.GU4519@ando> <5303F03A.9080801@stoneleaf.us> <53040E5E.5020901@btinternet.com> Message-ID: <2e72fa28e6862d863dc309559b7ad308@chopin.edu.pl> 19.02.2014 03:53, Chris Angelico wrote: > On Wed, Feb 19, 2014 at 1:04 PM, Jan Kaliszewski > wrote: >> Me too -- *except* the bare except syntax which, in case >> of except expression, would be a serious nuisance without >> any real advantage. (IMHO) > > Jan, and everyone else who's expressed opinions on the use of bare > except: > > https://raw2.github.com/Rosuav/ExceptExpr/master/pep-0463.txt > > (note that the file name and URL changed when a PEP number was > assigned) > > I've added two sections, one in "Open Issues" and the other in > "Rejected sub-proposals", on this topic. Have I covered the salient > points? Excellent. :) *j From denis.spir at gmail.com Thu Feb 20 21:44:57 2014 From: denis.spir at gmail.com (spir) Date: Thu, 20 Feb 2014 21:44:57 +0100 Subject: [Python-ideas] Raise exception if (not) true In-Reply-To: <53063B70.7090608@mrabarnett.plus.com> References: <53063B70.7090608@mrabarnett.plus.com> Message-ID: <53066949.2020105@gmail.com> On 02/20/2014 06:29 PM, MRAB wrote: > On 2014-02-20 17:10, Ryan Gonzalez wrote: >> In Python, you'll constantly see code like this: >> >> ```python >> if x != y: >> raise ValueError('x != y!!') >> ``` >> >> or: >> >> ```python >> if not isinstance(x,SomeType): >> raise TypeError('x is not SomeType!') >> ``` >> >> Assertion help a bit: >> >> ```python >> assert isinstance(x,SomeType), 'x is not SomeType!' >> ``` >> >> Notice I said "a bit". If optimizations are on, they're disabled. In >> addition, the only type of error thrown is an AssertionError. >> >> I propose a `raise_if` function. If the given condition is True, an >> exception is raised. So, the above examples would be shortened to: >> >> ```python >> >> raise_if(x!=y, ValueError, 'x != y!!') >> raise_if(not isinstance(x,SomeType),TypeError, 'x is not SomeType!') >> >> ``` >> >> There could also be a raise_if_not function that does the opposite: >> >> ```python >> raise_if_not(isinstance(x,SomeType), TypeError, 'x is not SomeType!') >> ``` >> >> Thoughts? >> > So: > > raise_if_not(isinstance(x, SomeType), TypeError, 'x is not SomeType!') > > is equivalent to: > > if not isinstance(x, SomeType): raise TypeError('x is not SomeType!') > > ? And: raise_if(x!=y, ValueError, 'x != y!!') is equivalent to: if x != y: raise ValueError('x != y!!') > It doesn't improve the language much, IMHO! :-) Ditto. But I would like to be able to add an error type to assertions (in addition to the optional message). This is particularly useful for people (like me) who systematically check func inputs (for client debugging comfort), using assert's. It would be mainly ValueError and TypeError. Example: assert x > 0, "x should be positive", ValueError gives: ValueError: x should be positive instead of: AssertionError: x should be positive This is very similar to the proposal above, semantically and practically, except we are here just reusing the builtin 'assert' instruction with an additional parameter. (Seems backward-compatible to me, at first sight, provided the new param comes last. Maybe an issue is the hypothetical mention of an error type, without message.) d From zuo at chopin.edu.pl Thu Feb 20 21:45:18 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 20 Feb 2014 21:45:18 +0100 Subject: [Python-ideas] except expression In-Reply-To: <53061E1C.8070301@mrabarnett.plus.com> References: " <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com>" <5305491D.40301@stoneleaf.us> <53060714.9050404@egenix.com> <53061545.8000103@btinternet.com> <53061E1C.8070301@mrabarnett.plus.com> Message-ID: 20.02.2014 16:24, MRAB wrote: > On 2014-02-20 14:46, Rob Cliffe wrote: >> >> On 20/02/2014 13:45, M.-A. Lemburg wrote: >>> On 20.02.2014 02:18, Chris Angelico wrote: >>>> On Thu, Feb 20, 2014 at 11:15 AM, Ethan Furman >>>> wrote: >>>>> result = 1/x except ZeroDivisionError -> NaN >>>>> >>>>> For the record, I could just as easily live with the colon >>>>> instead of the >>>>> arrow. >>>>> >>>> Time to open up this branch of the discussion... colon or arrow? >>>> >>>> For the purposes of this debate, I'm comparing these two >>>> notations, >>>> and nothing else: >>>> >>>> result = 1/x except ZeroDivisionError -> NaN >>>> result = 1/x except ZeroDivisionError: NaN >>> I'm -1 on both of them. >> I'm afraid answering your post will mean repeating stuff said >> earlier in >> this thread, but here goes: >>> The colon should stay reserved for starting new blocks of >>> statements. >> It isn't reserved for that - it is already used for slices, >> dictionary >> literals and lambdas. > > It should be said that slices and dict literals do "enclose" the > colon, > the first with in[...] and the second within {...}. > > @Marc-Andre: Would it be better if the expression containing the > except > clause were enclosed in (...), e.g. "result = (1/x except > ZeroDivisionError: NaN)" rather than "result = 1/x except > ZeroDivisionError: NaN"? +1 Cheers. *j From rymg19 at gmail.com Thu Feb 20 22:07:24 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Thu, 20 Feb 2014 15:07:24 -0600 Subject: [Python-ideas] Raise exception if (not) true In-Reply-To: <53066949.2020105@gmail.com> References: <53063B70.7090608@mrabarnett.plus.com> <53066949.2020105@gmail.com> Message-ID: Wouldn't that cause problems if the exception specified takes more than one parameter? It's a nice idea, though. I can't believe I didn't think of that. On Thu, Feb 20, 2014 at 2:44 PM, spir wrote: > On 02/20/2014 06:29 PM, MRAB wrote: > >> On 2014-02-20 17:10, Ryan Gonzalez wrote: >> >>> In Python, you'll constantly see code like this: >>> >>> ```python >>> if x != y: >>> raise ValueError('x != y!!') >>> ``` >>> >>> or: >>> >>> ```python >>> if not isinstance(x,SomeType): >>> raise TypeError('x is not SomeType!') >>> ``` >>> >>> Assertion help a bit: >>> >>> ```python >>> assert isinstance(x,SomeType), 'x is not SomeType!' >>> ``` >>> >>> Notice I said "a bit". If optimizations are on, they're disabled. In >>> addition, the only type of error thrown is an AssertionError. >>> >>> I propose a `raise_if` function. If the given condition is True, an >>> exception is raised. So, the above examples would be shortened to: >>> >>> ```python >>> >>> raise_if(x!=y, ValueError, 'x != y!!') >>> raise_if(not isinstance(x,SomeType),TypeError, 'x is not SomeType!') >>> >>> ``` >>> >>> There could also be a raise_if_not function that does the opposite: >>> >>> ```python >>> raise_if_not(isinstance(x,SomeType), TypeError, 'x is not SomeType!') >>> ``` >>> >>> Thoughts? >>> >>> So: >> >> raise_if_not(isinstance(x, SomeType), TypeError, 'x is not SomeType!') >> >> is equivalent to: >> >> if not isinstance(x, SomeType): raise TypeError('x is not SomeType!') >> >> ? >> > > And: > > raise_if(x!=y, ValueError, 'x != y!!') > is equivalent to: > > if x != y: raise ValueError('x != y!!') > > It doesn't improve the language much, IMHO! :-) >> > > Ditto. > > But I would like to be able to add an error type to assertions (in > addition to the optional message). This is particularly useful for people > (like me) who systematically check func inputs (for client debugging > comfort), using assert's. It would be mainly ValueError and TypeError. > > Example: > assert x > 0, "x should be positive", ValueError > gives: > ValueError: x should be positive > instead of: > AssertionError: x should be positive > > This is very similar to the proposal above, semantically and practically, > except we are here just reusing the builtin 'assert' instruction with an > additional parameter. (Seems backward-compatible to me, at first sight, > provided the new param comes last. Maybe an issue is the hypothetical > mention of an error type, without message.) > > > d > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From zuo at chopin.edu.pl Thu Feb 20 22:10:03 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 20 Feb 2014 22:10:03 +0100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> Message-ID: 20.02.2014 02:18, Chris Angelico wrote: > Time to open up this branch of the discussion... colon or arrow? > > For the purposes of this debate, I'm comparing these two notations, > and nothing else: > > result = 1/x except ZeroDivisionError -> NaN + 0.1 from me. > result = 1/x except ZeroDivisionError: NaN + 0.2 from me. But I see a problem with the latter when it is used at the beginning of a block statement, such as if or while -- consider the following example: while 1 % x except ZeroDivisionError: 0: ... These two colons, very close to each other, look weird. IMHO within parens it looks better a bit but still is not ideal: while (1 % x except ZeroDivisionError: 0): ... And with arrow? while 1 % x except ZeroDivisionError -> 0: ... I believe the arrow + parens variant is nicer a bit: while (1 % x except ZeroDivisionError -> 0): ... Also, I still like the "short-paren" variant (as the parens are closer to each other which allow my eyes to match the parens easier): while 1 % x except (ZeroDivisionError: 0): ... ...or maybe: while 1 % x except (ZeroDivisionError -> 0): ... So finally... Hm... Dunno. :) Cheers. *j From markus at unterwaditzer.net Thu Feb 20 22:13:05 2014 From: markus at unterwaditzer.net (Markus Unterwaditzer) Date: Thu, 20 Feb 2014 22:13:05 +0100 Subject: [Python-ideas] Raise exception if (not) true In-Reply-To: <53066949.2020105@gmail.com> References: <53063B70.7090608@mrabarnett.plus.com> <53066949.2020105@gmail.com> Message-ID: <52c145dd-9824-4d3c-b846-aeeaa145cf7b@email.android.com> Painting the bikeshed, i'd rather have the syntax assert x > 0, ValueError("x should be positive") -- Markus On 20 February 2014 21:44:57 CET, spir wrote: >On 02/20/2014 06:29 PM, MRAB wrote: >> On 2014-02-20 17:10, Ryan Gonzalez wrote: >>> In Python, you'll constantly see code like this: >>> >>> ```python >>> if x != y: >>> raise ValueError('x != y!!') >>> ``` >>> >>> or: >>> >>> ```python >>> if not isinstance(x,SomeType): >>> raise TypeError('x is not SomeType!') >>> ``` >>> >>> Assertion help a bit: >>> >>> ```python >>> assert isinstance(x,SomeType), 'x is not SomeType!' >>> ``` >>> >>> Notice I said "a bit". If optimizations are on, they're disabled. In >>> addition, the only type of error thrown is an AssertionError. >>> >>> I propose a `raise_if` function. If the given condition is True, an >>> exception is raised. So, the above examples would be shortened to: >>> >>> ```python >>> >>> raise_if(x!=y, ValueError, 'x != y!!') >>> raise_if(not isinstance(x,SomeType),TypeError, 'x is not SomeType!') >>> >>> ``` >>> >>> There could also be a raise_if_not function that does the opposite: >>> >>> ```python >>> raise_if_not(isinstance(x,SomeType), TypeError, 'x is not >SomeType!') >>> ``` >>> >>> Thoughts? >>> >> So: >> >> raise_if_not(isinstance(x, SomeType), TypeError, 'x is not >SomeType!') >> >> is equivalent to: >> >> if not isinstance(x, SomeType): raise TypeError('x is not SomeType!') >> >> ? > >And: > raise_if(x!=y, ValueError, 'x != y!!') >is equivalent to: > if x != y: raise ValueError('x != y!!') > >> It doesn't improve the language much, IMHO! :-) > >Ditto. > >But I would like to be able to add an error type to assertions (in >addition to >the optional message). This is particularly useful for people (like me) >who >systematically check func inputs (for client debugging comfort), using >assert's. >It would be mainly ValueError and TypeError. > >Example: > assert x > 0, "x should be positive", ValueError >gives: > ValueError: x should be positive >instead of: > AssertionError: x should be positive > >This is very similar to the proposal above, semantically and >practically, except >we are here just reusing the builtin 'assert' instruction with an >additional >parameter. (Seems backward-compatible to me, at first sight, provided >the new >param comes last. Maybe an issue is the hypothetical mention of an >error type, >without message.) > >d >_______________________________________________ >Python-ideas mailing list >Python-ideas at python.org >https://mail.python.org/mailman/listinfo/python-ideas >Code of Conduct: http://python.org/psf/codeofconduct/ From breamoreboy at yahoo.co.uk Thu Feb 20 22:14:39 2014 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 20 Feb 2014 21:14:39 +0000 Subject: [Python-ideas] Raise exception if (not) true In-Reply-To: <53066949.2020105@gmail.com> References: <53063B70.7090608@mrabarnett.plus.com> <53066949.2020105@gmail.com> Message-ID: On 20/02/2014 20:44, spir wrote: > > But I would like to be able to add an error type to assertions (in > addition to the optional message). This is particularly useful for > people (like me) who systematically check func inputs (for client > debugging comfort), using assert's. It would be mainly ValueError and > TypeError. > > Example: > assert x > 0, "x should be positive", ValueError > gives: > ValueError: x should be positive > instead of: > AssertionError: x should be positive > > This is very similar to the proposal above, semantically and > practically, except we are here just reusing the builtin 'assert' > instruction with an additional parameter. (Seems backward-compatible to > me, at first sight, provided the new param comes last. Maybe an issue is > the hypothetical mention of an error type, without message.) > > d I think this is a dreadful idea. There's enough confusion for newbies now as to when to use assert and when to use raise. Having a half way house like this is to me the worst of both worlds, so let's settle for one or the other. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com From steve at pearwood.info Thu Feb 20 22:16:54 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 21 Feb 2014 08:16:54 +1100 Subject: [Python-ideas] Raise exception if (not) true In-Reply-To: References: Message-ID: <20140220211654.GH3684@ando> On Thu, Feb 20, 2014 at 11:10:58AM -0600, Ryan Gonzalez wrote: > In Python, you'll constantly see code like this: > > ```python > if x != y: > raise ValueError('x != y!!') > ``` > > or: > > ```python > if not isinstance(x,SomeType): > raise TypeError('x is not SomeType!') > ``` > > Assertion help a bit: > > ```python > assert isinstance(x,SomeType), 'x is not SomeType!' > ``` > > Notice I said "a bit". If optimizations are on, they're disabled. In > addition, the only type of error thrown is an AssertionError. Which is why asserts do not help at all. If somebody is using an assertion merely to save typing out if cond: raise MoreAppropriateError(message) then their code is poorly-written and probably broken. > I propose a `raise_if` function. If the given condition is True, an > exception is raised. So, the above examples would be shortened to: Put this at the top of your module: def raise_if(condition, exception, message): if condition: raise exception(message) Not every three line function needs to be a built-in. -- Steven From steve at pearwood.info Thu Feb 20 22:28:30 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 21 Feb 2014 08:28:30 +1100 Subject: [Python-ideas] Raise exception if (not) true In-Reply-To: <53066949.2020105@gmail.com> References: <53063B70.7090608@mrabarnett.plus.com> <53066949.2020105@gmail.com> Message-ID: <20140220212830.GI3684@ando> On Thu, Feb 20, 2014 at 09:44:57PM +0100, spir wrote: > But I would like to be able to add an error type to assertions (in addition > to the optional message). This is particularly useful for people (like me) > who systematically check func inputs (for client debugging comfort), using > assert's. Then your code is systematically broken, and badly so. All anyone needs to do to disable your checking is pass -O to the Python interpreter. assert is not a short-cut for lazy programmers to avoid having to write an explicit "if cond: raise SomethingAppropriate(message)". Assertions have specific uses. You should read this post I made last November: https://mail.python.org/pipermail/python-list/2013-November/660401.html > It would be mainly ValueError and TypeError. > > Example: > assert x > 0, "x should be positive", ValueError -1 Apart from the optimization-disables-asserts issue, this breaks the readers expectation for what assertions are used for and what they will do. Here, you are actually using an assert for a critical piece of error checking, but you are disguising it as a checked comment or piece of less critical defensive programming. -- Steven From franck.michea at gmail.com Thu Feb 20 22:31:45 2014 From: franck.michea at gmail.com (Franck Michea) Date: Thu, 20 Feb 2014 22:31:45 +0100 Subject: [Python-ideas] decorator syntax limitation Message-ID: <20140220223146.168ee0c4@homer> Hi there, Today I hit a limitation of decorator syntax, and I was wondering if maybe this limitation could be removed, or if they could allow more. The grammar for decorators is[1]: decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE But `dotted_name` doesn't allow to write things like the comment put in the snippet attached, or this smaller snippet: @M(b).decorator() def wrapped_func(): pass Although it looks possible to me to add this syntax, I was wondering if it had been discussed previously, and if I could see that discussion. All discussions I found were a lot older, arround the time decorators were designed and the syntax was being choosen. I also read PEP306 and PEP318. This is not a blocking issue since you can do with a temporary variable, but I was wondering what were your thoughts on this. Thank you very much, [1]: http://docs.python.org/3.3/reference/grammar.html ===== Snippet ========== class foo_decorator: def __init__(self): pass def __call__(self, func): return func class Foo: def test(self): return foo_decorator() class Bar: def __init__(self): self._foo = Foo() def M(val): return val._foo b = Bar() # SyntaxError: @M(b).test() m = M(b) @m.test() def func(): print('Hello World!') func() ======================== -- Franck Michea - EPITA/LSE/GISTRE 2014 From kaiser.yann at gmail.com Thu Feb 20 22:33:07 2014 From: kaiser.yann at gmail.com (Yann Kaiser) Date: Thu, 20 Feb 2014 22:33:07 +0100 Subject: [Python-ideas] with expression In-Reply-To: References: Message-ID: The first one could be accomplished like: x = [line.split() for line in f] with open(thisfile) as f It would keep f opened while the listcomp is being evaluated. This makes me think however of a likely accident: x = (line.split() for line in f) with open('name') as f next(x) # ValueError: I/O operation on closed file. This does mirror this mistake though: with open('name') as f: return (line.split() for line in f) When what was meant was: with open('name') as f: for line in f: yield line.split() (Which is, unfortunately, a __del__ in disguise, which is frowned upon.) This does raise the question of if we need a "for..in..with" construct for purposes other than setting a new "highest keyword density" record :-) From haoyi.sg at gmail.com Thu Feb 20 22:37:51 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Thu, 20 Feb 2014 13:37:51 -0800 Subject: [Python-ideas] decorator syntax limitation In-Reply-To: <20140220223146.168ee0c4@homer> References: <20140220223146.168ee0c4@homer> Message-ID: I've found that a lot of the decorator syntax limitations can be worked around with an identity decorator: i = lambda x: x @i(M(b).decorator()) def wrapped_func(): pass kind of ugly, but it works On Thu, Feb 20, 2014 at 1:31 PM, Franck Michea wrote: > Hi there, > > Today I hit a limitation of decorator syntax, and I was wondering if > maybe this limitation could be removed, or if they could allow more. > > The grammar for decorators is[1]: > > decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE > > But `dotted_name` doesn't allow to write things like the comment put in > the snippet attached, or this smaller snippet: > > @M(b).decorator() > def wrapped_func(): > pass > > Although it looks possible to me to add this syntax, I was wondering > if it had been discussed previously, and if I could see that > discussion. > > All discussions I found were a lot older, arround the time decorators > were designed and the syntax was being choosen. I also read PEP306 and > PEP318. > > This is not a blocking issue since you can do with a temporary > variable, but I was wondering what were your thoughts on this. > > Thank you very much, > > [1]: http://docs.python.org/3.3/reference/grammar.html > > ===== Snippet ========== > class foo_decorator: > def __init__(self): > pass > > def __call__(self, func): > return func > > class Foo: > def test(self): > return foo_decorator() > > class Bar: > def __init__(self): > self._foo = Foo() > > def M(val): > return val._foo > > b = Bar() > > # SyntaxError: @M(b).test() > m = M(b) > @m.test() > def func(): > print('Hello World!') > > func() > ======================== > > -- > Franck Michea - EPITA/LSE/GISTRE 2014 > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kaiser.yann at gmail.com Thu Feb 20 22:32:04 2014 From: kaiser.yann at gmail.com (Yann Kaiser) Date: Thu, 20 Feb 2014 22:32:04 +0100 Subject: [Python-ideas] with expression In-Reply-To: <37E24F70-9B27-4FF4-BBDF-A48B14800BBC@yahoo.com> References: <37E24F70-9B27-4FF4-BBDF-A48B14800BBC@yahoo.com> Message-ID: On 20 February 2014 20:54, Andrew Barnert wrote: > On Feb 20, 2014, at 10:50, Yann Kaiser wrote: > >> In the use case of providing a default value, if the default value is >> the product of an expensive operation, an alternative context manager >> can be designed to compute the value only when needed, for instance: >> >> fa = factorials[n] with SetDefault(factorials, n, lambda: math.factorial(n)) > > This one is less useful. That SetDefault has to repeat the factorials and n references, and it makes that fact explicit to the reader, and writes the same thing in two different ways. > > But, more importantly, a simple "setdefault" function that did the same thing as your SetDefault class without the context management would be easier to write, and both nicer and easier to use: > > fa = setdefault(factorials, n, lambda: math.factorial(n)) > Agreed. I was asked in the other thread for a way to defer evaluation for the default value, which can simply be done by swapping the context manager. It boils down to: x = func() with ProduceDefault(lambda: expensive_func(), SomeException) I still can't come up with an example that doesn't ask for simply caching the expensive calculation, or where the cheap version's domain can't be checked beforehand. It may be possible that the "ProduceDefault" use case simply does not exist, but ultimately it is something that can be user-implemented at the python programmer's whim. Maybe someone will come up with a tangible use case for that particular aspect, but there are other aspects to an inline "with", as we've both noted. -yk >> Other examples using existing context managers: >> >> contents = f.read() with open('file') as f > > This probably buys you more in a context where a statement doesn't work just as well, like a function parameter, or a lambda callback: > > self.read = Button('Read', command=lambda: dostuff(f) with open(path) as f) > > But either way, it's something I've wanted before. > >> with open('file') as f: >> contents = f.read() >> >> >> >> d = Decimal(1) / Decimal(7) with Context(prec=5) >> >> with Context(prec=5): >> d = Decimal(1) / Decimal(7) >> >> I think that's all I can think of so far. Sorry as this might be a >> little too detailed to start with, so I will remind you there is no >> offense in rethinking any of what I posted here. >> >> -yk >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ From zuo at chopin.edu.pl Thu Feb 20 22:46:03 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 20 Feb 2014 22:46:03 +0100 Subject: [Python-ideas] with expression In-Reply-To: References: Message-ID: <0375b91e164416af34156bb0938a3c0a@chopin.edu.pl> 20.02.2014 22:33, Yann Kaiser wrote: > The first one could be accomplished like: > > x = [line.split() for line in f] with open(thisfile) as f > > It would keep f opened while the listcomp is being evaluated. This > makes me think however of a likely accident: > > x = (line.split() for line in f) with open('name') as f > next(x) # ValueError: I/O operation on closed file. > > This does mirror this mistake though: > > with open('name') as f: > return (line.split() for line in f) > > When what was meant was: > > with open('name') as f: > for line in f: > yield line.split() > > (Which is, unfortunately, a __del__ in disguise, which is frowned > upon.) Why 'unfortunately'? Cheers. *j From steve at pearwood.info Thu Feb 20 22:49:38 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 21 Feb 2014 08:49:38 +1100 Subject: [Python-ideas] decorator syntax limitation In-Reply-To: <20140220223146.168ee0c4@homer> References: <20140220223146.168ee0c4@homer> Message-ID: <20140220214938.GK3684@ando> On Thu, Feb 20, 2014 at 10:31:45PM +0100, Franck Michea wrote: > Hi there, > > Today I hit a limitation of decorator syntax, and I was wondering if > maybe this limitation could be removed, or if they could allow more. I recall that the limitation on decorator syntax was deliberate, because people weren't sure how confusing it would be to allow arbitrary expressions. I think that it might be time to extend the allowed syntax a bit. I think it is perfectly reasonable to allow: > @M(b).decorator() > def wrapped_func(): > pass as syntax. -- Steven From zuo at chopin.edu.pl Thu Feb 20 22:55:28 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 20 Feb 2014 22:55:28 +0100 Subject: [Python-ideas] decorator syntax limitation In-Reply-To: <20140220223146.168ee0c4@homer> References: <20140220223146.168ee0c4@homer> Message-ID: <25f71eb666beca0c7c5a5886540c97e8@chopin.edu.pl> 20.02.2014 22:31, Franck Michea wrote: > Today I hit a limitation of decorator syntax, and I was wondering if > maybe this limitation could be removed, or if they could allow more. > > The grammar for decorators is[1]: > > decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE > > But `dotted_name` doesn't allow to write things like the comment put > in > the snippet attached, or this smaller snippet: > > @M(b).decorator() > def wrapped_func(): > pass > > Although it looks possible to me to add this syntax, I was wondering > if it had been discussed previously, and if I could see that > discussion. > > All discussions I found were a lot older, arround the time decorators > were designed and the syntax was being choosen. I also read PEP306 > and > PEP318. > > This is not a blocking issue since you can do with a temporary > variable, but I was wondering what were your thoughts on this. AFAIR, there was some discussion a year or a few years ago -- unfortunately I don't remember the details (of course it must be in archives of python-ideas (or maybe python-dev?)). As far as I remember, generally, the belief that this restriction is necessary seemed not to be very strong. Cheers. *j From phd at phdru.name Thu Feb 20 22:57:13 2014 From: phd at phdru.name (Oleg Broytman) Date: Thu, 20 Feb 2014 22:57:13 +0100 Subject: [Python-ideas] decorator syntax limitation In-Reply-To: References: <20140220223146.168ee0c4@homer> Message-ID: <20140220215713.GA6105@phdru.name> On Thu, Feb 20, 2014 at 01:37:51PM -0800, Haoyi Li wrote: > I've found that a lot of the decorator syntax limitations can be worked > around with an identity decorator: > > i = lambda x: x > @i(M(b).decorator()) > def wrapped_func(): > pass Wow, what a trick! Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From ncoghlan at gmail.com Thu Feb 20 22:58:30 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 21 Feb 2014 07:58:30 +1000 Subject: [Python-ideas] decorator syntax limitation In-Reply-To: <25f71eb666beca0c7c5a5886540c97e8@chopin.edu.pl> References: <20140220223146.168ee0c4@homer> <25f71eb666beca0c7c5a5886540c97e8@chopin.edu.pl> Message-ID: On 21 February 2014 07:55, Jan Kaliszewski wrote: > > As far as I remember, generally, the belief that this restriction > is necessary seemed not to be very strong. Yup, but the gains aren't that great, either - it's never irritated anyone enough for them to write up the necessary PEP and implement the Grammar change :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From steve at pearwood.info Thu Feb 20 23:00:40 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 21 Feb 2014 09:00:40 +1100 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: <20140218230110.GT4519@ando> Message-ID: <20140220220040.GL3684@ando> On Thu, Feb 20, 2014 at 04:14:17PM +0000, Oscar Benjamin wrote: > On 20 February 2014 16:05, Terry Reedy wrote: > >> > >> An implementation of first() should raise some other exception than > >> StopIteration. [...] > It's easy enough to do if you know that bare next is a bad thing. Say what? Why do you say that next(it) is a bad thing? > More-itertools does it the way I would but has a long comment > wondering whether it should actually raise StopIteration: > https://github.com/erikrose/more-itertools/blob/master/more_itertools/more.py#L37 > > The thing is just that bare next is not something that's widely > recognised as being dangerous. I've seen examples of this kind of bug > in samples from many Python aficionados (including at least one from > you Terry). Why is it dangerous, and what kind of bug? If you're talking about the fact that next(it) can raise StopIteration, I think you are exaggerating the danger. Firstly, quite often you don't mind if it raises StopIteration, since that's what you would have done anyway. Secondly, I don't see why raising StopIteration is so much more dangerous than (say) IndexError or KeyError. Failure to plan for the empty case, or test it, is not a problem unique to next and iterators. I have read text books on Pascal written in the 70s and 80s that emphasise the need to test the linked list and tree walking code with empty data structures. -- Steven From abarnert at yahoo.com Thu Feb 20 23:01:46 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 20 Feb 2014 14:01:46 -0800 Subject: [Python-ideas] with expression In-Reply-To: References: Message-ID: <3605AACA-4260-4003-82B4-DCE6C6FEA389@yahoo.com> On Feb 20, 2014, at 13:33, Yann Kaiser wrote: > The first one could be accomplished like: > > x = [line.split() for line in f] with open(thisfile) as f Yes, if we have a fully general with expression, we don't need a with clause in comprehensions. > It would keep f opened while the listcomp is being evaluated. This > makes me think however of a likely accident: > > x = (line.split() for line in f) with open('name') as f > next(x) # ValueError: I/O operation on closed file. > > This does mirror this mistake though: > > with open('name') as f: > return (line.split() for line in f) > > When what was meant was: > > with open('name') as f: > for line in f: > yield line.split() Or just replace the return with yield from. This is pretty much the paradigm case of when you have to be a generator rather than just return an iterator, which is one of the two major reasons for yield from. > (Which is, unfortunately, a __del__ in disguise, which is frowned upon.) No it's not. The file will be closed as soon as the iterator is fully consumed, even if it's not deleted until much later. Of course it does mean that if you abandon the iterator in the middle you're leaking the generator and therefore the file until deletion. But there are many use cases where that doesn't come up. > This does raise the question of if we need a "for..in..with" construct > for purposes other than setting a new "highest keyword density" record > :-) As was pointed out to me when I suggested such a construct last year, the only cases where this helps are the two cases where you don't need it. Either it's a statement, so you just use statement with, or you can simulate it perfectly with a two-line function (which I believe is available as more_itertools.with_iter if you don't want to write it yourself). These are the two cases you just covered in this email. Anyone who wants to argue for this idea should read the previous thread first. (I would provide a link, but searching from my phone is painful.) From abarnert at yahoo.com Thu Feb 20 23:07:06 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 20 Feb 2014 14:07:06 -0800 Subject: [Python-ideas] with expression In-Reply-To: <0375b91e164416af34156bb0938a3c0a@chopin.edu.pl> References: <0375b91e164416af34156bb0938a3c0a@chopin.edu.pl> Message-ID: On Feb 20, 2014, at 13:46, Jan Kaliszewski wrote: > 20.02.2014 22:33, Yann Kaiser wrote: > >> The first one could be accomplished like: >> >> x = [line.split() for line in f] with open(thisfile) as f >> >> It would keep f opened while the listcomp is being evaluated. This >> makes me think however of a likely accident: >> >> x = (line.split() for line in f) with open('name') as f >> next(x) # ValueError: I/O operation on closed file. >> >> This does mirror this mistake though: >> >> with open('name') as f: >> return (line.split() for line in f) >> >> When what was meant was: >> >> with open('name') as f: >> for line in f: >> yield line.split() >> >> (Which is, unfortunately, a __del__ in disguise, which is frowned upon.) > > Why 'unfortunately'? Relying on __del__ to clean up your files (and other expensive resources) is a bad idea if you only use CPython, and a terrible idea if your code might ever be run on other implementations: it means you don't get deterministic cleanup. And writing something using a with statement--which was explicitly designed to solve that problem--that still relies on __del__ would be dangerously misleading. However, as I pointed out in my other message, this is not really a __del__ in disguise, because if you use this in cases where the returned iterator is fully consumed in normal circumstances (e.g., a chain of iterator transformations that feeds into a final listcomp or writerows or executemany or the like), you _do_ get deterministic cleanup. > > Cheers. > *j > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From elazarg at gmail.com Thu Feb 20 23:38:01 2014 From: elazarg at gmail.com (=?UTF-8?B?15DXnNei15bXqA==?=) Date: Fri, 21 Feb 2014 00:38:01 +0200 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: <20140220220040.GL3684@ando> References: <20140218230110.GT4519@ando> <20140220220040.GL3684@ando> Message-ID: 2014-02-21 0:00 GMT+02:00 Steven D'Aprano : > > On Thu, Feb 20, 2014 at 04:14:17PM +0000, Oscar Benjamin wrote: > > On 20 February 2014 16:05, Terry Reedy wrote: > > >> > > >> An implementation of first() should raise some other exception than > > >> StopIteration. > [...] > > > It's easy enough to do if you know that bare next is a bad thing. > > Say what? Why do you say that next(it) is a bad thing? > > > More-itertools does it the way I would but has a long comment > > wondering whether it should actually raise StopIteration: > > https://github.com/erikrose/more-itertools/blob/master/more_itertools/more.py#L37 > > > > The thing is just that bare next is not something that's widely > > recognised as being dangerous. I've seen examples of this kind of bug > > in samples from many Python aficionados (including at least one from > > you Terry). > > Why is it dangerous, and what kind of bug? > > If you're talking about the fact that next(it) can raise StopIteration, > I think you are exaggerating the danger. Firstly, quite often you don't > mind if it raises StopIteration, since that's what you would have done > anyway. Secondly, I don't see why raising StopIteration is so much more > dangerous than (say) IndexError or KeyError. > I had this bug just the other day. I did not plan for the empty case, since it was obvious that the empty case is a bug, so I relied on the exception being raised in this case. But I did not get the exception since it was caught in a completely unrelated for loop. It took me a while to figure out what's going on, and it would've taken even more for someone else, not familiar with my assumption or with the whole StopIteration thing (which I believe is the common case). An IndexError or a KeyError would have been great in such a case. It is *very* similar to the "and or" story. --- Elazar -------------- next part -------------- An HTML attachment was scrubbed... URL: From rob.cliffe at btinternet.com Thu Feb 20 23:41:50 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Thu, 20 Feb 2014 22:41:50 +0000 Subject: [Python-ideas] except expression In-Reply-To: <20140220174127.GA12116@phconnel-ws.cisco.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <20140220174127.GA12116@phconnel-ws.cisco.com> Message-ID: <530684AE.6010902@btinternet.com> On 20/02/2014 17:41, Phil Connell wrote: > There's a nice example that just came up in some code I was looking at. > Paraphrasing: > > if some_cond: > ... > > elif not might_be_none.foo: > ... > > else: > ... > > There's a bug in the elif expression, namely 'might_be_none' might be ..None :) > > > The current spelling to fix this is: > > elif not getattr(might_be_none, "foo", None): > ... > > > I'm was a fan of the colon version, but it's *really* ugly in this case: > > elif not might_be_none.foo except AttributeError: None: > ... > > Minor point: I would tend to add parentheses, either elif not (might_be_none.foo except AttributeError: None): # preferably or elif (not might_be_none.foo) except AttributeError: None: AFAICS they both work in this example, but I find either more explicit and therefore easier to read. (Mind you, if might_be_none was a single identifier, wouldn't it be clearer to write elif might_be_none is None or not might_be_none.foo: And if it was a more complicated expression, there is the risk that AttributeError is raised evaluating it, a probable error that would be masked by the "except" form. ) From ethan at stoneleaf.us Thu Feb 20 22:49:27 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 20 Feb 2014 13:49:27 -0800 Subject: [Python-ideas] with expression In-Reply-To: References: Message-ID: <53067867.8080009@stoneleaf.us> On 02/20/2014 01:33 PM, Yann Kaiser wrote: > > with open('name') as f: > for line in f: > yield line.split() > > (Which is, unfortunately, a __del__ in disguise, which is frowned upon.) How is that a __del__ in disguise? -- ~Ethan~ From ian at feete.org Thu Feb 20 23:58:39 2014 From: ian at feete.org (Ian Foote) Date: Thu, 20 Feb 2014 22:58:39 +0000 Subject: [Python-ideas] except expression In-Reply-To: <20140220174127.GA12116@phconnel-ws.cisco.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <20140220174127.GA12116@phconnel-ws.cisco.com> Message-ID: <5306889F.6080204@feete.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 20/02/14 17:41, Phil Connell wrote: > > > I'm was a fan of the colon version, but it's *really* ugly in this > case: > > elif not might_be_none.foo except AttributeError: None: ... > > ': None:' is just awful. > I'd spell that elif not (might_be_none.foo except AttributeError: None): ... which is a little less ugly. > > In this case, -> looks much better IMO: > > elif not might_be_none.foo except AttributeError -> None: ... > I'd also spell this version with parentheses. I agree this is still an advantage for `->` over `:`, but I don't think it is as big an advantage as it appeared at first glance. Regards, Ian F -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTBoifAAoJEODsV4MF7PWzyqMH/251JN7U0WANH6jw4j4SK7IE w/PhiZGk7FfKYXuXMuPHj9c9cSRvCD/117yz2NXBSYeskiwcB+BWMAr3nQ6GtN/T wpYNXUfA38+Zg/n6UyP5dmCsW9JCq2gMh/gunlxfFseRgvhJQJCti8Le6pwRJFwc x+tgFtjpKWVOo70X2Aug3P0Cl/RrKmN0XNY6uFXbd9Dsw3WL0M/7M6MU5DwH2+mf gXTCBIm5/NaK7LVx2EdiJtmva+EG2TDVfQ6OI14sUB/iIh6ie+TzAyzbRh36m/rk 1dQDP88EjTaSWoG7GeeQYtrx/ycEv5z7guSmU5fMCaAPQPcD0abaojrts1xREls= =lgjS -----END PGP SIGNATURE----- From zuo at chopin.edu.pl Fri Feb 21 00:10:23 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Fri, 21 Feb 2014 00:10:23 +0100 Subject: [Python-ideas] with expression In-Reply-To: References: <0375b91e164416af34156bb0938a3c0a@chopin.edu.pl> Message-ID: <5dbbe1325154f0ba0e8ac5a6ee2da4a6@chopin.edu.pl> W dniu 20.02.2014 23:07, Andrew Barnert napisa?(a): > On Feb 20, 2014, at 13:46, Jan Kaliszewski wrote: > >> 20.02.2014 22:33, Yann Kaiser wrote: >> >>> The first one could be accomplished like: >>> >>> x = [line.split() for line in f] with open(thisfile) as f >>> >>> It would keep f opened while the listcomp is being evaluated. This >>> makes me think however of a likely accident: >>> >>> x = (line.split() for line in f) with open('name') as f >>> next(x) # ValueError: I/O operation on closed file. >>> >>> This does mirror this mistake though: >>> >>> with open('name') as f: >>> return (line.split() for line in f) >>> >>> When what was meant was: >>> >>> with open('name') as f: >>> for line in f: >>> yield line.split() >>> >>> (Which is, unfortunately, a __del__ in disguise, which is frowned >>> upon.) >> >> Why 'unfortunately'? > > Relying on __del__ to clean up your files (and other expensive > resources) is a bad idea if you only use CPython, and a terrible idea > if your code might ever be run on other implementations: it means you > don't get deterministic cleanup. Yup, I see the point. At least, starting with CPython 3.4, the risk of creating uncollectable reference cycle no longer exists (PEP 442). (That, of course, does not invalidate the concern about deterministic finalization of resources). > And writing something using a with statement--which was explicitly > designed to solve that problem--that still relies on __del__ would be > dangerously misleading. Unless the programmer knows what (s)he does. :) > However, as I pointed out in my other message, this is not really a > __del__ in disguise, because if you use this in cases where the > returned iterator is fully consumed in normal circumstances (e.g., a > chain of iterator transformations that feeds into a final listcomp or > writerows or executemany or the like), you _do_ get deterministic > cleanup. Indeed. Regards. *j From steve at pearwood.info Fri Feb 21 00:51:17 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 21 Feb 2014 10:51:17 +1100 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: <20140218230110.GT4519@ando> <20140220220040.GL3684@ando> Message-ID: <20140220235116.GM3684@ando> On Fri, Feb 21, 2014 at 12:38:01AM +0200, ????? wrote: > 2014-02-21 0:00 GMT+02:00 Steven D'Aprano : > > > > On Thu, Feb 20, 2014 at 04:14:17PM +0000, Oscar Benjamin wrote: > > > On 20 February 2014 16:05, Terry Reedy wrote: > > > >> > > > >> An implementation of first() should raise some other exception than > > > >> StopIteration. > > [...] > > > > > It's easy enough to do if you know that bare next is a bad thing. > > > > Say what? Why do you say that next(it) is a bad thing? > > > > > More-itertools does it the way I would but has a long comment > > > wondering whether it should actually raise StopIteration: > > > > https://github.com/erikrose/more-itertools/blob/master/more_itertools/more.py#L37 > > > > > > The thing is just that bare next is not something that's widely > > > recognised as being dangerous. I've seen examples of this kind of bug > > > in samples from many Python aficionados (including at least one from > > > you Terry). > > > > Why is it dangerous, and what kind of bug? > > > > If you're talking about the fact that next(it) can raise StopIteration, > > I think you are exaggerating the danger. Firstly, quite often you don't > > mind if it raises StopIteration, since that's what you would have done > > anyway. Secondly, I don't see why raising StopIteration is so much more > > dangerous than (say) IndexError or KeyError. > > > I had this bug just the other day. I did not plan for the empty case, since > it was obvious that the empty case is a bug, so I relied on the exception > being raised in this case. But I did not get the exception since it was > caught in a completely unrelated for loop. It took me a while to figure out > what's going on, and it would've taken even more for someone else, not > familiar with my assumption or with the whole StopIteration thing (which I > believe is the common case). An IndexError or a KeyError would have been > great in such a case. IndexError can also be caught by for-loops, under some circumstances. (See the legacy sequence protocol for iteration.) Without knowing what your code does, and how it is written, it's hard to discuss in anything but generalities. But in general, any code that is expected to raise an exception should have a test that it actually does raise that exception. If you're raising an exception that has particular meaning to Python, then you have to be prepared that Python might intercept it before you can see it. Would you be surprised by this? py> class X: ... def method(self): ... # Perform some calculation, which fails. ... raise AttributeError ... def __getattr__(self, name): ... if name == 'spam': return self.method() ... py> x = X() py> getattr(x, 'spam', 23) 23 If the only place you are using x.spam is inside a getattr call, then you will never see the AttributeError. Likewise, if the only place you are using next() is inside a for-loop iterable, then the StopIteration exception will always be captured. (This may even be deliberate on the part of the programmer.) I don't see this as particularly noteworthy. It's one of the things that people probably won't think of until they've actually seen it happen, but programming is full of things like that. Not every tricky bug to find is a sign that the function is "dangerous". -- Steven From oscar.j.benjamin at gmail.com Fri Feb 21 00:51:54 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 20 Feb 2014 23:51:54 +0000 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: <20140218230110.GT4519@ando> <20140220220040.GL3684@ando> Message-ID: On 20 February 2014 22:38, ????? wrote: > > 2014-02-21 0:00 GMT+02:00 Steven D'Aprano : >> >> On Thu, Feb 20, 2014 at 04:14:17PM +0000, Oscar Benjamin wrote: >> > >> > The thing is just that bare next is not something that's widely >> > recognised as being dangerous. I've seen examples of this kind of bug >> > in samples from many Python aficionados (including at least one from >> > you Terry). >> >> Why is it dangerous, and what kind of bug? >> >> If you're talking about the fact that next(it) can raise StopIteration, >> I think you are exaggerating the danger. Firstly, quite often you don't >> mind if it raises StopIteration, since that's what you would have done >> anyway. Secondly, I don't see why raising StopIteration is so much more >> dangerous than (say) IndexError or KeyError. >> > I had this bug just the other day. I did not plan for the empty case, since > it was obvious that the empty case is a bug, so I relied on the exception > being raised in this case. But I did not get the exception since it was > caught in a completely unrelated for loop. It took me a while to figure out > what's going on, and it would've taken even more for someone else, not > familiar with my assumption or with the whole StopIteration thing (which I > believe is the common case). An IndexError or a KeyError would have been > great in such a case. Exactly. The bug I had manifested in a StopIteration that was raised in a semi-deterministic (dependent on slowly varying data) fashion after ~1 hour of processing. Had it resulted in an IndexError I would have seen a traceback and could have fixed the bug within about 5 minutes. But StopIteration doesn't necessarily bubble up in the same way as other exceptions because it can be caught and silently supressed by a for loop (or any consumer of iterators). So instead of a traceback I had truncated output data. It took some time to discover and verify that the output data was truncated. Then it took some time rerunning the script under pdb which was no help since it couldn't latch into the suppressed exception. I assumed that it must be an exception but there were no try/except clauses anywhere in the code. Eventually I found it by peppering the code with: try: ... except Exception as e: import pdb; pdb.set_trace() It took most of a day for me to track that down instead of 5 minutes precisely because StopIteration is not like other exceptions. Admittedly I'd spot a similar bug much quicker now simply because I'm aware of the possibility. A simplified version of the bug is shown below: def itermerge(datasources): for source in datasources: iterator = iter(source) first = next(iterator) for item in iterator: yield first * item data = [ [1, 1, 2, 3], [1, 4, 5, 6], [], # Who put that there? [1, 7, 8, 9], ] for item in itermerge(data): print(item) If you run the above then you get: $ python tmp.py 1 2 3 4 5 6 So the data is silently truncated at the empty iterable. > It is *very* similar to the "and or" story. Exactly. Some of the worst programming idioms are the ones that mostly work but fall apart in special cases. Leaking StopIteration is fine... as long as you don't do it in a generator, or a user-defined iterator, or *any* code called by a generator/iterator. Oscar From oscar.j.benjamin at gmail.com Fri Feb 21 00:54:25 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 20 Feb 2014 23:54:25 +0000 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: <20140220235116.GM3684@ando> References: <20140218230110.GT4519@ando> <20140220220040.GL3684@ando> <20140220235116.GM3684@ando> Message-ID: On 20 February 2014 23:51, Steven D'Aprano wrote: > > IndexError can also be caught by for-loops, under some circumstances. > (See the legacy sequence protocol for iteration.) Only if your code is actually using the legacy sequence protocol: >>> def f(): yield 2; raise IndexError ... >>> for x in f(): ... print(x) ... 2 Traceback (most recent call last): File "", line 1, in File "", line 1, in f IndexError Oscar From steve at pearwood.info Fri Feb 21 00:57:04 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 21 Feb 2014 10:57:04 +1100 Subject: [Python-ideas] except expression In-Reply-To: <1392860066.60595.YahooMailNeo@web181006.mail.ne1.yahoo.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <530554E7.7060102@stoneleaf.us> <1392860066.60595.YahooMailNeo@web181006.mail.ne1.yahoo.com> Message-ID: <20140220235704.GN3684@ando> On Wed, Feb 19, 2014 at 05:34:26PM -0800, Andrew Barnert wrote: > So, while there's no need to actually ban "os.unlink(some_file) except > OSError: None", it doesn't make a good argument for the PEP. What he said. It's actually a bad argument for the PEP, as it's a misuse of the expression form. A slightly less-worse example might be: errors = filter(bool, [os.unlink(file) except OSError as err: err.args[0] for file in files]) except we're deferring the "as err" part :-) -- Steven From steve at pearwood.info Fri Feb 21 01:03:03 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 21 Feb 2014 11:03:03 +1100 Subject: [Python-ideas] except expression In-Reply-To: <5304FE74.1060701@btinternet.com> References: <530356CD.3060506@mrabarnett.plus.com> <5304FE74.1060701@btinternet.com> Message-ID: <20140221000303.GO3684@ando> On Wed, Feb 19, 2014 at 06:56:52PM +0000, Rob Cliffe wrote: > I also found, although I wasn't originally looking for them: > - 2 instances where there was a temptation to abuse the new syntax: > var = expr except Exception1: ShutDownProgram() Any use of the expect *expression* for purely for side-effects, as opposed to calculating a value, is abuse of the syntax and should be discouraged in the strongest possible terms. > - 8 instances where I wanted to do an operation and ignore certain > exceptions, this sort of thing: > try: > dosomething() > except ValueError: > pass This is a statement. It doesn't calculate a result, or if it does, it shoves that result away somewhere else rather than returning it as the result of evaluating an expression. > and found I wanted the syntax "dosomething() except ValueError: > pass". > Under the current proposal this could be achieved by > "dosomething() except ValueError: None", but this does not read as > naturally. This is (1) an abuse of expression syntax, (2) taken literally, impossible, and (3) the fact that it doesn't read the way that you want is an excellent sign that what you want is a bad fit to the expression form. Expressions always evaluate to a value (unless they fail, or the computation never terminates, but let's ignore those cases). While you are free to ignore the calculated value, that's usually a sign that you're doing something wrong. Why calculate a result if you don't care about it? The way to tell if something is an expression or statement is to imagine you were assigning it to a variable. This works: result = x + y - (x*y)/((x**2) + (y**2)) but these doesn't: result = pass result = del variable because they don't return any value. The first case is, as far as the compiler is concerned, exactly equivalent to: result = with the right hand side left blank. So if you are tempted to put something in an except expression that doesn't return a result, or where you don't care about the result: who_cares = do_something() except Exception: pass that's a sign you trying to abuse the syntax. > - 13 instances where I wanted to do something that could _not_ be > written as an expression and ignore errors, broken down as follows: > 5 where I wanted to assign an expression value to a variable > but only if evaluating the expression did not raise an error (if it did > raise an error I wanted to leave the > variable untouched) That's easy with an except expression: value = some_expression() except SpamError: value Assigning value = value may seem a bit funny, but not excessively. > 1 where I wanted to return the value of an expression, but do > nothing (NOT return) if evaluating the expression raised an error. This relies on side-effects, namely, returning from a function, and as such, has no place in an expression. > 7 of the form "del x[y] except IndexError: pass" (so to speak) More side-effects. > So is there a case for extending the syntax to allow > expr except : pass > where the expression is used stand-alone (i.e. its value is thrown > away). "pass" could be semantically equivalent to "None" (re the > interactive interpreter). -1 That makes the syntax a freakish "neither fish nor fowl nor good red herring" hybrid, sometimes an expression, sometimes a statement, and you won't know which until runtime. That's nasty. -- Steven From steve at pearwood.info Fri Feb 21 01:04:12 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 21 Feb 2014 11:04:12 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> Message-ID: <20140221000412.GP3684@ando> On Thu, Feb 20, 2014 at 12:18:07PM +1100, Chris Angelico wrote: > Time to open up this branch of the discussion... colon or arrow? > result = 1/x except ZeroDivisionError -> NaN > result = 1/x except ZeroDivisionError: NaN +1 for the colon. +0.8 for the arrow. -- Steven From steve at pearwood.info Fri Feb 21 01:10:05 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 21 Feb 2014 11:10:05 +1100 Subject: [Python-ideas] except expression In-Reply-To: <53057CCE.2050505@gmail.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53057CCE.2050505@gmail.com> Message-ID: <20140221001005.GQ3684@ando> On Wed, Feb 19, 2014 at 10:55:58PM -0500, Yury Selivanov wrote: > > On 2/19/2014, 8:18 PM, Chris Angelico wrote: > >Advantages of the arrow include -> > I would refrain from introducing a new operator here, > especially '->'. It's not new. It's used in annotations: py> def func(a, b) -> "Return result": ... return "something" ... py> func.__annotations__ {'return': 'Return result'} -- Steven From steve at pearwood.info Fri Feb 21 01:25:58 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 21 Feb 2014 11:25:58 +1100 Subject: [Python-ideas] except expression In-Reply-To: <53060714.9050404@egenix.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53060714.9050404@egenix.com> Message-ID: <20140221002558.GR3684@ando> On Thu, Feb 20, 2014 at 02:45:56PM +0100, M.-A. Lemburg wrote: > The colon should stay reserved for starting new blocks of statements. The colon isn't reserved for starting new blocks of statements, so it can't "stay" that way. First we would have to get rid of dict displays, dict comprehensions, slices and lambda. That will require a PEP, and a period of depreciation (preferably lasting until Python 5.0, or maybe Python 9.0) before the existing syntax could be removed. > The arrow is used for return type annotations, which is a completely > different concept than returning values. Well sure, but then + is used for numeric addition and string or list concatenation, which are similarly different. It isn't a big stretch to go from "return this value" as code to "return this value" as an annotation, and an arrow operator -> is an obvious symbol for "map" or "return". I am surely not the only person who uses notation like: func -> 42 in sketches to indicate that func returns 42. > I also find it disturbing that people are actually considering > to use this expression form as a way to do quick&dirty suppression > of exceptions. I'm not surprised in the least. There are bad programmers everywhere, and I'm sure some of them use Python. But "quick&dirty suppression" is not the only use-case of this, and some of us are planning to use this syntax as a way to do appropriately thought-out catching of expressions. E.g. in the use-case that Raymond originally mentioned, to stop the proliferation of "default" arguments added to everything and anything that might raise. > The intended use case should really be limited to providing default > values for functions or methods that are expected to return a value. But not expressions? If you actually mean that, that's a strange limitation. Why should this be allowed: one_over(x) except ZeroDivisionError: INF but not this? 1/x except ZeroDivisionError: INF > Abusing the fact that procedures in Python return None (simply > because we don't have procedures and need to use functions instead) > to make use of except expressions would make code less readable. That I can agree with! > Sometimes I wish we had expression objects in Python to wrap > expressions without evaluating them - sort of like lambdas > without arguments but with a nicer syntax. These could then > be used to implement a default() builtin. Yes to this! But that requires a PEP of its own. -- Steven From steve at pearwood.info Fri Feb 21 01:46:47 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 21 Feb 2014 11:46:47 +1100 Subject: [Python-ideas] except expression In-Reply-To: References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53060714.9050404@egenix.com> <81F4D48F-F758-4095-8657-0E581D5D77CA@yahoo.com> Message-ID: <20140221004647.GS3684@ando> On Fri, Feb 21, 2014 at 03:13:19AM +1100, Chris Angelico wrote: > Proper layout of an expression-except that goes > across multiple lines is still open for debate, but the most common > cases will fit onto a single line anyway. Put round brackets around the entire expression, then split where appropriate. # Yes. value = (some_long_expression except LongExceptionName: 42) # Or this value = (some_long_expression except LongExceptionName: 42) # But not this: value = (some_long expression except LongExceptionName: 42) -- Steven From steve at pearwood.info Fri Feb 21 02:01:01 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 21 Feb 2014 12:01:01 +1100 Subject: [Python-ideas] except expression In-Reply-To: <20140220174127.GA12116@phconnel-ws.cisco.com> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <20140220174127.GA12116@phconnel-ws.cisco.com> Message-ID: <20140221010101.GT3684@ando> On Thu, Feb 20, 2014 at 05:41:27PM +0000, Phil Connell wrote: > There's a nice example that just came up in some code I was looking at. > Paraphrasing: [...] > The current spelling to fix this is: > > elif not getattr(might_be_none, "foo", None): > ... And that won't go away. You can still do that. > I'm was a fan of the colon version, but it's *really* ugly in this case: > > elif not might_be_none.foo except AttributeError: None: > ... > > ': None:' is just awful. I disagree that it's ugly. But even if you do, just don't use it in this case! Or wrap it in parens. There is no syntax ever created that cannot find itself used in an ugly way: func(("(",")")) Sometimes you just have to decide that a particular piece of code is ugly, and either don't use it, or live with the ugliness. If we're going to reject syntax because there are scenarios where it looks a bit ick, we're going to reject *everything*. -- Steven From denis.spir at gmail.com Fri Feb 21 02:47:17 2014 From: denis.spir at gmail.com (spir) Date: Fri, 21 Feb 2014 02:47:17 +0100 Subject: [Python-ideas] Raise exception if (not) true In-Reply-To: <52c145dd-9824-4d3c-b846-aeeaa145cf7b@email.android.com> References: <53063B70.7090608@mrabarnett.plus.com> <53066949.2020105@gmail.com> <52c145dd-9824-4d3c-b846-aeeaa145cf7b@email.android.com> Message-ID: <5306B025.1060506@gmail.com> On 02/20/2014 10:13 PM, Markus Unterwaditzer wrote: > Painting the bikeshed, i'd rather have the syntax > > assert x > 0, ValueError("x should be positive") Looks good as well; (but maybe slightly more job to implement, due to change of syntax?) d From carl.input at gmail.com Fri Feb 21 03:13:13 2014 From: carl.input at gmail.com (Carl Smith) Date: Fri, 21 Feb 2014 02:13:13 +0000 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword Message-ID: Sometimes you need to build a list in a loop, but break from the loop if some condition is met, keeping the list up to that point. This is so common it doesn't really need an example. Trying to shoehorn the break keyword in to the generator expression syntax doesn't look pretty, unless you add `and break if expr` to the end, but that has its own issues. Besides, overloading `while` is much cuter... ls = [ expr for name in iterable while expr ] ls = [ expr for name in iterable if expr while expr ] ls = [ expr for name in iterable if expr else expr while expr ] With regular generator expressions, it gives you a kind of base case, which may assist with showing off. -------------- next part -------------- An HTML attachment was scrubbed... URL: From denis.spir at gmail.com Fri Feb 21 03:23:21 2014 From: denis.spir at gmail.com (spir) Date: Fri, 21 Feb 2014 03:23:21 +0100 Subject: [Python-ideas] Raise exception if (not) true In-Reply-To: <20140220212830.GI3684@ando> References: <53063B70.7090608@mrabarnett.plus.com> <53066949.2020105@gmail.com> <20140220212830.GI3684@ando> Message-ID: <5306B899.4000905@gmail.com> On 02/20/2014 10:28 PM, Steven D'Aprano wrote: > On Thu, Feb 20, 2014 at 09:44:57PM +0100, spir wrote: > >> >But I would like to be able to add an error type to assertions (in addition >> >to the optional message). This is particularly useful for people (like me) >> >who systematically check func inputs (for client debugging comfort), using >> >assert's. > Then your code is systematically broken, and badly so. Why do you speak _that_ negatively? (or should I say: _that_ violently?) And this, maybe not _that_ logically? Do you own the (copy)rights on proper usage of assert's? > All anyone needs > to do to disable your checking is pass -O to the Python interpreter. > > assert is not a short-cut for lazy programmers to avoid having to write > an explicit "if cond: raise SomethingAppropriate(message)". Assertions > have specific uses. You should read this post I made last November: > > https://mail.python.org/pipermail/python-list/2013-November/660401.html I think you are wrong, at least in this very case. Assert's for me are a debugging tool (or maybe more generally a tool for improving reliability). Such checks help finding errors and correcting them (thank to hopefully clear error messages), with a higher chance these corrections happen before "too late", meaning before users pay for our bugs. What else is the proper usage of assertions? If not checked on function input, either with assert's or an if-raise combination, execution will break anyway, just later (later in time, and slightly further in code), because some input variable's value or type is wrong. Assert's placed that way are not strictly necessary (it's like duck typing: you don't need to check), instead they're a helpful tool for all users of your "service". def average (numbers): n = len(numbers) assert n != 0, "Cannot compute average of 'zero number'.", ValueError return sum(numbers) / n On the proposal: having a correct error type instead of "AssertionError" just helps a bit making the error report clearer for developpers. I note in your post that, according to you, "assertions should be used for" (among other cases): "* checking contracts (e.g. pre-conditions and post-conditions);" This is precisely what I do (pre-conditions). If you don't use assert's for checking assertions which are logical truths if the code is correct, then what for? [Side-note: I do not use here assertions for exception handling (properly speaking) at all; instead for catching errors (properly speaking). It's just that, in python and a long list of other languages, there is a complete confusion between exceptions (which belong to the app's logic, but need to be processed specially) and errors (which don't, and are our fault and our problem). Is this topic clear?] d From ethan at stoneleaf.us Fri Feb 21 01:59:42 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 20 Feb 2014 16:59:42 -0800 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: <20140218230110.GT4519@ando> <20140220220040.GL3684@ando> Message-ID: <5306A4FE.9000504@stoneleaf.us> On 02/20/2014 02:38 PM, ????? wrote: > 2014-02-21 0:00 GMT+02:00 Steven D'Aprano wrote: >> >> Why is it dangerous, and what kind of bug? >> >> If you're talking about the fact that next(it) can raise StopIteration, >> I think you are exaggerating the danger. Firstly, quite often you don't >> mind if it raises StopIteration, since that's what you would have done >> anyway. Secondly, I don't see why raising StopIteration is so much more >> dangerous than (say) IndexError or KeyError. >> > I had this bug just the other day. I did not plan for the empty case, since it was obvious that the empty case is a bug, > so I relied on the exception being raised in this case. But I did not get the exception since it was caught in a > completely unrelated for loop. It took me a while to figure out what's going on, and it would've taken even more for > someone else, not familiar with my assumption or with the whole StopIteration thing (which I believe is the common > case). An IndexError or a KeyError would have been great in such a case. I think the basic problem is that a few exceptions are intended for the Python interpreter itself, but that's easy to forget, particularly since that list is so small: StopIteration ... um, any others? Consequently we need to remind ourselves that we are not likely to see that exception under normal circumstances; and vice-a-versa, we need to remind ourselves to catch it if we are playing with next(). -- ~Ethan~ From grosser.meister.morti at gmx.net Fri Feb 21 04:14:16 2014 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Fri, 21 Feb 2014 04:14:16 +0100 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: References: Message-ID: <5306C488.4020901@gmx.net> Am 2014-02-21 03:13, schrieb Carl Smith: > Sometimes you need to build a list in a loop, but break from the loop if some condition is met, keeping the list up to > that point. This is so common it doesn't really need an example. > > Trying to shoehorn the break keyword in to the generator expression syntax doesn't look pretty, unless you add `and > break if expr` to the end, but that has its own issues. Besides, overloading `while` is much cuter... > > ls = [ expr for name in iterable while expr ] > > ls = [ expr for name in iterable if expr while expr ] > > ls = [ expr for name in iterable if expr else expr while expr ] > > With regular generator expressions, it gives you a kind of base case, which may assist with showing off. > > I think this was already suggested once on this list. See: https://groups.google.com/forum/#!topic/python-ideas/4O1TCOa2fo8 From rosuav at gmail.com Fri Feb 21 04:37:19 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 21 Feb 2014 14:37:19 +1100 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: <5306A4FE.9000504@stoneleaf.us> References: <20140218230110.GT4519@ando> <20140220220040.GL3684@ando> <5306A4FE.9000504@stoneleaf.us> Message-ID: On Fri, Feb 21, 2014 at 11:59 AM, Ethan Furman wrote: > I think the basic problem is that a few exceptions are intended for the > Python interpreter itself, but that's easy to forget, particularly since > that list is so small: > > StopIteration > ... > um, any others? There are other exceptions that have particular semantics associated with them. SystemExit is not printed to stderr if it propagates all the way up ChrisA From greg.ewing at canterbury.ac.nz Fri Feb 21 04:50:34 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 21 Feb 2014 16:50:34 +1300 Subject: [Python-ideas] except expression In-Reply-To: References: <53037C9F.4050405@btinternet.com> <20140219001048.GX4519@ando> <530572F4.9020902@canterbury.ac.nz> <5305833F.2010203@canterbury.ac.nz> Message-ID: <5306CD0A.6020407@canterbury.ac.nz> Chris Angelico wrote: > things[i] except None if IndexError if issubclass(things, list) else KeyError That should probably be illegal without parens too: things[i] except None if (IndexError if issubclass(things, list) else KeyError) -- Greg From steve at pearwood.info Fri Feb 21 05:03:19 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 21 Feb 2014 15:03:19 +1100 Subject: [Python-ideas] Raise exception if (not) true In-Reply-To: <5306B025.1060506@gmail.com> References: <53063B70.7090608@mrabarnett.plus.com> <53066949.2020105@gmail.com> <52c145dd-9824-4d3c-b846-aeeaa145cf7b@email.android.com> <5306B025.1060506@gmail.com> Message-ID: <20140221040319.GU3684@ando> On Fri, Feb 21, 2014 at 02:47:17AM +0100, spir wrote: > On 02/20/2014 10:13 PM, Markus Unterwaditzer wrote: > >Painting the bikeshed, i'd rather have the syntax > > > >assert x > 0, ValueError("x should be positive") > > Looks good as well; (but maybe slightly more job to implement, due to > change of syntax?) You might not be aware that this piece of code already works in Python and shouldn't be changed due to backwards compatibility. py> assert False, ValueError("x should be positive") Traceback (most recent call last): File "", line 1, in AssertionError: x should be positive -- Steven From rosuav at gmail.com Fri Feb 21 05:47:17 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 21 Feb 2014 15:47:17 +1100 Subject: [Python-ideas] with expression In-Reply-To: <3605AACA-4260-4003-82B4-DCE6C6FEA389@yahoo.com> References: <3605AACA-4260-4003-82B4-DCE6C6FEA389@yahoo.com> Message-ID: On Fri, Feb 21, 2014 at 9:01 AM, Andrew Barnert wrote: > No it's not. The file will be closed as soon as the iterator is fully consumed, even if it's not deleted until much later. > > Of course it does mean that if you abandon the iterator in the middle you're leaking the generator and therefore the file until deletion. But there are many use cases where that doesn't come up. ISTM that could be dealt with by explicitly ending the generator. with open('name') as f: for line in f: if (yield line.split()): break Then you just send it a nonzero value and there you are, out of your difficulty at once! ChrisA From abarnert at yahoo.com Fri Feb 21 06:58:16 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 20 Feb 2014 21:58:16 -0800 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: <5306A4FE.9000504@stoneleaf.us> References: <20140218230110.GT4519@ando> <20140220220040.GL3684@ando> <5306A4FE.9000504@stoneleaf.us> Message-ID: <460F536B-CF5F-4855-BF87-F2CF53D01079@yahoo.com> On Feb 20, 2014, at 16:59, Ethan Furman wrote: > On 02/20/2014 02:38 PM, ????? wrote: >> 2014-02-21 0:00 GMT+02:00 Steven D'Aprano wrote: >>> >>> Why is it dangerous, and what kind of bug? >>> >>> If you're talking about the fact that next(it) can raise StopIteration, >>> I think you are exaggerating the danger. Firstly, quite often you don't >>> mind if it raises StopIteration, since that's what you would have done >>> anyway. Secondly, I don't see why raising StopIteration is so much more >>> dangerous than (say) IndexError or KeyError. >> I had this bug just the other day. I did not plan for the empty case, since it was obvious that the empty case is a bug, >> so I relied on the exception being raised in this case. But I did not get the exception since it was caught in a >> completely unrelated for loop. It took me a while to figure out what's going on, and it would've taken even more for >> someone else, not familiar with my assumption or with the whole StopIteration thing (which I believe is the common >> case). An IndexError or a KeyError would have been great in such a case. I don't think most people unfamiliar with StopIteration are calling next manually. Maybe there's a brief period where you've learned enough about how the iteration protocol works to be calling iter and next but not enough to know about StopIteration--but that period ends the first time you have to debug your code, and from then on, you know. > I think the basic problem is that a few exceptions are intended for the Python interpreter itself, but that's easy to forget, particularly since that list is so small: > > StopIteration > ... > um, any others? GeneratorExit, and in a different way KeyboardInterrupt and SystemExit. Presumably that's why none of these four inherit from StandardError and everything else does. And of course IndexError in the special case where you're using the legacy iteration protocol. Meanwhile, attribute access (and/or the default __getattribute__ implementation) handles both KeyError and AttributeError at various levels. Operators and augmented assignments use attribute access, and also directly handle AttributeError and NotImplementedError. If you count builtins that get direct data-model support as part of the interpreter, they do similar things to operators. The import system handles various IOError subclasses, and I think it may also internally raise and handle ImportError in some cases. Maybe the interpreter handles EOFError as well somewhere, I'm not sure. Probably more I didn't think of off the top of my head. So, the class of "dangerous" exceptions where you have to know that the interpreter might treat them specially includes most of the most common exceptions. > Consequently we need to remind ourselves that we are not likely to see that exception under normal circumstances; and vice-a-versa, we need to remind ourselves to catch it if we are playing with next(). This is the point. You have to know which exceptions to deal with in different kinds of code. If you're playing with next, you need to know about StopIteration. If you're playing with generator.send, you need to know about GeneratorExit. If you're creating custom number-like classes or __getattr__-based proxies, you need to know exactly when AttributeError is handled and what it means. Maybe some of this info isn't easy to find until the first time you try to do it (without a good tutorial) and run into bugs, but once you know what to look for the docs are pretty clear. From abarnert at yahoo.com Fri Feb 21 07:11:40 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 20 Feb 2014 22:11:40 -0800 Subject: [Python-ideas] Raise exception if (not) true In-Reply-To: <5306B899.4000905@gmail.com> References: <53063B70.7090608@mrabarnett.plus.com> <53066949.2020105@gmail.com> <20140220212830.GI3684@ando> <5306B899.4000905@gmail.com> Message-ID: On Feb 20, 2014, at 18:23, spir wrote: > On 02/20/2014 10:28 PM, Steven D'Aprano wrote: >> On Thu, Feb 20, 2014 at 09:44:57PM +0100, spir wrote: >> >>> >But I would like to be able to add an error type to assertions (in addition >>> >to the optional message). This is particularly useful for people (like me) >>> >who systematically check func inputs (for client debugging comfort), using >>> >assert's. >> Then your code is systematically broken, and badly so. > > Why do you speak _that_ negatively? (or should I say: _that_ violently?) And this, maybe not _that_ logically? Do you own the (copy)rights on proper usage of assert's? > >> All anyone needs >> to do to disable your checking is pass -O to the Python interpreter. >> >> assert is not a short-cut for lazy programmers to avoid having to write >> an explicit "if cond: raise SomethingAppropriate(message)". Assertions >> have specific uses. You should read this post I made last November: >> >> https://mail.python.org/pipermail/python-list/2013-November/660401.html > > I think you are wrong, at least in this very case. Assert's for me are a debugging tool (or maybe more generally a tool for improving reliability). Such checks help finding errors and correcting them (thank to hopefully clear error messages), with a higher chance these corrections happen before "too late", meaning before users pay for our bugs. What else is the proper usage of assertions? > > If not checked on function input, either with assert's or an if-raise combination, execution will break anyway, just later (later in time, and slightly further in code), because some input variable's value or type is wrong. Assert's placed that way are not strictly necessary (it's like duck typing: you don't need to check), instead they're a helpful tool for all users of your "service". > > def average (numbers): > n = len(numbers) > assert n != 0, "Cannot compute average of 'zero number'.", ValueError > return sum(numbers) / n I think you're actually suffering from the confusion you accused Steven of. His post explains the difference between internal preconditions and external value checks. Then difference has nothing to do with the form of the function, but with how it's used. If average is only called by your own code with your own values, so you know it can never be called with an empty list unless there's a bug somewhere, then you're asserting a precondition, which is exactly what asserts are for--but in that case this should be an AssertionError, not a ValueError. If average is part of an external API, or is called with user data, so you're testing for something that could fail because of user error rather than a bug in your code, then this is a perfect case for a ValueError--but it's not an assertion, it's an error check. The fact that assertions happen to work by exception handling doesn't mean you should ignore the difference between them. > On the proposal: having a correct error type instead of "AssertionError" just helps a bit making the error report clearer for developpers. > > I note in your post that, according to you, "assertions should be used for" (among other cases): > "* checking contracts (e.g. pre-conditions and post-conditions);" > This is precisely what I do (pre-conditions). Checking contracts _between parts of your code_ is the paradigm case for assert. Checking contracts _between your code and your user_ is not the same thing, and you shouldn't be conflating the two. > > If you don't use assert's for checking assertions which are logical truths if the code is correct, then what for? > > [Side-note: I do not use here assertions for exception handling (properly speaking) at all; instead for catching errors (properly speaking). It's just that, in python and a long list of other languages, there is a complete confusion between exceptions (which belong to the app's logic, but need to be processed specially) and errors (which don't, and are our fault and our problem). Is this topic clear?] > > d > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From abarnert at yahoo.com Fri Feb 21 07:16:37 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 20 Feb 2014 22:16:37 -0800 Subject: [Python-ideas] with expression In-Reply-To: <5dbbe1325154f0ba0e8ac5a6ee2da4a6@chopin.edu.pl> References: <0375b91e164416af34156bb0938a3c0a@chopin.edu.pl> <5dbbe1325154f0ba0e8ac5a6ee2da4a6@chopin.edu.pl> Message-ID: On Feb 20, 2014, at 15:10, Jan Kaliszewski wrote: > W dniu 20.02.2014 23:07, Andrew Barnert napisa?(a): >> On Feb 20, 2014, at 13:46, Jan Kaliszewski wrote: >> >>> 20.02.2014 22:33, Yann Kaiser wrote: >>> >>>> The first one could be accomplished like: >>>> >>>> x = [line.split() for line in f] with open(thisfile) as f >>>> >>>> It would keep f opened while the listcomp is being evaluated. This >>>> makes me think however of a likely accident: >>>> >>>> x = (line.split() for line in f) with open('name') as f >>>> next(x) # ValueError: I/O operation on closed file. >>>> >>>> This does mirror this mistake though: >>>> >>>> with open('name') as f: >>>> return (line.split() for line in f) >>>> >>>> When what was meant was: >>>> >>>> with open('name') as f: >>>> for line in f: >>>> yield line.split() >>>> >>>> (Which is, unfortunately, a __del__ in disguise, which is frowned upon.) >>> >>> Why 'unfortunately'? >> >> Relying on __del__ to clean up your files (and other expensive >> resources) is a bad idea if you only use CPython, and a terrible idea >> if your code might ever be run on other implementations: it means you >> don't get deterministic cleanup. > > Yup, I see the point. > > At least, starting with CPython 3.4, the risk of creating > uncollectable reference cycle no longer exists (PEP 442). > > (That, of course, does not invalidate the concern about deterministic > finalization of resources). > >> And writing something using a with statement--which was explicitly >> designed to solve that problem--that still relies on __del__ would be >> dangerously misleading. > > Unless the programmer knows what (s)he does. :) It doesn't matter what the programmer knows, it matters what the reader knows. And the reader may be the same programmer 6 months after he forgot what he was doing, or a completely different person. So misleading code is always dangerous. Sure, sometimes the best answer is to write it anyway and comment it to explain away the confusion, but that's never an ideal situation. >> However, as I pointed out in my other message, this is not really a >> __del__ in disguise, because if you use this in cases where the >> returned iterator is fully consumed in normal circumstances (e.g., a >> chain of iterator transformations that feeds into a final listcomp or >> writerows or executemany or the like), you _do_ get deterministic >> cleanup. > > Indeed. > > Regards. > *j > From abarnert at yahoo.com Fri Feb 21 07:20:52 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 20 Feb 2014 22:20:52 -0800 Subject: [Python-ideas] with expression In-Reply-To: References: <3605AACA-4260-4003-82B4-DCE6C6FEA389@yahoo.com> Message-ID: <7DD3A0DB-A07E-4452-AFB5-FFF0F8521BC0@yahoo.com> On Feb 20, 2014, at 20:47, Chris Angelico wrote: > On Fri, Feb 21, 2014 at 9:01 AM, Andrew Barnert wrote: >> No it's not. The file will be closed as soon as the iterator is fully consumed, even if it's not deleted until much later. >> >> Of course it does mean that if you abandon the iterator in the middle you're leaking the generator and therefore the file until deletion. But there are many use cases where that doesn't come up. > > ISTM that could be dealt with by explicitly ending the generator. > > with open('name') as f: > for line in f: > if (yield line.split()): break > > Then you just send it a nonzero value and there you are, out of your > difficulty at once! Good point! But there are still plenty of cases where you're feeding an iterator to some function that may just abandon it mid-way through. In that case, you cannot use the with-iter trick. Only when you know the consumer will consume the whole thing--or, as you point out, when you can make the consumer cooperate with you (which is easy if you're writing the consumer code, not so much otherwise)--is it helpful. I have a lot of scripts that use this; it's just important to know the limitations on it. From mertz at gnosis.cx Fri Feb 21 07:42:46 2014 From: mertz at gnosis.cx (David Mertz) Date: Thu, 20 Feb 2014 22:42:46 -0800 Subject: [Python-ideas] except expression In-Reply-To: <20140221001005.GQ3684@ando> References: <332da417-5439-4c13-8657-27f7bf0aee74@googlegroups.com> <5305491D.40301@stoneleaf.us> <53057CCE.2050505@gmail.com> <20140221001005.GQ3684@ando> Message-ID: While I'm growing to like the '->' operator, at least better than the colon that just looks wrong to me; I would defend Chris' characterization of it as "new." That is, the symbols exist as function annotations already, but the current use is not as an *operator*, so that's technically true. On Thu, Feb 20, 2014 at 4:10 PM, Steven D'Aprano wrote: > On Wed, Feb 19, 2014 at 10:55:58PM -0500, Yury Selivanov wrote: > > > > On 2/19/2014, 8:18 PM, Chris Angelico wrote: > > >Advantages of the arrow include -> > > I would refrain from introducing a new operator here, > > especially '->'. > > It's not new. It's used in annotations: > > py> def func(a, b) -> "Return result": > ... return "something" > ... > py> func.__annotations__ > {'return': 'Return result'} > > > -- > Steven > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mertz at gnosis.cx Fri Feb 21 07:47:55 2014 From: mertz at gnosis.cx (David Mertz) Date: Thu, 20 Feb 2014 22:47:55 -0800 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: References: Message-ID: This seems *so* close to what itertools.takewhile() does that it's really hard for me to see why we need special syntax for it. On Thu, Feb 20, 2014 at 6:13 PM, Carl Smith wrote: > Sometimes you need to build a list in a loop, but break from the loop if > some condition is met, keeping the list up to that point. This is so common > it doesn't really need an example. > > Trying to shoehorn the break keyword in to the generator expression syntax > doesn't look pretty, unless you add `and break if expr` to the end, but > that has its own issues. Besides, overloading `while` is much cuter... > > ls = [ expr for name in iterable while expr ] > > ls = [ expr for name in iterable if expr while expr ] > > ls = [ expr for name in iterable if expr else expr while expr ] > > With regular generator expressions, it gives you a kind of base case, > which may assist with showing off. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ian.team.python at gmail.com Fri Feb 21 07:52:30 2014 From: ian.team.python at gmail.com (ian o) Date: Thu, 20 Feb 2014 22:52:30 -0800 (PST) Subject: [Python-ideas] Create Python 2.8 as a transition step to Python 3.x In-Reply-To: <20140118032219.GA11381@python.ca> References: <20140118032219.GA11381@python.ca> Message-ID: <271166a7-3c33-4996-959f-056ccd87b68c@googlegroups.com> On Saturday, January 18, 2014 2:22:19 PM UTC+11, Neil Schemenauer wrote: > > The transition to Python 3 is happening but there is still a massive > amount of code that needs to be ported. One of the most disruptive > changes in Python 3 is the strict separation of bytes from unicode > strings. Most of the other incompatible changes can be handled by > 2to3. > > Here is a far out idea to make transition smoother. Release version > 2.8 of Python with nearly all Python 3.x incompatible changes except > for the bytes/unicode changes. ..... > > I now feel the python 3 - python 2 bridge idea is a far better solution. The problem is real, but this is not the solution -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Fri Feb 21 08:31:42 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 20 Feb 2014 23:31:42 -0800 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: References: Message-ID: <666A033D-E44A-412D-BA60-81B29B04FACA@yahoo.com> On Feb 20, 2014, at 22:47, David Mertz wrote: > This seems *so* close to what itertools.takewhile() does that it's really hard for me to see why we need special syntax for it. Nick Coghlan made the same observation last time around. The counter is that if clauses in comprehensions are just as close to what filter() does and we have special syntax for them. But I think that's a false analogy. I wrote up a blog post at http://stupidpythonideas.blogspot.com/2013/07/syntactic-takewhile.html that you might find interesting. It's also worth looking at the other already-working alternatives from the thread, like putting a StopIteration-raising function in the expression or a higher if clause. Also, variations of the idea have come up multiple times before that, and in 2009 one of them was written up as PEP 3142 to be officially rejected. > On Thu, Feb 20, 2014 at 6:13 PM, Carl Smith wrote: >> Sometimes you need to build a list in a loop, but break from the loop if some condition is met, keeping the list up to that point. This is so common it doesn't really need an example. >> >> Trying to shoehorn the break keyword in to the generator expression syntax doesn't look pretty, unless you add `and break if expr` to the end, but that has its own issues. Besides, overloading `while` is much cuter... >> >> ls = [ expr for name in iterable while expr ] >> >> ls = [ expr for name in iterable if expr while expr ] >> >> ls = [ expr for name in iterable if expr else expr while expr ] >> >> With regular generator expressions, it gives you a kind of base case, which may assist with showing off. >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ > > > > -- > Keeping medicines from the bloodstreams of the sick; food > from the bellies of the hungry; books from the hands of the > uneducated; technology from the underdeveloped; and putting > advocates of freedom in prisons. Intellectual property is > to the 21st century what the slave trade was to the 16th. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From carl.input at gmail.com Fri Feb 21 08:44:35 2014 From: carl.input at gmail.com (Carl Smith) Date: Fri, 21 Feb 2014 07:44:35 +0000 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: <666A033D-E44A-412D-BA60-81B29B04FACA@yahoo.com> References: <666A033D-E44A-412D-BA60-81B29B04FACA@yahoo.com> Message-ID: Cheers for all your input, and sorry for suggesting something stale. I'll pick it up in the other thread. I'll read the blog first too. Thanks again. -------------- next part -------------- An HTML attachment was scrubbed... URL: From carl.input at gmail.com Fri Feb 21 09:24:18 2014 From: carl.input at gmail.com (Carl Smith) Date: Fri, 21 Feb 2014 00:24:18 -0800 (PST) Subject: [Python-ideas] while conditional in list comprehension ?? In-Reply-To: References: <00b701cdfd5c$18d01100$4a703300$@biologie.uni-freiburg.de> <51072650.5090808@pearwood.info> <97262C84-D345-44FC-9A10-BD3D07023D6F@umbrellacode.com> Message-ID: <17075eae-1724-4d87-844f-a6ac98126398@googlegroups.com> > I'm not sure if it is the goal to be able to break out of any level of > nesting or at least that's not how I interpreted the original > proposal. It is what happens for this stop() function but only because > there's no other way. As I'm reading it, the while clause would be evaluated once for each iteration of the innermost loop, and would be able to access the current values of every loop. *ls = [ line for file in files for line in file while file or line ]* ...would become... ls = [] for file in files: for line in file: if not (file or line): break # break if not while_expression * ls.append(line)* -------------- next part -------------- An HTML attachment was scrubbed... URL: From carl.input at gmail.com Fri Feb 21 09:22:02 2014 From: carl.input at gmail.com (Carl Smith) Date: Fri, 21 Feb 2014 00:22:02 -0800 (PST) Subject: [Python-ideas] while conditional in list comprehension ?? In-Reply-To: References: <00b701cdfd5c$18d01100$4a703300$@biologie.uni-freiburg.de> <51072650.5090808@pearwood.info> <97262C84-D345-44FC-9A10-BD3D07023D6F@umbrellacode.com> Message-ID: > I'm not sure if it is the goal to be able to break out of any level of > nesting or at least that's not how I interpreted the original > proposal. It is what happens for this stop() function but only because > there's no other way. As I'm reading it, the while clause would be evaluated once for each iteration of the innermost loop, and would be able to access the current values of every loop. *ls = [ line for file in files for line in file while file or line ]* ...would become... ls = [] for file in files: for line in file: if not (file or line): break # break if not while_expression * ls.append(line) * -------------- next part -------------- An HTML attachment was scrubbed... URL: From __peter__ at web.de Fri Feb 21 10:25:55 2014 From: __peter__ at web.de (Peter Otten) Date: Fri, 21 Feb 2014 10:25:55 +0100 Subject: [Python-ideas] Function to return first(or last) true value from list References: <20140218230110.GT4519@ando> <20140220220040.GL3684@ando> Message-ID: Oscar Benjamin wrote: > On 20 February 2014 22:38, ????? wrote: >> >> 2014-02-21 0:00 GMT+02:00 Steven D'Aprano : >>> >>> On Thu, Feb 20, 2014 at 04:14:17PM +0000, Oscar Benjamin wrote: >>> > >>> > The thing is just that bare next is not something that's widely >>> > recognised as being dangerous. I've seen examples of this kind of bug >>> > in samples from many Python aficionados (including at least one from >>> > you Terry). >>> >>> Why is it dangerous, and what kind of bug? >>> >>> If you're talking about the fact that next(it) can raise StopIteration, >>> I think you are exaggerating the danger. Firstly, quite often you don't >>> mind if it raises StopIteration, since that's what you would have done >>> anyway. Secondly, I don't see why raising StopIteration is so much more >>> dangerous than (say) IndexError or KeyError. >>> >> I had this bug just the other day. I did not plan for the empty case, >> since it was obvious that the empty case is a bug, so I relied on the >> exception being raised in this case. But I did not get the exception >> since it was caught in a completely unrelated for loop. It took me a >> while to figure out what's going on, and it would've taken even more for >> someone else, not familiar with my assumption or with the whole >> StopIteration thing (which I believe is the common case). An IndexError >> or a KeyError would have been great in such a case. > > Exactly. The bug I had manifested in a StopIteration that was raised > in a semi-deterministic (dependent on slowly varying data) fashion > after ~1 hour of processing. Had it resulted in an IndexError I would > have seen a traceback and could have fixed the bug within about 5 > minutes. > > But StopIteration doesn't necessarily bubble up in the same way as > other exceptions because it can be caught and silently supressed by a > for loop (or any consumer of iterators). So instead of a traceback I > had truncated output data. It took some time to discover and verify > that the output data was truncated. Then it took some time rerunning > the script under pdb which was no help since it couldn't latch into > the suppressed exception. I assumed that it must be an exception but > there were no try/except clauses anywhere in the code. Eventually I > found it by peppering the code with: > > try: > ... > except Exception as e: > import pdb; pdb.set_trace() > > It took most of a day for me to track that down instead of 5 minutes > precisely because StopIteration is not like other exceptions. > Admittedly I'd spot a similar bug much quicker now simply because I'm > aware of the possibility. > > A simplified version of the bug is shown below: > > def itermerge(datasources): > for source in datasources: > iterator = iter(source) > first = next(iterator) > for item in iterator: > yield first * item > > data = [ > [1, 1, 2, 3], > [1, 4, 5, 6], > [], # Who put that there? > [1, 7, 8, 9], > ] > > for item in itermerge(data): > print(item) > > If you run the above then you get: > > $ python tmp.py > 1 > 2 > 3 > 4 > 5 > 6 > > So the data is silently truncated at the empty iterable. > >> It is *very* similar to the "and or" story. I think the difference is that once you've learned the lesson you stop using `and...or` while you change your usage pattern for next() to minimal scopes def process_source(source): it = iter(source) first = next(it) for item in it: yield first * item def itermerge(sources): for source in sources: yield from process_source(source) From elazarg at gmail.com Fri Feb 21 10:34:17 2014 From: elazarg at gmail.com (=?UTF-8?B?15DXnNei15bXqA==?=) Date: Fri, 21 Feb 2014 11:34:17 +0200 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: <20140218230110.GT4519@ando> <20140220220040.GL3684@ando> Message-ID: What is the "classic" use case for next() raising StopIteration, to be silently caught ? We need __next__ to do so in for loops, but when do we need it in the functional form? Elazar 2014-02-21 11:25 GMT+02:00 Peter Otten <__peter__ at web.de>: > Oscar Benjamin wrote: > > > On 20 February 2014 22:38, ????? wrote: > >> > >> 2014-02-21 0:00 GMT+02:00 Steven D'Aprano : > >>> > >>> On Thu, Feb 20, 2014 at 04:14:17PM +0000, Oscar Benjamin wrote: > >>> > > >>> > The thing is just that bare next is not something that's widely > >>> > recognised as being dangerous. I've seen examples of this kind of bug > >>> > in samples from many Python aficionados (including at least one from > >>> > you Terry). > >>> > >>> Why is it dangerous, and what kind of bug? > >>> > >>> If you're talking about the fact that next(it) can raise StopIteration, > >>> I think you are exaggerating the danger. Firstly, quite often you don't > >>> mind if it raises StopIteration, since that's what you would have done > >>> anyway. Secondly, I don't see why raising StopIteration is so much more > >>> dangerous than (say) IndexError or KeyError. > >>> > >> I had this bug just the other day. I did not plan for the empty case, > >> since it was obvious that the empty case is a bug, so I relied on the > >> exception being raised in this case. But I did not get the exception > >> since it was caught in a completely unrelated for loop. It took me a > >> while to figure out what's going on, and it would've taken even more for > >> someone else, not familiar with my assumption or with the whole > >> StopIteration thing (which I believe is the common case). An IndexError > >> or a KeyError would have been great in such a case. > > > > Exactly. The bug I had manifested in a StopIteration that was raised > > in a semi-deterministic (dependent on slowly varying data) fashion > > after ~1 hour of processing. Had it resulted in an IndexError I would > > have seen a traceback and could have fixed the bug within about 5 > > minutes. > > > > But StopIteration doesn't necessarily bubble up in the same way as > > other exceptions because it can be caught and silently supressed by a > > for loop (or any consumer of iterators). So instead of a traceback I > > had truncated output data. It took some time to discover and verify > > that the output data was truncated. Then it took some time rerunning > > the script under pdb which was no help since it couldn't latch into > > the suppressed exception. I assumed that it must be an exception but > > there were no try/except clauses anywhere in the code. Eventually I > > found it by peppering the code with: > > > > try: > > ... > > except Exception as e: > > import pdb; pdb.set_trace() > > > > It took most of a day for me to track that down instead of 5 minutes > > precisely because StopIteration is not like other exceptions. > > Admittedly I'd spot a similar bug much quicker now simply because I'm > > aware of the possibility. > > > > A simplified version of the bug is shown below: > > > > def itermerge(datasources): > > for source in datasources: > > iterator = iter(source) > > first = next(iterator) > > for item in iterator: > > yield first * item > > > > data = [ > > [1, 1, 2, 3], > > [1, 4, 5, 6], > > [], # Who put that there? > > [1, 7, 8, 9], > > ] > > > > for item in itermerge(data): > > print(item) > > > > If you run the above then you get: > > > > $ python tmp.py > > 1 > > 2 > > 3 > > 4 > > 5 > > 6 > > > > So the data is silently truncated at the empty iterable. > > > >> It is *very* similar to the "and or" story. > > I think the difference is that once you've learned the lesson you stop > using > `and...or` while you change your usage pattern for next() to minimal scopes > > def process_source(source): > it = iter(source) > first = next(it) > for item in it: > yield first * item > > def itermerge(sources): > for source in sources: > yield from process_source(source) > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From __peter__ at web.de Fri Feb 21 10:56:30 2014 From: __peter__ at web.de (Peter Otten) Date: Fri, 21 Feb 2014 10:56:30 +0100 Subject: [Python-ideas] Function to return first(or last) true value from list References: <20140218230110.GT4519@ando> <20140220220040.GL3684@ando> Message-ID: ????? wrote: > What is the "classic" use case for next() raising StopIteration, to be > silently caught ? We need __next__ to do so in for loops, but when do we > need it in the functional form? Pretty much every generator that treats the first item(s) specially, like the one I gave above: >> def process_source(source): >> it = iter(source) >> first = next(it) >> for item in it: >> yield first * item Or these: http://docs.python.org/dev/library/itertools.html#itertools.accumulate http://docs.python.org/dev/library/itertools.html#itertools.groupby http://docs.python.org/dev/library/itertools.html#itertools.islice ... The behaviour of next() is really a feature rather than a bug. From haael at interia.pl Fri Feb 21 10:40:51 2014 From: haael at interia.pl (haael at interia.pl) Date: Fri, 21 Feb 2014 10:40:51 +0100 Subject: [Python-ideas] Joining dicts again Message-ID: Hello I know this has been mangled thousand times, but let's do it once again. Why does Python not have a simple dict joining operator? >From what I read, it seems the biggest concern is: which value to pick up if both dicts have the same key. a = {'x':1} b = {'x':2} c = a | b print(c['x']) # 1 or 2? My proposal is: the value should be derermined as the result of the operation 'or'. The 'or' operator returns the first operand that evaluates to boolean True, or the last operand if all are False. So, provided we have 2 dicts 'a' and 'b' and c = a | b 1. If a key is not present in 'a' nor 'b', then it is not present in c. 2. If a key is present in 'a' and not in 'b', then c[k] = a[k]. 3. If a key is present in 'b' and not in 'a', then c[k] = b[k]. 4. If a key is present both in 'a' and 'b', then c[k] = a[k] or b[k]. We could also naturally define dict intersection operator using the operator 'and' to pick values. The key would have to be present in both operands. Forgive me I wasn't able to read all previous discussions. Was this proposed before? Thanks haael From rosuav at gmail.com Fri Feb 21 11:15:09 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 21 Feb 2014 21:15:09 +1100 Subject: [Python-ideas] Joining dicts again In-Reply-To: References: Message-ID: On Fri, Feb 21, 2014 at 8:40 PM, wrote: > I know this has been mangled thousand times, but let's do it once again. > > Why does Python not have a simple dict joining operator? > > From what I read, it seems the biggest concern is: which value to pick up if both dicts have the same key. > a = {'x':1} > b = {'x':2} > c = a | b > print(c['x']) # 1 or 2? If you can pick one of the dicts to "win", then you can use this notation: c = dict(a, **b) Anything in b will override a. It's short and a single expression (unlike "c = a.copy(); c.update(b)"). It's not perfectly clear what's happening, though, so it may warrant a comment. ChrisA From cory at lukasa.co.uk Fri Feb 21 11:17:38 2014 From: cory at lukasa.co.uk (Cory Benfield) Date: Fri, 21 Feb 2014 10:17:38 +0000 Subject: [Python-ideas] Joining dicts again In-Reply-To: References: Message-ID: On 21 February 2014 09:40, wrote: > From what I read, it seems the biggest concern is: which value to pick up if both dicts have the same key. > a = {'x':1} > b = {'x':2} > c = a | b > print(c['x']) # 1 or 2? > > My proposal is: the value should be derermined as the result of the operation 'or'. The 'or' operator returns the first operand that evaluates to boolean True, or the last operand if all are False. > > So, provided we have 2 dicts 'a' and 'b' and c = a | b > > 1. If a key is not present in 'a' nor 'b', then it is not present in c. > 2. If a key is present in 'a' and not in 'b', then c[k] = a[k]. > 3. If a key is present in 'b' and not in 'a', then c[k] = b[k]. > 4. If a key is present both in 'a' and 'b', then c[k] = a[k] or b[k]. This seems to me to be exactly the same as dict.update(): a = {'x': 1} b = {'x': 2} c = b.copy() c.update(a) I'd be very reluctant to overload the bitwise-or operator to mean 'dictionary merging'. It's just not totally obvious to me that this is what would happen. From steve at pearwood.info Fri Feb 21 11:24:03 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 21 Feb 2014 21:24:03 +1100 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: References: Message-ID: <20140221102403.GV3684@ando> On Fri, Feb 21, 2014 at 02:13:13AM +0000, Carl Smith wrote: > Sometimes you need to build a list in a loop, but break from the loop if > some condition is met, keeping the list up to that point. This is so common > it doesn't really need an example. > > Trying to shoehorn the break keyword in to the generator expression syntax > doesn't look pretty, unless you add `and break if expr` to the end, but > that has its own issues. The main issue being that it gives a SyntaxError :-) > Besides, overloading `while` is much cuter... > > ls = [ expr for name in iterable while expr ] I would *love* this syntax. I've read the various arguments against it, and they don't convince me. It is functionality which is often needed and requested, e.g. http://stackoverflow.com/questions/5505891/using-while-in-list-comprehension-or-generator-expressions Clojure has list comprehensions that behave this way: http://clojuredocs.org/clojure_core/clojure.core/for but alas both Nick Coglan and (if I recall correctly) Guido have ruled that Python won't get this, so until the Revolution comes, it isn't going to happen. -- Steven From abarnert at yahoo.com Fri Feb 21 12:22:22 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 21 Feb 2014 03:22:22 -0800 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: <20140221102403.GV3684@ando> References: <20140221102403.GV3684@ando> Message-ID: On Feb 21, 2014, at 2:24, Steven D'Aprano wrote: > On Fri, Feb 21, 2014 at 02:13:13AM +0000, Carl Smith wrote: >> Sometimes you need to build a list in a loop, but break from the loop if >> some condition is met, keeping the list up to that point. This is so common >> it doesn't really need an example. >> >> Trying to shoehorn the break keyword in to the generator expression syntax >> doesn't look pretty, unless you add `and break if expr` to the end, but >> that has its own issues. > > The main issue being that it gives a SyntaxError :-) > > >> Besides, overloading `while` is much cuter... >> >> ls = [ expr for name in iterable while expr ] > > I would *love* this syntax. I've read the various arguments against it, > and they don't convince me. It is functionality which is often needed > and requested, e.g. > > http://stackoverflow.com/questions/5505891/using-while-in-list-comprehension-or-generator-expressions > > > Clojure has list comprehensions that behave this way: > > http://clojuredocs.org/clojure_core/clojure.core/for As I pointed out in the thread last summer (and briefly mentioned in the blog post linked earlier in this thread), that's because Clojure comprehensions are not actually the same as Python comprehensions. Python comprehensions, as borrowed from Haskell, have a free sequence of clauses that nest from left to right. There is no sensible way to interpret a while clause that nests under a for or if clause. Clojure-style comprehensions, as borrowed from (I think) Racket only have nested loops, no other clauses--but then each loop has modifiers. Instead of if clauses, you have a "filter" that modifies a loop. You can't have a while clause, but you can have a "takewhile" that modifies a loop. You get a clean, simple, and easy-to-extend syntax. This has the down side that you can't toss two if conditions into a single loop, but that almost never matters (just and them together) It has the up side that new modifiers don't have to nest under arbitrary clauses, they just have to modify a loop. In Haskell, because it's so trivial to modify the iterable itself (e.g., with a takewhile whose predicate is composed on the fly), there really is no cost to using the Haskell-style comprehension. Sadly, that isn't true in Python, so the limitation hits us harder. Can we get from here to there? Well, it might be possible to come up with a syntax for Clojure-style comprehensions that handles all currently valid Python comprehensions without two adjacent if clauses with the same meaning, despite getting there by a slightly different route. If so, just deprecate adjacent if clauses, then switch to the syntax and semantics where if is a modifier on the for clause instead of a separate clause and adjust the docs accordingly, then add while as another modifier. Or, it would almost certainly be possible to design a hybrid, where while is a loop modifier on the for clause rather than a nested clause, but if stays the way it is today. And if you go that way, it probably makes sense to add the same modifier onto the for statement. Then all of the docs that explain comprehensions could remain unchanged. And no need for a deprecation period or removing adjacent ifs. This may sound more radical than "just add a while clause", but given that the latter isn't actually a sensible proposal in the first place, you're not going to be able to flesh it out in a way that gets Nick Coghlan to reconsider; the former, you might have a shot at. > but alas both Nick Coglan and (if I recall correctly) Guido have ruled > that Python won't get this, so until the Revolution comes, it isn't > going to happen. You could always just fork Python instead of revolting. ;) From ncoghlan at gmail.com Fri Feb 21 13:18:26 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 21 Feb 2014 22:18:26 +1000 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: <20140221102403.GV3684@ando> References: <20140221102403.GV3684@ando> Message-ID: On 21 February 2014 20:24, Steven D'Aprano wrote: > > but alas both Nick Coglan and (if I recall correctly) Guido have ruled > that Python won't get this, so until the Revolution comes, it isn't > going to happen. It's not that it can't happen - it's that someone would need to build a case of a similar calibre to the one Chris Angelico is currently putting together for except expressions in PEP 463 :) However, it's a *much* bigger challenge in this case, as itertools.takewhile exists, whereas there's currently no way to do exception handling as part of a larger expression without major contortions. Re-reading Andrew's post at http://stupidpythonideas.blogspot.com.au/2013/07/syntactic-takewhile.html, I'm actually more intrigued by Haskell's shorthand for lambda functions that consist of a single binary operator. Consider: isprime = all(n % p for p in takewhile((lambda p: p**2 < n), primes_seen)) Is there are a nicer way to write that? The Haskell equivalent is: (< n) . (** 2) That's not very readable to most Python programmers, but what if you could write something like: isprime = all(n % p for p in takewhile((: ? ** 2 < n), primes_seen)) This is somewhat similar to the implicit lambda proposal in http://legacy.python.org/dev/peps/pep-0312/, but with the following two essential differences: 1. The parentheses would be required (as per generator expressions, and as is being discussed for except expressions) 2. By using a "?" token within the implicit lambda, you would create a lambda that takes a single argument. If there is no such token, then it would take no arguments. However, the examples in that PEP are no longer useful, as they have since been addressed by dedicated constructs (conditional expressions in PEP 308 and context managers in PEP 343). Note that I'm not saying this is a good idea. However, I am saying I think it is an idea that may be worth exploring further to see if it also addresses at least some of the use cases given in PEP's 403 and 3150 (especially in combination with PEP 463) by making custom key and predicate functions easier to define. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Fri Feb 21 13:25:39 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 21 Feb 2014 22:25:39 +1000 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: References: <20140221102403.GV3684@ando> Message-ID: On 21 February 2014 22:18, Nick Coghlan wrote: > That's not very readable to most Python programmers, but what if you > could write something like: > > isprime = all(n % p for p in takewhile((: ? ** 2 < n), primes_seen)) > > This is somewhat similar to the implicit lambda proposal in > http://legacy.python.org/dev/peps/pep-0312/, but with the following > two essential differences: > > 1. The parentheses would be required (as per generator expressions, > and as is being discussed for except expressions) > 2. By using a "?" token within the implicit lambda, you would create a > lambda that takes a single argument. If there is no such token, then > it would take no arguments. Oh, and under such a proposal, the generator expression: (x for x in seq) would be semantically equivalent to: (: yield x for x in ?)(seq) Currently, there's no underlying construct you can decompose a generator expression into, because there's no notation for a lambda expression with an anonymous parameter. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From mal at egenix.com Fri Feb 21 14:03:20 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 21 Feb 2014 14:03:20 +0100 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: References: <20140221102403.GV3684@ando> Message-ID: <53074E98.4020205@egenix.com> On 21.02.2014 13:25, Nick Coghlan wrote: > On 21 February 2014 22:18, Nick Coghlan wrote: >> That's not very readable to most Python programmers, but what if you >> could write something like: >> >> isprime = all(n % p for p in takewhile((: ? ** 2 < n), primes_seen)) >> >> This is somewhat similar to the implicit lambda proposal in >> http://legacy.python.org/dev/peps/pep-0312/, but with the following >> two essential differences: >> >> 1. The parentheses would be required (as per generator expressions, >> and as is being discussed for except expressions) >> 2. By using a "?" token within the implicit lambda, you would create a >> lambda that takes a single argument. If there is no such token, then >> it would take no arguments. > > Oh, and under such a proposal, the generator expression: > > (x for x in seq) > > would be semantically equivalent to: > > (: yield x for x in ?)(seq) > > Currently, there's no underlying construct you can decompose a > generator expression into, because there's no notation for a lambda > expression with an anonymous parameter. Hmm, this reminds me too much of regular expression syntax :-) I wonder why people are so keen on stuffing too much logic into a single line. Must be a twitter/SMS side-effect. Programs don't get faster that way, they don't get more readable, you don't get to do more things that couldn't do otherwise and requiring a master in computer science to be able to understand what goes on in one of those magical lines doesn't feel right to me either, given that we are promoting Python as first programming language. Of course, tossing around ideas like these is fun and I don't want to spoil it. Eventually something useful will come out of these discussions, I'm sure :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 21 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From masklinn at masklinn.net Fri Feb 21 14:34:49 2014 From: masklinn at masklinn.net (Masklinn) Date: Fri, 21 Feb 2014 14:34:49 +0100 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: <53074E98.4020205@egenix.com> References: <20140221102403.GV3684@ando> <53074E98.4020205@egenix.com> Message-ID: <88C967DE-DC83-4456-8B72-1FCA5577D381@masklinn.net> On 2014-02-21, at 14:03 , M.-A. Lemburg wrote: > On 21.02.2014 13:25, Nick Coghlan wrote: >> On 21 February 2014 22:18, Nick Coghlan wrote: >>> That's not very readable to most Python programmers, but what if you >>> could write something like: >>> >>> isprime = all(n % p for p in takewhile((: ? ** 2 < n), primes_seen)) >>> >>> This is somewhat similar to the implicit lambda proposal in >>> http://legacy.python.org/dev/peps/pep-0312/, but with the following >>> two essential differences: >>> >>> 1. The parentheses would be required (as per generator expressions, >>> and as is being discussed for except expressions) >>> 2. By using a "?" token within the implicit lambda, you would create a >>> lambda that takes a single argument. If there is no such token, then >>> it would take no arguments. >> >> Oh, and under such a proposal, the generator expression: >> >> (x for x in seq) >> >> would be semantically equivalent to: >> >> (: yield x for x in ?)(seq) >> >> Currently, there's no underlying construct you can decompose a >> generator expression into, because there's no notation for a lambda >> expression with an anonymous parameter. > > Hmm, this reminds me too much of regular expression syntax :-) > > I wonder why people are so keen on stuffing too much logic into > a single line. Must be a twitter/SMS side-effect. "Line" is a red herring, I think. Nobody *really* cares about lines. "Expression" is the important factor. Moving stuff out of expressions and into separate statements requires finding out a name for the result, is generally significantly more verbose and isn't necessarily clearer. Depending on the situation, it may also require significant reworking of existing (supposedly working) code. I usually write non-trivial generator expressions over multiple lines, I may be after them remaining a single (readable) expression depending on context. Well that's just my experience anyway. > Programs don't get faster that way, they don't get more readable, > you don't get to do more things that couldn't do otherwise > and requiring a master in computer science to be able to understand > what goes on in one of those magical lines doesn't feel right > to me either, given that we are promoting Python as first > programming language. Was the MSCS insult really necessary? > Of course, tossing around ideas like these is fun and I don't > want to spoil it. Eventually something useful will come out of > these discussions, I'm sure :-) From elazarg at gmail.com Fri Feb 21 14:39:04 2014 From: elazarg at gmail.com (=?UTF-8?B?15DXnNei15bXqA==?=) Date: Fri, 21 Feb 2014 15:39:04 +0200 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: <88C967DE-DC83-4456-8B72-1FCA5577D381@masklinn.net> References: <20140221102403.GV3684@ando> <53074E98.4020205@egenix.com> <88C967DE-DC83-4456-8B72-1FCA5577D381@masklinn.net> Message-ID: 2014-02-21 15:34 GMT+02:00 Masklinn : > > On 2014-02-21, at 14:03 , M.-A. Lemburg wrote: > > On 21.02.2014 13:25, Nick Coghlan wrote: > > I wonder why people are so keen on stuffing too much logic into > > a single line. Must be a twitter/SMS side-effect. > > "Line" is a red herring, I think. Nobody *really* cares about lines. People do. There are only a handful of them in the screen. And the eyes catch even less in a single glance. --- Elazar -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Fri Feb 21 14:45:51 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 21 Feb 2014 14:45:51 +0100 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword References: <20140221102403.GV3684@ando> <53074E98.4020205@egenix.com> <88C967DE-DC83-4456-8B72-1FCA5577D381@masklinn.net> Message-ID: <20140221144551.19aed69e@fsol> > 2014-02-21 15:34 GMT+02:00 Masklinn : > > > > On 2014-02-21, at 14:03 , M.-A. Lemburg wrote: > > > On 21.02.2014 13:25, Nick Coghlan wrote: > > > I wonder why people are so keen on stuffing too much logic into > > > a single line. Must be a twitter/SMS side-effect. > > > > "Line" is a red herring, I think. Nobody *really* cares about lines. > > People do. There are only a handful of them in the screen. And the eyes > catch even less in a single glance. But on the contrary, trying to compress control flow in a single line breaks the visual structure expectation for Python code (in which most control flow is introduced by a colon and a linefeed). Regards Antoine. From mal at egenix.com Fri Feb 21 14:47:33 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 21 Feb 2014 14:47:33 +0100 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: <88C967DE-DC83-4456-8B72-1FCA5577D381@masklinn.net> References: <20140221102403.GV3684@ando> <53074E98.4020205@egenix.com> <88C967DE-DC83-4456-8B72-1FCA5577D381@masklinn.net> Message-ID: <530758F5.3050702@egenix.com> On 21.02.2014 14:34, Masklinn wrote: > On 2014-02-21, at 14:03 , M.-A. Lemburg wrote: >> On 21.02.2014 13:25, Nick Coghlan wrote: >>> On 21 February 2014 22:18, Nick Coghlan wrote: >>>> That's not very readable to most Python programmers, but what if you >>>> could write something like: >>>> >>>> isprime = all(n % p for p in takewhile((: ? ** 2 < n), primes_seen)) >>>> >>>> This is somewhat similar to the implicit lambda proposal in >>>> http://legacy.python.org/dev/peps/pep-0312/, but with the following >>>> two essential differences: >>>> >>>> 1. The parentheses would be required (as per generator expressions, >>>> and as is being discussed for except expressions) >>>> 2. By using a "?" token within the implicit lambda, you would create a >>>> lambda that takes a single argument. If there is no such token, then >>>> it would take no arguments. >>> >>> Oh, and under such a proposal, the generator expression: >>> >>> (x for x in seq) >>> >>> would be semantically equivalent to: >>> >>> (: yield x for x in ?)(seq) >>> >>> Currently, there's no underlying construct you can decompose a >>> generator expression into, because there's no notation for a lambda >>> expression with an anonymous parameter. >> >> Hmm, this reminds me too much of regular expression syntax :-) >> >> I wonder why people are so keen on stuffing too much logic into >> a single line. Must be a twitter/SMS side-effect. > > "Line" is a red herring, I think. Nobody *really* cares about lines. > "Expression" is the important factor. Moving stuff out of expressions > and into separate statements requires finding out a name for the result, > is generally significantly more verbose and isn't necessarily clearer. But it helps a lot in debugging the code :-) Debugging complicated expressions becomes really difficult and adding a few local variables to put on the watch list helps a lot. > Depending on the situation, it may also require significant reworking of > existing (supposedly working) code. > > I usually write non-trivial generator expressions over multiple lines, > I may be after them remaining a single (readable) expression depending > on context. > > Well that's just my experience anyway. > >> Programs don't get faster that way, they don't get more readable, >> you don't get to do more things that couldn't do otherwise >> and requiring a master in computer science to be able to understand >> what goes on in one of those magical lines doesn't feel right >> to me either, given that we are promoting Python as first >> programming language. > > Was the MSCS insult really necessary? Which "insult" ? I have no intention insulting anyone and apologize if the above was interpreted that way. I was just referring to the complicated nature of the resulting expressions and raising a concern that by introducing more constructs to write such things, we actively support writing Python programs which are no longer easy to read. >> Of course, tossing around ideas like these is fun and I don't >> want to spoil it. Eventually something useful will come out of >> these discussions, I'm sure :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 21 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From ncoghlan at gmail.com Fri Feb 21 15:17:54 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 22 Feb 2014 00:17:54 +1000 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: <53074E98.4020205@egenix.com> References: <20140221102403.GV3684@ando> <53074E98.4020205@egenix.com> Message-ID: On 21 February 2014 23:03, M.-A. Lemburg wrote: > On 21.02.2014 13:25, Nick Coghlan wrote: >> On 21 February 2014 22:18, Nick Coghlan wrote: >>> That's not very readable to most Python programmers, but what if you >>> could write something like: >>> >>> isprime = all(n % p for p in takewhile((: ? ** 2 < n), primes_seen)) >>> >>> This is somewhat similar to the implicit lambda proposal in >>> http://legacy.python.org/dev/peps/pep-0312/, but with the following >>> two essential differences: >>> >>> 1. The parentheses would be required (as per generator expressions, >>> and as is being discussed for except expressions) >>> 2. By using a "?" token within the implicit lambda, you would create a >>> lambda that takes a single argument. If there is no such token, then >>> it would take no arguments. >> >> Oh, and under such a proposal, the generator expression: >> >> (x for x in seq) >> >> would be semantically equivalent to: >> >> (: yield x for x in ?)(seq) >> >> Currently, there's no underlying construct you can decompose a >> generator expression into, because there's no notation for a lambda >> expression with an anonymous parameter. > > Hmm, this reminds me too much of regular expression syntax :-) > > I wonder why people are so keen on stuffing too much logic into > a single line. Must be a twitter/SMS side-effect. > > Programs don't get faster that way, they don't get more readable, > you don't get to do more things that couldn't do otherwise > and requiring a master in computer science to be able to understand > what goes on in one of those magical lines doesn't feel right > to me either, given that we are promoting Python as first > programming language. > > Of course, tossing around ideas like these is fun and I don't > want to spoil it. Eventually something useful will come out of > these discussions, I'm sure :-) The reason I keep beating my head against this particular wall (cf. PEP 403's @in clauses and PEP 3150's given suites) is that my personal goal for Python is that it should be a tool that lets people express what they are thinking clearly and relatively concisely. As far as I have been able to tell, the persistent requests for "multi-line lambdas", cleaner lambda syntax, etc, are because Python doesn't currently make it easy to express a lot of operations that involve higher order manipulation of "one shot" callables - closures or custom functions where you *don't* want to re-use them, but Python still forces you to pull them out and name them. I think PEP 403 is my best current description of the mental speed bump involved: http://www.python.org/dev/peps/pep-0403/ Decorators are an example where this used to be a problem (compounded by other issues related to repeating the name multiple times): def f(cls): ... f = classmethod(f) But this was replaced by the much cleaner: @classmethod def f(cls): ... Like try/except/finally -> with statements, decorators replaced a "bracketed" construct with a "prefix" only construct, greatly improving the readability in the process. Extracting a named one-shot function inline is problematic in much the same way: def custom_key(v): # Helper for a sorted call you don't know is coming yet ... x = sorted(seq, key=custom_key) # Oh, that's what it is for The idea behind both PEP 403 and 3150 is to be able to do a similar bracketed -> prefix only transformation for more constructs where the callable is incidental to the real operation. The PEP 403 approach to custom key functions: @in x = sorted(seq, key=custom_key) def custom_key(v): ... I'm not actually happy with the PEP 403 syntax (or the complexity of PEP 3150), but I think the problem they both attempt to solve is one worth having a better answer for. I just haven't figured out what that answer might look like yet (and neither has anyone else) :P Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From denis.spir at gmail.com Fri Feb 21 16:28:51 2014 From: denis.spir at gmail.com (spir) Date: Fri, 21 Feb 2014 16:28:51 +0100 Subject: [Python-ideas] Raise exception if (not) true In-Reply-To: References: <53063B70.7090608@mrabarnett.plus.com> <53066949.2020105@gmail.com> <20140220212830.GI3684@ando> <5306B899.4000905@gmail.com> Message-ID: <530770B3.8030103@gmail.com> On 02/21/2014 07:11 AM, Andrew Barnert wrote: > On Feb 20, 2014, at 18:23, spir wrote: > >> On 02/20/2014 10:28 PM, Steven D'Aprano wrote: >>> On Thu, Feb 20, 2014 at 09:44:57PM +0100, spir wrote: >>> >>>>> But I would like to be able to add an error type to assertions (in addition >>>>> to the optional message). This is particularly useful for people (like me) >>>>> who systematically check func inputs (for client debugging comfort), using >>>>> assert's. >>> Then your code is systematically broken, and badly so. >> >> Why do you speak _that_ negatively? (or should I say: _that_ violently?) And this, maybe not _that_ logically? Do you own the (copy)rights on proper usage of assert's? >> >>> All anyone needs >>> to do to disable your checking is pass -O to the Python interpreter. >>> >>> assert is not a short-cut for lazy programmers to avoid having to write >>> an explicit "if cond: raise SomethingAppropriate(message)". Assertions >>> have specific uses. You should read this post I made last November: >>> >>> https://mail.python.org/pipermail/python-list/2013-November/660401.html >> >> I think you are wrong, at least in this very case. Assert's for me are a debugging tool (or maybe more generally a tool for improving reliability). Such checks help finding errors and correcting them (thank to hopefully clear error messages), with a higher chance these corrections happen before "too late", meaning before users pay for our bugs. What else is the proper usage of assertions? >> >> If not checked on function input, either with assert's or an if-raise combination, execution will break anyway, just later (later in time, and slightly further in code), because some input variable's value or type is wrong. Assert's placed that way are not strictly necessary (it's like duck typing: you don't need to check), instead they're a helpful tool for all users of your "service". >> >> def average (numbers): >> n = len(numbers) >> assert n != 0, "Cannot compute average of 'zero number'.", ValueError >> return sum(numbers) / n > > I think you're actually suffering from the confusion you accused Steven of. His post explains the difference between internal preconditions and external value checks. Then difference has nothing to do with the form of the function, but with how it's used. > > If average is only called by your own code with your own values, so you know it can never be called with an empty list unless there's a bug somewhere, then you're asserting a precondition, which is exactly what asserts are for--but in that case this should be an AssertionError, not a ValueError. > > If average is part of an external API, or is called with user data, so you're testing for something that could fail because of user error rather than a bug in your code, then this is a perfect case for a ValueError--but it's not an assertion, it's an error check. > > The fact that assertions happen to work by exception handling doesn't mean you should ignore the difference between them. See below. >> [Side-note: I do not use here assertions for exception handling (properly speaking) at all; instead for catching errors (properly speaking). It's just that, in python and a long list of other languages, there is a complete confusion between exceptions (which belong to the app's logic, but need to be processed specially) and errors (which don't, and are our fault and our problem). Is this topic clear?] I think this is the _relevant_ difference. You and Steven point at the difference between illogical facts proper to my code, say internal, and ones due to client (user) code, say external. This is a true difference, indeed, but maybe not that relevant here. If this is an external interface, meaning in my example the function 'average' is to be used by clients, then calling it with an empty array of numbers is an error in any case. Or rather, it is an obvious _anomaly_ on the side of the procedure, which i can detect, and it is certainly a *symptom* of a true error on their side. (The detected anomaly is but an effect of an error, which is the cause, _somewhere else_ in their code.) The reason is obvious: average of no-number-at-all makes no sense. [And this precisely is what the error message should say, and the improvement I wish to make, instead of a potential division-by-zero error which is just a consequence.] Whether this error is in their code or mine does not change it signicantly, I guess. It is an error. [And is is a value error in fact, reason why I wish to change the feature to have a better message.] Note that an error is a _break_ of the the application logic; it is actual semantics of the program that does not belong to the app's logic. In any case, it is certainly _not_ an exception. That it were an exception would mean that this case (the array of numbers is empty) _belongs to_ the application logic. If so, the client code should just deal with this special case specially. Right? And certainly not call average (blindly). In fact, there are good chances that this exceptional case is not only not properly dealt with _here_, where we are about to call 'average', but also elsewhere in code; and probably we should not reach this very point in code at all, but instead have branched on another code path earlier. Exception handling, exception catching precisely, should (in my view) only be used for such exceptional situations that (1) nevertheless belong to the application logic, so should be dealt with properly (2) are impredictable on the client side, unlike the example case. Such cases happen for instance with search/find procedures (we don't know whether the searched item is there before trying to find it) or in relation with elements external to the app's "world", such as the file system or the user. This is the proper usage of exception machinaries in my view (and I only use them for that). [1] d [1] Side-note: I have come to think that the common exception handling mechanisms (again, to be used only in impredictable exception cases) are wrong. The reason is that only clients know whether a case of anomaly (which would cause the procedure to "throw", to "raise") is actually an unknown error (the collection should hold this item) or instead an exception (the item may not be there). Thus, clients should be in control of exception handling, not service procedures. Presently, the latter just lauch the exception machinary (big, complicated, with longjumps and stack unwinding mechanisms). Instead, what we need is allow clients to tell that this is an exceptional, but impredictable, case, and no throwing and catching should happen at all. Some coding practices end up with a similar result by adding an "optional" param to such procedures; in which case if an anomaly happens they just say it, somehow, instead of failing. def item_index (col, item, opt=false): ... search item ... if index: return index if opt: return None # or -1 raise ... This means, for me, that in a sufficiently flexible language (read: dynamic or sophisticated), a programming community could live without exception handling features, instead just promote a coding style for such particular service procedures (a limited list). One possible view is that languages where this is impracticle (too rigid) are otherwise wrongly-designed, but things are not that simple, maybe. Or, we could have a builtin feature making such handling clear & simple. In case of predictable exceptions, the simple (and imo right) way is just to use an if/else branching. if numbers: avg = average(numbers) else: # deal with exception Similarly, an alternative construct may simply allow not failing in case of unpredictable exceptions: maybe: idx = item_index(col, item) else: # deal with exception This superficially looks like typical try/except or try/catch, but in fact here no throwing & catching happen at all. The failure is just signaled to the caller side, somehow (a plain flag). We may refine the construct with discrimination of error types, just like except/catch branches (may be good, indeed, but requires proper usage by client, which is not always well done). But in any case clients control the exception handling mechanism, everything is far simpler and easier, and there are no unneeded processes behing the stage. From denis.spir at gmail.com Fri Feb 21 16:35:38 2014 From: denis.spir at gmail.com (spir) Date: Fri, 21 Feb 2014 16:35:38 +0100 Subject: [Python-ideas] Joining dicts again In-Reply-To: References: Message-ID: <5307724A.4020102@gmail.com> On 02/21/2014 10:40 AM, haael at interia.pl wrote: > > Hello > > I know this has been mangled thousand times, but let's do it once again. > > Why does Python not have a simple dict joining operator? > > From what I read, it seems the biggest concern is: which value to pick up if both dicts have the same key. > a = {'x':1} > b = {'x':2} > c = a | b > print(c['x']) # 1 or 2? > > My proposal is: the value should be derermined as the result of the operation 'or'. The 'or' operator returns the first operand that evaluates to boolean True, or the last operand if all are False. > > So, provided we have 2 dicts 'a' and 'b' and c = a | b > > 1. If a key is not present in 'a' nor 'b', then it is not present in c. > 2. If a key is present in 'a' and not in 'b', then c[k] = a[k]. > 3. If a key is present in 'b' and not in 'a', then c[k] = b[k]. > 4. If a key is present both in 'a' and 'b', then c[k] = a[k] or b[k]. > > > We could also naturally define dict intersection operator using the operator 'and' to pick values. The key would have to be present in both operands. > > Forgive me I wasn't able to read all previous discussions. Was this proposed before? I don't see how this significantly differs from dict update, except that for you the first one to speak should win, instead of the opposite; so that you have to reverse the operands. d From franck.michea at gmail.com Fri Feb 21 16:44:10 2014 From: franck.michea at gmail.com (Franck Michea) Date: Fri, 21 Feb 2014 16:44:10 +0100 Subject: [Python-ideas] decorator syntax limitation In-Reply-To: References: <20140220223146.168ee0c4@homer> <25f71eb666beca0c7c5a5886540c97e8@chopin.edu.pl> Message-ID: On Thu, Feb 20, 2014 at 10:58 PM, Nick Coghlan wrote: > On 21 February 2014 07:55, Jan Kaliszewski wrote: >> >> As far as I remember, generally, the belief that this restriction >> is necessary seemed not to be very strong. > > Yup, but the gains aren't that great, either - it's never irritated > anyone enough for them to write up the necessary PEP and implement the > Grammar change :) > > Cheers, > Nick. > Well, I started to work on a public API design and was hoping I could use this pattern, but I had another idea since then that would work without syntax modification. Not promising anything, but I may look into this within the next few months (~9) if it bothers me enough :) Haoyi's trick is awesome, but I don't think it's a good idea for something public :/ Though I'll keep that in mind if I ever need that in my code :) Thank you for your answers guys! -- Franck Michea - EPITA/GISTRE/LSE 2014 From oscar.j.benjamin at gmail.com Fri Feb 21 16:44:54 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Fri, 21 Feb 2014 15:44:54 +0000 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: <20140218230110.GT4519@ando> <20140220220040.GL3684@ando> Message-ID: On 21 February 2014 09:25, Peter Otten <__peter__ at web.de> wrote: >>> >>> It is *very* similar to the "and or" story. > > I think the difference is that once you've learned the lesson you stop using > `and...or` while you change your usage pattern for next() to minimal scopes > > def process_source(source): > it = iter(source) > first = next(it) > for item in it: > yield first * item > > def itermerge(sources): > for source in sources: > yield from process_source(source) Maybe but is it really correct to just ignore that empty iterable? When I use sequences and write first = seq[0] I'm deliberately asserting that seq is non-empty. I want to see a traceback if for whatever reason it should turn out to be an empty sequence. Using next() and allowing the StopIteration to terminate what you're doing assumes that it's okay to just ignore an empty iterable and go do something else. Leaking StopIteration may be a conscious decision but it's not clear when looking at first = next(it) whether it is. In my own code I would put a comment there to indicate that I have considered the implication and decided that it's okay (and then when I see a bare next with no comment it instantly arouses suspicion). Oscar From oscar.j.benjamin at gmail.com Fri Feb 21 17:05:19 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Fri, 21 Feb 2014 16:05:19 +0000 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: <20140218230110.GT4519@ando> <20140220220040.GL3684@ando> Message-ID: On 21 February 2014 09:56, Peter Otten <__peter__ at web.de> wrote: > ????? wrote: > >> What is the "classic" use case for next() raising StopIteration, to be >> silently caught ? We need __next__ to do so in for loops, but when do we >> need it in the functional form? > > Pretty much every generator that treats the first item(s) specially, like > the one I gave above: But there are also cases where that implicit behaviour is not desired. I would rather have to explicitly return when that's what I want so that the control flow is very clear. For example when you use csv.DictReader and don't supply the fieldnames argument you are saying that you want to read a csv file with one header line containing column labels and zero or more data lines. To me a fully empty file (with no header line) is invalid but csv.DictReader will accept it as a csv file with zero rows. I would prefer an error in this case since it would only happen in my usage if an error had occurred somewhere else. One of the examples you linked to shows exactly my own practice of marking a next call with a comment: def __next__(self): while self.currkey == self.tgtkey: self.currvalue = next(self.it) # Exit on StopIteration self.currkey = self.keyfunc(self.currvalue) self.tgtkey = self.currkey return (self.currkey, self._grouper(self.tgtkey)) def _grouper(self, tgtkey): while self.currkey == tgtkey: yield self.currvalue self.currvalue = next(self.it) # Exit on StopIteration self.currkey = self.keyfunc(self.currvalue) IMO if you want that behaviour then you should mark it to show that you thought about it and otherwise I'll treat any bare next with suspicion. Oscar From markus at unterwaditzer.net Fri Feb 21 17:05:58 2014 From: markus at unterwaditzer.net (Markus Unterwaditzer) Date: Fri, 21 Feb 2014 17:05:58 +0100 Subject: [Python-ideas] Joining dicts again In-Reply-To: References: Message-ID: <3f21074da4e8e68b4863c1f312848a5b@draco.uberspace.de> On 2014-02-21 11:17, Cory Benfield wrote: > On 21 February 2014 09:40, wrote: >> From what I read, it seems the biggest concern is: which value to pick >> up if both dicts have the same key. >> a = {'x':1} >> b = {'x':2} >> c = a | b >> print(c['x']) # 1 or 2? >> >> My proposal is: the value should be derermined as the result of the >> operation 'or'. The 'or' operator returns the first operand that >> evaluates to boolean True, or the last operand if all are False. >> >> So, provided we have 2 dicts 'a' and 'b' and c = a | b >> >> 1. If a key is not present in 'a' nor 'b', then it is not present in >> c. >> 2. If a key is present in 'a' and not in 'b', then c[k] = a[k]. >> 3. If a key is present in 'b' and not in 'a', then c[k] = b[k]. >> 4. If a key is present both in 'a' and 'b', then c[k] = a[k] or b[k]. > > This seems to me to be exactly the same as dict.update(): > > a = {'x': 1} > b = {'x': 2} > > c = b.copy() > c.update(a) > It seems that everybody misses the part of the OP where he states that conflict resolution shouldn't happen via "one dict wins" but rather with the "or"-operator. > I'd be very reluctant to overload the bitwise-or operator to mean > 'dictionary merging'. It's just not totally obvious to me that this is > what would happen. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From rymg19 at gmail.com Fri Feb 21 17:10:48 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Fri, 21 Feb 2014 10:10:48 -0600 Subject: [Python-ideas] Joining dicts again In-Reply-To: References: Message-ID: ...but that doesn't work if you have non-string keys. http://stackoverflow.com/a/39858/2097780 On Fri, Feb 21, 2014 at 4:15 AM, Chris Angelico wrote: > On Fri, Feb 21, 2014 at 8:40 PM, wrote: > > I know this has been mangled thousand times, but let's do it once again. > > > > Why does Python not have a simple dict joining operator? > > > > From what I read, it seems the biggest concern is: which value to pick > up if both dicts have the same key. > > a = {'x':1} > > b = {'x':2} > > c = a | b > > print(c['x']) # 1 or 2? > > If you can pick one of the dicts to "win", then you can use this notation: > > c = dict(a, **b) > > Anything in b will override a. It's short and a single expression > (unlike "c = a.copy(); c.update(b)"). It's not perfectly clear what's > happening, though, so it may warrant a comment. > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From oscar.j.benjamin at gmail.com Fri Feb 21 17:36:56 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Fri, 21 Feb 2014 16:36:56 +0000 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: References: <20140221102403.GV3684@ando> Message-ID: On 21 February 2014 12:18, Nick Coghlan wrote: > On 21 February 2014 20:24, Steven D'Aprano wrote: >> >> but alas both Nick Coglan and (if I recall correctly) Guido have ruled >> that Python won't get this, so until the Revolution comes, it isn't >> going to happen. > > It's not that it can't happen - it's that someone would need to build > a case of a similar calibre to the one Chris Angelico is currently > putting together for except expressions in PEP 463 :) > > However, it's a *much* bigger challenge in this case, as > itertools.takewhile exists, whereas there's currently no way to do > exception handling as part of a larger expression without major > contortions. I think the two are very similar. PEP 463 proposes that you could do item = stuff['name'] except KeyError: 'default' which you can already do in an ugly sort of way with a helper function: item = catchdefault(lambda: stuff['name'], KeyError, 'default') This proposal is that you could do: isprime = all(n % p for p in primes_seen while p ** 2 <= n) which you can already do in an ugly sort of way with a helper function isprime = all(n % p for p in takewhile(lambda p: p**2 <= n, primes_seen)) Neither syntax proposal gives anything that fundamentally can't be done with helpers and lambda functions. Both syntax ideas are about finding an intuitive and readable way of representing commonly used patterns without too much visual clutter. Similarly in either case you could just refactor the ugly part out into a different function. But splitting code up into different functions also comes with a readability cost. Do you think that the following is any clearer than either of the previous two? isprime = all(n % p for p in take_le_sqrt(primes_seen, n)) Oscar From rosuav at gmail.com Fri Feb 21 18:30:03 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 22 Feb 2014 04:30:03 +1100 Subject: [Python-ideas] Method chaining notation Message-ID: Yeah, I'm insane, opening another theory while I'm busily championing a PEP. But it was while writing up the other PEP that I came up with a possible syntax for this. In Python, as in most languages, method chaining requires the method to return its own object. class Count: def __init__(self): self.n = 0 def inc(self): self.n += 1 return self dracula = Count() dracula.inc().inc().inc() print(dracula.n) It's common in languages like C++ to return *this by reference if there's nothing else useful to return. It's convenient, it doesn't cost anything much, and it allows method chaining. The Python convention, on the other hand, is to return self only if there's a very good reason to, and to return None any time there's mutation that could plausibly return a new object of the same type (compare list.sort() vs sorted()). Method chaining is therefore far less common than it could be, with the result that, often, intermediate objects need to be separately named and assigned to. I pulled up one file from Lib/tkinter (happened to pick filedialog) and saw what's fairly typical of Python GUI code: ... self.midframe = Frame(self.top) self.midframe.pack(expand=YES, fill=BOTH) self.filesbar = Scrollbar(self.midframe) self.filesbar.pack(side=RIGHT, fill=Y) self.files = Listbox(self.midframe, exportselection=0, yscrollcommand=(self.filesbar, 'set')) self.files.pack(side=RIGHT, expand=YES, fill=BOTH) ... Every frame has to be saved away somewhere (incidentally, I don't see why self.midframe rather than just midframe - it's not used outside of __init__). With Tkinter, that's probably necessary (since the parent is part of the construction of the children), but in GTK, widget parenting is done in a more method-chaining-friendly fashion. Compare these examples of PyGTK and Pike GTK: # Cut down version of http://pygtk.org/pygtk2tutorial/examples/helloworld2.py import pygtk pygtk.require('2.0') import gtk def callback(widget, data): print "Hello again - %s was pressed" % data def delete_event(widget, event, data=None): gtk.main_quit() return False window = gtk.Window(gtk.WINDOW_TOPLEVEL) window.set_title("Hello Buttons!") window.connect("delete_event", delete_event) window.set_border_width(10) box1 = gtk.HBox(False, 0) window.add(box1) button1 = gtk.Button("Button 1") button1.connect("clicked", callback, "button 1") box1.pack_start(button1, True, True, 0) button2 = gtk.Button("Button 2") button2.connect("clicked", callback, "button 2") box1.pack_start(button2, True, True, 0) window.show_all() gtk.main() //Pike equivalent of the above: void callback(object widget, string data) {write("Hello again - %s was pressed\n", data);} void delete_event() {exit(0);} int main() { GTK2.setup_gtk(); object button1, button2; GTK2.Window(GTK2.WINDOW_TOPLEVEL) ->set_title("Hello Buttons!") ->set_border_width(10) ->add(GTK2.Hbox(0,0) ->pack_start(button1 = GTK2.Button("Button 1"), 1, 1, 0) ->pack_start(button2 = GTK2.Button("Button 2"), 1, 1, 0) ) ->show_all() ->signal_connect("delete_event", delete_event); button1->signal_connect("clicked", callback, "button 1"); button2->signal_connect("clicked", callback, "button 2"); return -1; } Note that in the Pike version, I capture the button objects, but not the Hbox. There's no name ever given to that box. I have to capture the buttons, because signal_connect doesn't return the object (it returns a signal ID). The more complicated the window layout, the more noticeable this is: The structure of code using chained methods mirrors the structure of the window with its widgets containing widgets; but the structure of the Python equivalent is strictly linear. So here's the proposal. Introduce a new operator to Python, just like the dot operator but behaving differently when it returns a bound method. We can possibly use ->, or maybe create a new operator that currently makes no sense, like .. or .> or something. Its semantics would be: 1) Look up the attribute following it on the object, exactly as per the current . operator 2) If the result is not a function, return it, exactly as per current. 3) If it is a function, though, return a wrapper which, when called, calls the inner function and then returns self. This can be done with an external wrapper, so it might be possible to do this with MacroPy. It absolutely must be a compact notation, though. This probably wouldn't interact at all with __getattr__ (because the attribute has to already exist for this to work), and definitely not with __setattr__ or __delattr__ (mutations aren't affected). How it interacts with __getattribute__ I'm not sure; whether it adds the wrapper around any returned functions or applies only to something that's looked up "the normal way" can be decided by ease of implementation. Supposing this were done, using the -> token that currently is used for annotations as part of 'def'. Here's how the PyGTK code would look: import pygtk pygtk.require('2.0') import gtk def callback(widget, data): print "Hello again - %s was pressed" % data def delete_event(widget, event, data=None): gtk.main_quit() return False window = (gtk.Window(gtk.WINDOW_TOPLEVEL) ->set_title("Hello Buttons!") ->connect("delete_event", delete_event) ->set_border_width(10) ->add(gtk.HBox(False, 0) ->pack_start( gtk.Button("Button 1")->connect("clicked", callback, "button 1"), True, True, 0) ->pack_start( gtk.Button("Button 1")->connect("clicked", callback, "button 1"), True, True, 0) ) ->show_all() ) gtk.main() Again, the structure of the code would match the structure of the window. Unlike the Pike version, this one can even connect signals as part of the method chaining. Effectively, x->y would be equivalent to chain(x.y): def chain(func): def chainable(self, *args, **kwargs): func(self, *args, **kwargs) return self return chainable Could be useful in a variety of contexts. Thoughts? ChrisA From liam.marsh.home at gmail.com Fri Feb 21 18:37:58 2014 From: liam.marsh.home at gmail.com (Liam Marsh) Date: Fri, 21 Feb 2014 18:37:58 +0100 Subject: [Python-ideas] is that expensive? Message-ID: Hello everyone, is it possible to create(or tell me its name) a command to evaluate compute length of a command/procedure? (in number of: -processor operations(for 32 and 64 bit processors) -RAM read and write operations -hard disk write and read operations -eventual graphic operations ) this may be difficult, but it ables users to optimise their programs. -------------- next part -------------- An HTML attachment was scrubbed... URL: From denis.spir at gmail.com Fri Feb 21 18:42:41 2014 From: denis.spir at gmail.com (spir) Date: Fri, 21 Feb 2014 18:42:41 +0100 Subject: [Python-ideas] Raise exception if (not) true In-Reply-To: References: <53063B70.7090608@mrabarnett.plus.com> <53066949.2020105@gmail.com> <20140220212830.GI3684@ando> <5306B899.4000905@gmail.com> Message-ID: <53079011.9030606@gmail.com> On 02/21/2014 07:11 AM, Andrew Barnert wrote: > On Feb 20, 2014, at 18:23, spir wrote: > >> On 02/20/2014 10:28 PM, Steven D'Aprano wrote: >>> On Thu, Feb 20, 2014 at 09:44:57PM +0100, spir wrote: >>> >>>>> But I would like to be able to add an error type to assertions (in addition >>>>> to the optional message). This is particularly useful for people (like me) >>>>> who systematically check func inputs (for client debugging comfort), using >>>>> assert's. >>> Then your code is systematically broken, and badly so. >> >> Why do you speak _that_ negatively? (or should I say: _that_ violently?) And this, maybe not _that_ logically? Do you own the (copy)rights on proper usage of assert's? >> >>> All anyone needs >>> to do to disable your checking is pass -O to the Python interpreter. >>> >>> assert is not a short-cut for lazy programmers to avoid having to write >>> an explicit "if cond: raise SomethingAppropriate(message)". Assertions >>> have specific uses. You should read this post I made last November: >>> >>> https://mail.python.org/pipermail/python-list/2013-November/660401.html >> >> I think you are wrong, at least in this very case. Assert's for me are a debugging tool (or maybe more generally a tool for improving reliability). Such checks help finding errors and correcting them (thank to hopefully clear error messages), with a higher chance these corrections happen before "too late", meaning before users pay for our bugs. What else is the proper usage of assertions? >> >> If not checked on function input, either with assert's or an if-raise combination, execution will break anyway, just later (later in time, and slightly further in code), because some input variable's value or type is wrong. Assert's placed that way are not strictly necessary (it's like duck typing: you don't need to check), instead they're a helpful tool for all users of your "service". >> >> def average (numbers): >> n = len(numbers) >> assert n != 0, "Cannot compute average of 'zero number'.", ValueError >> return sum(numbers) / n > > I think you're actually suffering from the confusion you accused Steven of. His post explains the difference between internal preconditions and external value checks. Then difference has nothing to do with the form of the function, but with how it's used. > > If average is only called by your own code with your own values, so you know it can never be called with an empty list unless there's a bug somewhere, then you're asserting a precondition, which is exactly what asserts are for--but in that case this should be an AssertionError, not a ValueError. > > If average is part of an external API, or is called with user data, so you're testing for something that could fail because of user error rather than a bug in your code, then this is a perfect case for a ValueError--but it's not an assertion, it's an error check. > > The fact that assertions happen to work by exception handling doesn't mean you should ignore the difference between them. side-note: from wikipedia [https://en.wikipedia.org/wiki/Assertion_%28software_development%29]: Assertions during the development cycle During the development cycle, the programmer will typically run the program with assertions enabled. When an assertion failure occurs, the programmer is immediately notified of the problem. Many assertion implementations will also halt the program's execution: this is useful, since if the program continued to run after an assertion violation occurred, it might corrupt its state and make the cause of the problem more difficult to locate. Using the information provided by the assertion failure (such as the location of the failure and perhaps a stack trace, or even the full program state if the environment supports core dumps or if the program is running in a debugger), the programmer can usually fix the problem. Thus assertions provide a very powerful tool in debugging. This is what I do. Note that whether assertion instructions were written by the client developper (who gets assertion errors) or by the author of a tool used for this project, does not change anything: in both cases, the client is properly notified of his/her errors. In fact, there is no other border between internal and external components (modules, libs...) than a purely practical one. Semantically, it makes no sense (for me). The very same component can be written for a given project and distributed together with the rest, or written by someone else and installed apart. What is important is _coherence_ (or meaningfulness); this is what assertions check. d From rosuav at gmail.com Fri Feb 21 18:52:24 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 22 Feb 2014 04:52:24 +1100 Subject: [Python-ideas] is that expensive? In-Reply-To: References: Message-ID: On Sat, Feb 22, 2014 at 4:37 AM, Liam Marsh wrote: > Hello everyone, > is it possible to create(or tell me its name) a command to evaluate compute > length of a command/procedure? > (in number of: -processor operations(for 32 and 64 bit > processors) > -RAM read and write operations > -hard disk write and read operations > -eventual graphic operations > ) > this may be difficult, but it ables users to optimise their programs. Here's a first-cut timing routine: def how_fast_is_this(func): return "Fast enough." Trust me, that's accurate enough for most cases. For anything else, you need to be timing it in your actual code. The forms of measurement you're asking for make no sense for most Python functions. The best you could do would probably be to look at the size of the compiled byte-code, but that's not going to be particularly accurate anyway. No, the only way to profile your code is to put timing points in your actual code. You may want to try the timeit module for some help with that, but the simplest is to just pepper your code with calls to time.time(). I hope there never is a function for counting processor instructions required for a particular Python function. Apart from being nearly impossible to calculate, it'd be almost never correctly used. People would warp their code around using "the one with the smaller number", when high level languages these days should be written primarily with a view to being readable by a human. Make your code look right and act right, and worry about how fast it is only when you have evidence that it really isn't fast enough. Incidentally, the newer Python versions will tend to be faster than older ones, because the developers of Python itself care about performance. A bit of time spent optimizing CPython will improve execution time of every Python script, but a bit of time spent optimizing your one script improves only that one script. For further help with optimizing scripts, ask on python-list. We can help with back-of-the-envelope calculations (if you're concerned that your server can't handle X network requests a second, first ascertain whether your server's network connection can feed it that many a second - that exact question came up on the list a few months ago), and also with tips and tricks when you come to the optimizing itself. And who knows, maybe we can save you a huge amount of time... programmer time, which is usually more expensive than processor time :) ChrisA From __peter__ at web.de Fri Feb 21 19:00:22 2014 From: __peter__ at web.de (Peter Otten) Date: Fri, 21 Feb 2014 19:00:22 +0100 Subject: [Python-ideas] Function to return first(or last) true value from list References: <20140218230110.GT4519@ando> <20140220220040.GL3684@ando> Message-ID: Oscar Benjamin wrote: > But there are also cases where that implicit behaviour is not desired. > I would rather have to explicitly return when that's what I want so > that the control flow is very clear. > > For example when you use csv.DictReader and don't supply the > fieldnames argument you are saying that you want to read a csv file > with one header line containing column labels and zero or more data > lines. To me a fully empty file (with no header line) is invalid but > csv.DictReader will accept it as a csv file with zero rows. I would > prefer an error in this case since it would only happen in my usage if > an error had occurred somewhere else. I think we constantly have to deal with libraries that do almost but not exactly what we want them to do. If you look at the code it is clear that the author made a conscious design decision @property def fieldnames(self): if self._fieldnames is None: try: self._fieldnames = next(self.reader) except StopIteration: pass self.line_num = self.reader.line_num return self._fieldnames totally unrelated to for loops catching StopIterations. You can of course easily revert that decision: reader = csv.DictReader(...) if reader.fieldnames is None: raise EmptyCsvError > When I use sequences and write first = seq[0] I'm deliberately > asserting that seq is non-empty. > One of the examples you linked to shows exactly my own practice of > marking a next call with a comment: > > def __next__(self): > while self.currkey == self.tgtkey: > self.currvalue = next(self.it) # Exit on StopIteration > self.currkey = self.keyfunc(self.currvalue) > self.tgtkey = self.currkey > return (self.currkey, self._grouper(self.tgtkey)) > IMO if you want that behaviour then you should mark it to show that > you thought about it and otherwise I'll treat any bare next with > suspicion. Similar comments are possible for obj[...]: def heapreplace(heap, item): ... returnitem = heap[0] # raises appropriate IndexError if heap is empty ... I fail to see the fundamental difference between next(...) and sequence[...]. There are edge cases you have to consider, and you add comments where you expect them to help your readers to understand your intentions. From yselivanov.ml at gmail.com Fri Feb 21 21:08:39 2014 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Fri, 21 Feb 2014 15:08:39 -0500 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: Message-ID: <5307B247.8030504@gmail.com> I suggest you to take a look at cascades in Dart language And this article: http://en.wikipedia.org/wiki/Method_cascading Yury On 2/21/2014, 12:30 PM, Chris Angelico wrote: > Yeah, I'm insane, opening another theory while I'm busily championing > a PEP. But it was while writing up the other PEP that I came up with a > possible syntax for this. > > In Python, as in most languages, method chaining requires the method > to return its own object. > > class Count: > def __init__(self): self.n = 0 > def inc(self): > self.n += 1 > return self > > dracula = Count() > dracula.inc().inc().inc() > print(dracula.n) > > It's common in languages like C++ to return *this by reference if > there's nothing else useful to return. It's convenient, it doesn't > cost anything much, and it allows method chaining. The Python > convention, on the other hand, is to return self only if there's a > very good reason to, and to return None any time there's mutation that > could plausibly return a new object of the same type (compare > list.sort() vs sorted()). Method chaining is therefore far less common > than it could be, with the result that, often, intermediate objects > need to be separately named and assigned to. I pulled up one file from > Lib/tkinter (happened to pick filedialog) and saw what's fairly > typical of Python GUI code: > > ... > self.midframe = Frame(self.top) > self.midframe.pack(expand=YES, fill=BOTH) > > self.filesbar = Scrollbar(self.midframe) > self.filesbar.pack(side=RIGHT, fill=Y) > self.files = Listbox(self.midframe, exportselection=0, > yscrollcommand=(self.filesbar, 'set')) > self.files.pack(side=RIGHT, expand=YES, fill=BOTH) > ... > > Every frame has to be saved away somewhere (incidentally, I don't see > why self.midframe rather than just midframe - it's not used outside of > __init__). With Tkinter, that's probably necessary (since the parent > is part of the construction of the children), but in GTK, widget > parenting is done in a more method-chaining-friendly fashion. Compare > these examples of PyGTK and Pike GTK: > > # Cut down version of http://pygtk.org/pygtk2tutorial/examples/helloworld2.py > import pygtk > pygtk.require('2.0') > import gtk > > def callback(widget, data): > print "Hello again - %s was pressed" % data > > def delete_event(widget, event, data=None): > gtk.main_quit() > return False > > window = gtk.Window(gtk.WINDOW_TOPLEVEL) > window.set_title("Hello Buttons!") > window.connect("delete_event", delete_event) > window.set_border_width(10) > box1 = gtk.HBox(False, 0) > window.add(box1) > button1 = gtk.Button("Button 1") > button1.connect("clicked", callback, "button 1") > box1.pack_start(button1, True, True, 0) > button2 = gtk.Button("Button 2") > button2.connect("clicked", callback, "button 2") > box1.pack_start(button2, True, True, 0) > window.show_all() > > gtk.main() > > //Pike equivalent of the above: > void callback(object widget, string data) {write("Hello again - %s was > pressed\n", data);} > void delete_event() {exit(0);} > > int main() > { > GTK2.setup_gtk(); > object button1, button2; > GTK2.Window(GTK2.WINDOW_TOPLEVEL) > ->set_title("Hello Buttons!") > ->set_border_width(10) > ->add(GTK2.Hbox(0,0) > ->pack_start(button1 = GTK2.Button("Button 1"), 1, 1, 0) > ->pack_start(button2 = GTK2.Button("Button 2"), 1, 1, 0) > ) > ->show_all() > ->signal_connect("delete_event", delete_event); > button1->signal_connect("clicked", callback, "button 1"); > button2->signal_connect("clicked", callback, "button 2"); > return -1; > } > > > Note that in the Pike version, I capture the button objects, but not > the Hbox. There's no name ever given to that box. I have to capture > the buttons, because signal_connect doesn't return the object (it > returns a signal ID). The more complicated the window layout, the more > noticeable this is: The structure of code using chained methods > mirrors the structure of the window with its widgets containing > widgets; but the structure of the Python equivalent is strictly > linear. > > So here's the proposal. Introduce a new operator to Python, just like > the dot operator but behaving differently when it returns a bound > method. We can possibly use ->, or maybe create a new operator that > currently makes no sense, like .. or .> or something. Its semantics > would be: > > 1) Look up the attribute following it on the object, exactly as per > the current . operator > 2) If the result is not a function, return it, exactly as per current. > 3) If it is a function, though, return a wrapper which, when called, > calls the inner function and then returns self. > > This can be done with an external wrapper, so it might be possible to > do this with MacroPy. It absolutely must be a compact notation, > though. > > This probably wouldn't interact at all with __getattr__ (because the > attribute has to already exist for this to work), and definitely not > with __setattr__ or __delattr__ (mutations aren't affected). How it > interacts with __getattribute__ I'm not sure; whether it adds the > wrapper around any returned functions or applies only to something > that's looked up "the normal way" can be decided by ease of > implementation. > > Supposing this were done, using the -> token that currently is used > for annotations as part of 'def'. Here's how the PyGTK code would > look: > > import pygtk > pygtk.require('2.0') > import gtk > > def callback(widget, data): > print "Hello again - %s was pressed" % data > > def delete_event(widget, event, data=None): > gtk.main_quit() > return False > > window = (gtk.Window(gtk.WINDOW_TOPLEVEL) > ->set_title("Hello Buttons!") > ->connect("delete_event", delete_event) > ->set_border_width(10) > ->add(gtk.HBox(False, 0) > ->pack_start( > gtk.Button("Button 1")->connect("clicked", callback, "button 1"), > True, True, 0) > ->pack_start( > gtk.Button("Button 1")->connect("clicked", callback, "button 1"), > True, True, 0) > ) > ->show_all() > ) > > gtk.main() > > > Again, the structure of the code would match the structure of the > window. Unlike the Pike version, this one can even connect signals as > part of the method chaining. > > Effectively, x->y would be equivalent to chain(x.y): > > def chain(func): > def chainable(self, *args, **kwargs): > func(self, *args, **kwargs) > return self > return chainable > > Could be useful in a variety of contexts. > > Thoughts? > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From abarnert at yahoo.com Fri Feb 21 21:34:51 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 21 Feb 2014 12:34:51 -0800 (PST) Subject: [Python-ideas] The Return of Implicit Lambda (Re: Allowing breaks in generator expressions by overloading the while keyword) In-Reply-To: References: <20140221102403.GV3684@ando> Message-ID: <1393014891.79562.YahooMailNeo@web181003.mail.ne1.yahoo.com> From: Nick Coghlan Sent: Friday, February 21, 2014 4:18 AM > On 21 February 2014 20:24, Steven D'Aprano > wrote: >> >> but alas both Nick Coglan and (if I recall correctly) Guido have ruled >> that Python won't get this, so until the Revolution comes, it isn't >> going to happen. > > It's not that it can't happen - it's that someone would need to > build > a case of a similar calibre to the one Chris Angelico is currently > putting together for except expressions in PEP 463 :) > > However, it's a *much* bigger challenge in this case, as > itertools.takewhile exists, whereas there's currently no way to do > exception handling as part of a larger expression without major > contortions. > > Re-reading Andrew's post at > http://stupidpythonideas.blogspot.com.au/2013/07/syntactic-takewhile.html, > I'm actually more intrigued by Haskell's shorthand for lambda > functions that consist of a single binary operator. > > Consider: > > ? ? isprime = all(n % p for p in takewhile((lambda p: p**2 < n), > primes_seen)) > > Is there are a nicer way to write that? The Haskell equivalent is: > > ? (< n) . (** 2) >? > That's not very readable to most Python programmers That's really not a shorthand for lambda, it's a combination of other features that make lambdas often unnecessary. I think the C++ boost::lambda example is a better fit: ? ? _1 ** 2 < n > but what if you > could write something like: > > ? ? isprime = all(n % p for p in takewhile((: ? ** 2 < n), primes_seen)) I like this. The ? is a lot better than the?_ or _1 I suggested (especially to people who do a lot of SQL programming in Python and already use ? as an inline parameter), and using a colon makes it explicit without adding too much visual noise. The only concern is whether using up one of the handful of as-yet-unused ASCII symbols raises the bar a little too high? > This is somewhat similar to the implicit lambda proposal in > http://legacy.python.org/dev/peps/pep-0312/, but with the following > two essential differences: > > 1. The parentheses would be required (as per generator expressions, > and as is being discussed for except expressions) > 2. By using a "?" token within the implicit lambda, you would create a > lambda that takes a single argument. If there is no such token, then > it would take no arguments. Would it be worth extending this to multiple arguments? I think it would be handy, but I can't think of any way to make it look nearly as good as the 0- and 1-arg forms. Obviously this should still be a one-arg lambda that just uses its parameter repeatedly: ? ? : ? * x**2 + ? * x + ? Sticking a number on the ? looks terrible: ? ? : ?1 * x**2 + ?2 * x + ?3 Looking at the other DB-API 2.0 param styles, this is hideous because of the way it uses colons twice: ? ? : :1 * x**2 + :2 * x + :3 Something like this wouldn't be too bad, except that it already has a much more useful meaning, set literals: ? ? : {1} * x**2 + {2} * x + {3} So really, the best I can come up with is the boost::lambda version I already didn't like and you already dismissed without comment: ? ? : _1 * x**2 + _2 * x + _3 On the other hand, any expression this complex?even with just one parameter?is already starting to look pretty bad as an implicit lambda, and the downsides of an explicit lambda go down the longer it gets. So maybe the best answer is: ? ? lambda a, b, c: a * x**2 + b * x + c Anyway, we can always start with just 0- and 1-argument forms and come back to expand the idea later, right? > However, the examples in that PEP are no longer useful, as they have > since been addressed by dedicated constructs (conditional expressions > in PEP 308 and context managers in PEP 343). > > Note that I'm not saying this is a good idea. However, I am saying I > think it is an idea that may be worth exploring further to see if it > also addresses at least some of the use cases given in PEP's 403 and > 3150 (especially in combination with PEP 463) by making custom key and > predicate functions easier to define. I'll start digging up some examples; it is hard to predict how nice this will look until we see it in a wide range of realistic code. From antony.lee at berkeley.edu Fri Feb 21 21:47:40 2014 From: antony.lee at berkeley.edu (Antony Lee) Date: Fri, 21 Feb 2014 12:47:40 -0800 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: Message-ID: While method chaining may be useful in general, I found that for GUI code, it is possible (and helps readability, IMO) to define a context manager that maintains a "current" object and allows calling methods on the object (I'm more used to PyQt but I guess it can easily be adapted to PyGTK). Something that allows code like (reusing your example): build = builder(gtk.Window(gtk.WINDOW_TOPLEVEL)) build.calls(("set_title", "Hello Buttons!"), ("connect", "delete_event", delete_event), ("set_border_width", 10)) with build.enter("add", gtk.HBox(False, 0)): build.enter("pack_start", gtk.Button("Button 1"), True, True, 0).call( "connect", "clicked", callback, "button 1") build.enter("pack_start", gtk.Button("Button 2"), True, True, 0).call( "connect", "clicked", callback, "button 2") build.call("show_all") The build object maintains a stack of current objects. "call{,s}" calls one or multiple methods on the topmost object; "enter" calls a method and returns the object added by this method (this requires to hard-code some knowledge about the API of the widget library) and also pushes the object on the stack if used in context manager form. Antony 2014-02-21 9:30 GMT-08:00 Chris Angelico : > Yeah, I'm insane, opening another theory while I'm busily championing > a PEP. But it was while writing up the other PEP that I came up with a > possible syntax for this. > > In Python, as in most languages, method chaining requires the method > to return its own object. > > class Count: > def __init__(self): self.n = 0 > def inc(self): > self.n += 1 > return self > > dracula = Count() > dracula.inc().inc().inc() > print(dracula.n) > > It's common in languages like C++ to return *this by reference if > there's nothing else useful to return. It's convenient, it doesn't > cost anything much, and it allows method chaining. The Python > convention, on the other hand, is to return self only if there's a > very good reason to, and to return None any time there's mutation that > could plausibly return a new object of the same type (compare > list.sort() vs sorted()). Method chaining is therefore far less common > than it could be, with the result that, often, intermediate objects > need to be separately named and assigned to. I pulled up one file from > Lib/tkinter (happened to pick filedialog) and saw what's fairly > typical of Python GUI code: > > ... > self.midframe = Frame(self.top) > self.midframe.pack(expand=YES, fill=BOTH) > > self.filesbar = Scrollbar(self.midframe) > self.filesbar.pack(side=RIGHT, fill=Y) > self.files = Listbox(self.midframe, exportselection=0, > yscrollcommand=(self.filesbar, 'set')) > self.files.pack(side=RIGHT, expand=YES, fill=BOTH) > ... > > Every frame has to be saved away somewhere (incidentally, I don't see > why self.midframe rather than just midframe - it's not used outside of > __init__). With Tkinter, that's probably necessary (since the parent > is part of the construction of the children), but in GTK, widget > parenting is done in a more method-chaining-friendly fashion. Compare > these examples of PyGTK and Pike GTK: > > # Cut down version of > http://pygtk.org/pygtk2tutorial/examples/helloworld2.py > import pygtk > pygtk.require('2.0') > import gtk > > def callback(widget, data): > print "Hello again - %s was pressed" % data > > def delete_event(widget, event, data=None): > gtk.main_quit() > return False > > window = gtk.Window(gtk.WINDOW_TOPLEVEL) > window.set_title("Hello Buttons!") > window.connect("delete_event", delete_event) > window.set_border_width(10) > box1 = gtk.HBox(False, 0) > window.add(box1) > button1 = gtk.Button("Button 1") > button1.connect("clicked", callback, "button 1") > box1.pack_start(button1, True, True, 0) > button2 = gtk.Button("Button 2") > button2.connect("clicked", callback, "button 2") > box1.pack_start(button2, True, True, 0) > window.show_all() > > gtk.main() > > //Pike equivalent of the above: > void callback(object widget, string data) {write("Hello again - %s was > pressed\n", data);} > void delete_event() {exit(0);} > > int main() > { > GTK2.setup_gtk(); > object button1, button2; > GTK2.Window(GTK2.WINDOW_TOPLEVEL) > ->set_title("Hello Buttons!") > ->set_border_width(10) > ->add(GTK2.Hbox(0,0) > ->pack_start(button1 = GTK2.Button("Button 1"), 1, 1, 0) > ->pack_start(button2 = GTK2.Button("Button 2"), 1, 1, 0) > ) > ->show_all() > ->signal_connect("delete_event", delete_event); > button1->signal_connect("clicked", callback, "button 1"); > button2->signal_connect("clicked", callback, "button 2"); > return -1; > } > > > Note that in the Pike version, I capture the button objects, but not > the Hbox. There's no name ever given to that box. I have to capture > the buttons, because signal_connect doesn't return the object (it > returns a signal ID). The more complicated the window layout, the more > noticeable this is: The structure of code using chained methods > mirrors the structure of the window with its widgets containing > widgets; but the structure of the Python equivalent is strictly > linear. > > So here's the proposal. Introduce a new operator to Python, just like > the dot operator but behaving differently when it returns a bound > method. We can possibly use ->, or maybe create a new operator that > currently makes no sense, like .. or .> or something. Its semantics > would be: > > 1) Look up the attribute following it on the object, exactly as per > the current . operator > 2) If the result is not a function, return it, exactly as per current. > 3) If it is a function, though, return a wrapper which, when called, > calls the inner function and then returns self. > > This can be done with an external wrapper, so it might be possible to > do this with MacroPy. It absolutely must be a compact notation, > though. > > This probably wouldn't interact at all with __getattr__ (because the > attribute has to already exist for this to work), and definitely not > with __setattr__ or __delattr__ (mutations aren't affected). How it > interacts with __getattribute__ I'm not sure; whether it adds the > wrapper around any returned functions or applies only to something > that's looked up "the normal way" can be decided by ease of > implementation. > > Supposing this were done, using the -> token that currently is used > for annotations as part of 'def'. Here's how the PyGTK code would > look: > > import pygtk > pygtk.require('2.0') > import gtk > > def callback(widget, data): > print "Hello again - %s was pressed" % data > > def delete_event(widget, event, data=None): > gtk.main_quit() > return False > > window = (gtk.Window(gtk.WINDOW_TOPLEVEL) > ->set_title("Hello Buttons!") > ->connect("delete_event", delete_event) > ->set_border_width(10) > ->add(gtk.HBox(False, 0) > ->pack_start( > gtk.Button("Button 1")->connect("clicked", callback, "button > 1"), > True, True, 0) > ->pack_start( > gtk.Button("Button 1")->connect("clicked", callback, "button > 1"), > True, True, 0) > ) > ->show_all() > ) > > gtk.main() > > > Again, the structure of the code would match the structure of the > window. Unlike the Pike version, this one can even connect signals as > part of the method chaining. > > Effectively, x->y would be equivalent to chain(x.y): > > def chain(func): > def chainable(self, *args, **kwargs): > func(self, *args, **kwargs) > return self > return chainable > > Could be useful in a variety of contexts. > > Thoughts? > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rymg19 at gmail.com Fri Feb 21 22:36:24 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Fri, 21 Feb 2014 15:36:24 -0600 Subject: [Python-ideas] The Return of Implicit Lambda (Re: Allowing breaks in generator expressions by overloading the while keyword) In-Reply-To: <1393014891.79562.YahooMailNeo@web181003.mail.ne1.yahoo.com> References: <20140221102403.GV3684@ando> <1393014891.79562.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: On Fri, Feb 21, 2014 at 2:34 PM, Andrew Barnert wrote: > From: Nick Coghlan > > Sent: Friday, February 21, 2014 4:18 AM > > > > On 21 February 2014 20:24, Steven D'Aprano > > wrote: > >> > >> but alas both Nick Coglan and (if I recall correctly) Guido have ruled > >> that Python won't get this, so until the Revolution comes, it isn't > >> going to happen. > > > > It's not that it can't happen - it's that someone would need to > > build > > a case of a similar calibre to the one Chris Angelico is currently > > putting together for except expressions in PEP 463 :) > > > > However, it's a *much* bigger challenge in this case, as > > itertools.takewhile exists, whereas there's currently no way to do > > exception handling as part of a larger expression without major > > contortions. > > > > Re-reading Andrew's post at > > > http://stupidpythonideas.blogspot.com.au/2013/07/syntactic-takewhile.html, > > I'm actually more intrigued by Haskell's shorthand for lambda > > functions that consist of a single binary operator. > > > > Consider: > > > > isprime = all(n % p for p in takewhile((lambda p: p**2 < n), > > primes_seen)) > > > > Is there are a nicer way to write that? The Haskell equivalent is: > > > > (< n) . (** 2) > > > > > That's not very readable to most Python programmers > > That's really not a shorthand for lambda, it's a combination of other > features that make lambdas often unnecessary. I think the C++ boost::lambda > example is a better fit: > > _1 ** 2 < n > > > but what if you > > > could write something like: > > > > isprime = all(n % p for p in takewhile((: ? ** 2 < n), primes_seen)) > > I like this. The ? is a lot better than the _ or _1 I suggested > (especially to people who do a lot of SQL programming in Python and already > use ? as an inline parameter), and using a colon makes it explicit without > adding too much visual noise. The only concern is whether using up one of > the handful of as-yet-unused ASCII symbols raises the bar a little too high... > > > This is somewhat similar to the implicit lambda proposal in > > > http://legacy.python.org/dev/peps/pep-0312/, but with the following > > two essential differences: > > > > 1. The parentheses would be required (as per generator expressions, > > and as is being discussed for except expressions) > > 2. By using a "?" token within the implicit lambda, you would create a > > lambda that takes a single argument. If there is no such token, then > > it would take no arguments. > > Would it be worth extending this to multiple arguments? I think it would > be handy, but I can't think of any way to make it look nearly as good as > the 0- and 1-arg forms. > > Obviously this should still be a one-arg lambda that just uses its > parameter repeatedly: > > : ? * x**2 + ? * x + ? > > Sticking a number on the ? looks terrible: > > : ?1 * x**2 + ?2 * x + ?3 > > Looking at the other DB-API 2.0 param styles, this is hideous because of > the way it uses colons twice: > > > : :1 * x**2 + :2 * x + :3 > > Something like this wouldn't be too bad, except that it already has a much > more useful meaning, set literals: > > : {1} * x**2 + {2} * x + {3} > > So really, the best I can come up with is the boost::lambda version I > already didn't like and you already dismissed without comment: > > : _1 * x**2 + _2 * x + _3 > > On the other hand, any expression this complex--even with just one > parameter--is already starting to look pretty bad as an implicit lambda, and > the downsides of an explicit lambda go down the longer it gets. So maybe > the best answer is: > > lambda a, b, c: a * x**2 + b * x + c > > Anyway, we can always start with just 0- and 1-argument forms and come > back to expand the idea later, right? > > > However, the examples in that PEP are no longer useful, as they have > > since been addressed by dedicated constructs (conditional expressions > > in PEP 308 and context managers in PEP 343). > > > > Note that I'm not saying this is a good idea. However, I am saying I > > think it is an idea that may be worth exploring further to see if it > > also addresses at least some of the use cases given in PEP's 403 and > > 3150 (especially in combination with PEP 463) by making custom key and > > predicate functions easier to define. > > I'll start digging up some examples; it is hard to predict how nice this > will look until we see it in a wide range of realistic code. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ +1 for the question mark. However, there's the problem of multi-argument lambdas. The ?(number) looks to much like Perl. What about something like ? being an array of arguments? -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Fri Feb 21 22:37:07 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 21 Feb 2014 16:37:07 -0500 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: Message-ID: On 2/21/2014 12:30 PM, Chris Angelico wrote: > It's common in languages like C++ to return *this by reference if > there's nothing else useful to return. It's convenient, it doesn't > cost anything much, and it allows method chaining. The Python > convention, on the other hand, is to return self only if there's a > very good reason to, Off the top of my head, I cannot think of any methods that return self. (If there are some, someone please remind or inform me.) > and to return None any time there's mutation that > could plausibly return a new object of the same type (compare > list.sort() vs sorted()). The rule for mutation methods is to return None unless there is something *other than* self to return, like list.pop and similar remove (mutate) and return methods. List.sort and list.reverse returned None from the beginning, long before sorted and reversed were added. -- Terry Jan Reedy From rymg19 at gmail.com Fri Feb 21 22:40:27 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Fri, 21 Feb 2014 15:40:27 -0600 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: Message-ID: On Fri, Feb 21, 2014 at 11:30 AM, Chris Angelico wrote: > Yeah, I'm insane, opening another theory while I'm busily championing > a PEP. But it was while writing up the other PEP that I came up with a > possible syntax for this. > > In Python, as in most languages, method chaining requires the method > to return its own object. > > class Count: > def __init__(self): self.n = 0 > def inc(self): > self.n += 1 > return self > > dracula = Count() > dracula.inc().inc().inc() > print(dracula.n) > > It's common in languages like C++ to return *this by reference if > there's nothing else useful to return. It's convenient, it doesn't > cost anything much, and it allows method chaining. The Python > convention, on the other hand, is to return self only if there's a > very good reason to, and to return None any time there's mutation that > could plausibly return a new object of the same type (compare > list.sort() vs sorted()). Method chaining is therefore far less common > than it could be, with the result that, often, intermediate objects > need to be separately named and assigned to. I pulled up one file from > Lib/tkinter (happened to pick filedialog) and saw what's fairly > typical of Python GUI code: > > ... > self.midframe = Frame(self.top) > self.midframe.pack(expand=YES, fill=BOTH) > > self.filesbar = Scrollbar(self.midframe) > self.filesbar.pack(side=RIGHT, fill=Y) > self.files = Listbox(self.midframe, exportselection=0, > yscrollcommand=(self.filesbar, 'set')) > self.files.pack(side=RIGHT, expand=YES, fill=BOTH) > ... > > Ugh! > Every frame has to be saved away somewhere (incidentally, I don't see > why self.midframe rather than just midframe - it's not used outside of > __init__). With Tkinter, that's probably necessary (since the parent > is part of the construction of the children), but in GTK, widget > parenting is done in a more method-chaining-friendly fashion. Compare > these examples of PyGTK and Pike GTK: > > # Cut down version of > http://pygtk.org/pygtk2tutorial/examples/helloworld2.py > import pygtk > pygtk.require('2.0') > import gtk > > def callback(widget, data): > print "Hello again - %s was pressed" % data > > def delete_event(widget, event, data=None): > gtk.main_quit() > return False > > window = gtk.Window(gtk.WINDOW_TOPLEVEL) > window.set_title("Hello Buttons!") > window.connect("delete_event", delete_event) > window.set_border_width(10) > box1 = gtk.HBox(False, 0) > window.add(box1) > button1 = gtk.Button("Button 1") > button1.connect("clicked", callback, "button 1") > box1.pack_start(button1, True, True, 0) > button2 = gtk.Button("Button 2") > button2.connect("clicked", callback, "button 2") > box1.pack_start(button2, True, True, 0) > window.show_all() > > gtk.main() > > //Pike equivalent of the above: > void callback(object widget, string data) {write("Hello again - %s was > pressed\n", data);} > void delete_event() {exit(0);} > > int main() > { > GTK2.setup_gtk(); > object button1, button2; > GTK2.Window(GTK2.WINDOW_TOPLEVEL) > ->set_title("Hello Buttons!") > ->set_border_width(10) > ->add(GTK2.Hbox(0,0) > ->pack_start(button1 = GTK2.Button("Button 1"), 1, 1, 0) > ->pack_start(button2 = GTK2.Button("Button 2"), 1, 1, 0) > ) > ->show_all() > ->signal_connect("delete_event", delete_event); > button1->signal_connect("clicked", callback, "button 1"); > button2->signal_connect("clicked", callback, "button 2"); > return -1; > } > > > Note that in the Pike version, I capture the button objects, but not > the Hbox. There's no name ever given to that box. I have to capture > the buttons, because signal_connect doesn't return the object (it > returns a signal ID). The more complicated the window layout, the more > noticeable this is: The structure of code using chained methods > mirrors the structure of the window with its widgets containing > widgets; but the structure of the Python equivalent is strictly > linear. > > So here's the proposal. Introduce a new operator to Python, just like > the dot operator but behaving differently when it returns a bound > method. We can possibly use ->, or maybe create a new operator that > currently makes no sense, like .. or .> or something. Its semantics > would be: > > 1) Look up the attribute following it on the object, exactly as per > the current . operator > 2) If the result is not a function, return it, exactly as per current. > 3) If it is a function, though, return a wrapper which, when called, > calls the inner function and then returns self. > > This can be done with an external wrapper, so it might be possible to > do this with MacroPy. It absolutely must be a compact notation, > though. > > This probably wouldn't interact at all with __getattr__ (because the > attribute has to already exist for this to work), and definitely not > with __setattr__ or __delattr__ (mutations aren't affected). How it > interacts with __getattribute__ I'm not sure; whether it adds the > wrapper around any returned functions or applies only to something > that's looked up "the normal way" can be decided by ease of > implementation. > > Supposing this were done, using the -> token that currently is used > for annotations as part of 'def'. Here's how the PyGTK code would > look: > > import pygtk > pygtk.require('2.0') > import gtk > > def callback(widget, data): > print "Hello again - %s was pressed" % data > > def delete_event(widget, event, data=None): > gtk.main_quit() > return False > > window = (gtk.Window(gtk.WINDOW_TOPLEVEL) > ->set_title("Hello Buttons!") > ->connect("delete_event", delete_event) > ->set_border_width(10) > ->add(gtk.HBox(False, 0) > ->pack_start( > gtk.Button("Button 1")->connect("clicked", callback, "button > 1"), > True, True, 0) > ->pack_start( > gtk.Button("Button 1")->connect("clicked", callback, "button > 1"), > True, True, 0) > ) > ->show_all() > ) > > gtk.main() > > > Again, the structure of the code would match the structure of the > window. Unlike the Pike version, this one can even connect signals as > part of the method chaining. > > Effectively, x->y would be equivalent to chain(x.y): > > def chain(func): > def chainable(self, *args, **kwargs): > func(self, *args, **kwargs) > return self > return chainable > > Could be useful in a variety of contexts. > > Thoughts? > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > Seems weird to me, since I'm used to -> being for C++ pointers. I prefer "..", because it gives the impression that it's something additional. Either that, or I've used Lua too much. -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Fri Feb 21 22:57:26 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 21 Feb 2014 13:57:26 -0800 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: Message-ID: On Fri, Feb 21, 2014 at 1:37 PM, Terry Reedy wrote: > Off the top of my head, I cannot think of any methods that return self. > (If there are some, someone please remind or inform me.) > That's because I don't like this pattern. :-) But there's some code in the stdlib that uses it (IIRC Eric Raymond was a fan), and I'm sure there's lots of 3rd party Python code too. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From denis.spir at gmail.com Fri Feb 21 23:00:54 2014 From: denis.spir at gmail.com (spir) Date: Fri, 21 Feb 2014 23:00:54 +0100 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: Message-ID: <5307CC96.9020200@gmail.com> On 02/21/2014 10:37 PM, Terry Reedy wrote: > On 2/21/2014 12:30 PM, Chris Angelico wrote: > >> It's common in languages like C++ to return *this by reference if >> there's nothing else useful to return. It's convenient, it doesn't >> cost anything much, and it allows method chaining. The Python >> convention, on the other hand, is to return self only if there's a >> very good reason to, > > Off the top of my head, I cannot think of any methods that return self. (If > there are some, someone please remind or inform me.) It is (in my experience) a common practice, precisely to allow method chaining, in diverse libs. The most common cases happen at object construction, where chaining adds or changes diverse object properties. This is also at times combined with overloading of language features. In the following, from a parsing lib, the init method returns its object, and __call__ is overriden to a method that sets an match actions on the pattern: symbol_def = Compose(id, ":=", expr)(put_symbol) [This defines a patten for symbol defs (read: assignment of a new var); when the pattern matches, the symbol is put in a symbol table.] However, this is probably only ok for such libs that developpers are forced to study intensely before being able to use them efficiently. Otherwise, in itself the code is pretty unreadable I guess. Also, I don't find the idea of having a builtin construct for such hacks a good idea. Libs for which this may be practicle can return self --end of the story. d From abarnert at yahoo.com Fri Feb 21 23:02:56 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 21 Feb 2014 14:02:56 -0800 (PST) Subject: [Python-ideas] The Return of Implicit Lambda (Re: Allowing breaks in generator expressions by overloading the while keyword) In-Reply-To: References: <20140221102403.GV3684@ando> <1393014891.79562.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: <1393020176.19093.YahooMailNeo@web181002.mail.ne1.yahoo.com> From: Ryan Gonzalez Sent: Friday, February 21, 2014 1:36 PM >On Fri, Feb 21, 2014 at 2:34 PM, Andrew Barnert wrote: > >From: Nick Coghlan >> >>Sent: Friday, February 21, 2014 4:18 AM >> >> >>> On 21 February 2014 20:24, Steven D'Aprano >>> wrote: >>>> >>>> ?but alas both Nick Coglan and (if I recall correctly) Guido have ruled >>>> ?that Python won't get this, so until the Revolution comes, it isn't >>>> ?going to happen. >>> >>> It's not that it can't happen - it's that someone would need to >>> build >>> a case of a similar calibre to the one Chris Angelico is currently >>> putting together for except expressions in PEP 463 :) >>> >>> However, it's a *much* bigger challenge in this case, as >>> itertools.takewhile exists, whereas there's currently no way to do >>> exception handling as part of a larger expression without major >>> contortions. >>> >>> Re-reading Andrew's post at >>> http://stupidpythonideas.blogspot.com.au/2013/07/syntactic-takewhile.html, >>> I'm actually more intrigued by Haskell's shorthand for lambda >>> functions that consist of a single binary operator. >>> >>> Consider: >>> >>> ? ? isprime = all(n % p for p in takewhile((lambda p: p**2 < n), >>> primes_seen)) >>> >>> Is there are a nicer way to write that? The Haskell equivalent is: >>> >>> ? ?(< n) . (** 2) >>>? >> >>> That's not very readable to most Python programmers >> >>That's really not a shorthand for lambda, it's a combination of other features that make lambdas often unnecessary. I think the C++ boost::lambda example is a better fit: >> >>? ? _1 ** 2 < n >> >>> but what if you >> >>> could write something like: >>> >>> ? ? isprime = all(n % p for p in takewhile((: ? ** 2 < n), primes_seen)) >> >>I like this. The ? is a lot better than the?_ or _1 I suggested (especially to people who do a lot of SQL programming in Python and already use ? as an inline parameter), and using a colon makes it explicit without adding too much visual noise. The only concern is whether using up one of the handful of as-yet-unused ASCII symbols raises the bar a little too high? >> >>> This is somewhat similar to the implicit lambda proposal in >> >>> http://legacy.python.org/dev/peps/pep-0312/, but with the following >>> two essential differences: >>> >>> 1. The parentheses would be required (as per generator expressions, >>> and as is being discussed for except expressions) >>> 2. By using a "?" token within the implicit lambda, you would create a >>> lambda that takes a single argument. If there is no such token, then >>> it would take no arguments. >> >>Would it be worth extending this to multiple arguments? I think it would be handy, but I can't think of any way to make it look nearly as good as the 0- and 1-arg forms. >> >>Obviously this should still be a one-arg lambda that just uses its parameter repeatedly: >> >>? ? : ? * x**2 + ? * x + ? >> >>Sticking a number on the ? looks terrible: >> >>? ? : ?1 * x**2 + ?2 * x + ?3 >> >>Looking at the other DB-API 2.0 param styles, this is hideous because of the way it uses colons twice: >> >> >>? ? : :1 * x**2 + :2 * x + :3 >> >>Something like this wouldn't be too bad, except that it already has a much more useful meaning, set literals: >> >>? ? : {1} * x**2 + {2} * x + {3} >> >>So really, the best I can come up with is the boost::lambda version I already didn't like and you already dismissed without comment: >> >>? ? : _1 * x**2 + _2 * x + _3 >> >>On the other hand, any expression this complex?even with just one parameter?is already starting to look pretty bad as an implicit lambda, and the downsides of an explicit lambda go down the longer it gets. So maybe the best answer is: >> >>? ? lambda a, b, c: a * x**2 + b * x + c >> >>Anyway, we can always start with just 0- and 1-argument forms and come back to expand the idea later, right? >> >>> However, the examples in that PEP are no longer useful, as they have >>> since been addressed by dedicated constructs (conditional expressions >>> in PEP 308 and context managers in PEP 343). >>> >>> Note that I'm not saying this is a good idea. However, I am saying I >>> think it is an idea that may be worth exploring further to see if it >>> also addresses at least some of the use cases given in PEP's 403 and >>> 3150 (especially in combination with PEP 463) by making custom key and >>> predicate functions easier to define. >> >>I'll start digging up some examples; it is hard to predict how nice this will look until we see it in a wide range of realistic code. >>_______________________________________________ >>Python-ideas mailing list >>Python-ideas at python.org >>https://mail.python.org/mailman/listinfo/python-ideas >>Code of Conduct: http://python.org/psf/codeofconduct/ >+1 for the question mark. However, there's the problem of multi-argument lambdas. The ?(number) looks to much like Perl. What about something like ? being an array of arguments? Doesn't that just make it even _more_ like perl, where arguments are always passed as an array? Anyway, presumably you mean that this: ? ? lambda *args: args[0] * x**2 + args[1] * x + args[2] ? could be written as: ? ? : ?[0] * x**2 + ?[1] * x + ?[2] I think that's horribly unreadable?partly because of the use of *args in the first place, and partly because ?[0] looks too? perlesque. I think the best answer is the one I suggested above?and which I think Nick implicitly assumed because he's smart enough to get there directly: At least initially, just do 0-argument and 1-argument lambdas. As long as you leave the door open for extended syntaxes in a future proposal, we don't need to figure them out yet. From abarnert at yahoo.com Fri Feb 21 23:05:04 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 21 Feb 2014 14:05:04 -0800 (PST) Subject: [Python-ideas] Infix functions Message-ID: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> While we're discussing crazy ideas inspired by a combination of a long-abandoned PEP and Haskell idioms (see the implicit lambda thread), here's another: arbitrary infix operators: ? ? a `foo` b == foo(a, b) I'm not sure there's any use for this, I just have a nagging feeling there _might_ be, based on thinking about how Haskell uses them to avoid the need for special syntax in a lot of cases where Python can't. This isn't a new idea; it came up a lot in the early days of Numeric. PEP 225 (http://legacy.python.org/dev/peps/pep-0225/) has a side discussion on "Impact on named operators" that starts off with: ? ? The discussions made it generally clear that infix operators is a ? ? scarce resource in Python, not only in numerical computation, but ? ? in other fields as well. ?Several proposals and ideas were put ? ? forward that would allow infix operators be introduced in ways ? ? similar to named functions. ?We show here that the current ? ? extension does not negatively impact on future extensions in this ? ? regard. The future extension was never written as a PEP because 225 and its competitors were were all deferred/abandoned. Also, most of the anticipated use cases for it back then were solved in other ways. The question is whether there are _other_ use cases that make the idea worth reviving. The preferred syntax at that time was @opname. There are other alternatives in that PEP, but they all look a lot worse.?Nobody proposed the `opname` because it meant repr(opname), but in 3.x that isn't a problem, so I'm going to use that instead, because? In Haskell, you can turn prefix function into an infix operator by enclosing it in backticks, and turn any infix operator into a prefix function by enclosing it in parens. (Ignore the second half of that, because Python has a small, fixed set of operators, and they all have short, readable names in the operator module.)?And, both in the exception-expression discussion and the while-clause discussion, I noticed that this feature is essential to the way Haskell deals with both of these features without requiring lambdas all over the place. The Numeric community wanted this as a way of defining new mathematical operators. For example, there are three different ways you can "multiply" two vectors?element-wise, dot-product, or cross-product?and you can't spell all of them as a * b. So, how do you spell the others? There?were proposals to add a new @ operator, or to double the set of operators by adding a ~-prefixed version of each, or to allow custom operators made up of any string of symbols (which Haskell allows), but none of those are remotely plausible extensions to Python. (There's a reason those PEPs were all deferred/abandoned.) However, you?could solve the problem easily with infix functions: ? ? m `cross` n ? ? m `dot` n In Haskell, it's used for all kinds of things beyond that, from type constructors: ? ? a `Pair` b ? ? a `Tree` (b `Tree` c `Tree` d) `Tree` e ? to higher-order functions. The motivating example here is that exception handling is done with the catch function, and instead of this: ? ?catch func (\e -> 0) ? you can write: ? ? func `catch` \e -> 0 Or, in Python terms, instead of this: ? ? catch(func, lambda e: 0) ? it's: ? ? func `catch` lambda e: 0 ? which isn't miles away from: ? ? func() except Exception as e: 0 ? and that's (part of) why Haskell doesn't have or need custom exception expression syntax. PEP 225 assumed that infix functions would be defined in terms of special methods. The PEP implicitly assumed they were going to convince Guido to rename m.__add__(n) to m."+"(n), so m @cross n would obviously be m."@cross"(n). But failing that, there are other obvious possibilities, like m.__ at cross__(n), m.__cross__(n), m.__infix__('cross')(n), etc. But really, there's no reason for special methods at all?especially with PEP 443 generic functions. Instead, it could just mean this: ? ? cross(m, n) ? just as in Haskell. In fact, in cases where infix functions map to methods, there's really no reason not to just _write_ them as methods. That's how NumPy solves the vector-multiplication problem; the dot product of m and n is just: ? ? m.dot(n) (The other two meanings are disambiguated in a different way?m*n means cross-product for perpendicular vectors, element-wise multiplication for parallel vectors.) But this can still get ugly for long expressions. Compare: ? ? a `cross` b + c `cross` (d `dot` e) ? ? a.cross(b).add(c.cross(d.dot(e))) ? ? add(cross(a, b), cross(c, dot(d, e)) The difference between the first and second isn't as stark as between the second and third, but it's still pretty clear. And consider the catch example again: ? ? func.catch(lambda e: 0) Unless catch is a method on all callables, this makes no sense, which means?method syntax isn't exactly extensible. There are obviously lots of questions raised.?The biggest one is: are there actually real-life use cases (especially given that NumPy has for the most part satisfactorily solved this problem for most numeric Python users)? Beyond that: What can go inside the backticks? In Haskell, it's an identifier, but the Haskell wiki (http://www.haskell.org/haskellwiki/Infix_expressions) notes that "In?ABC the stuff between backquotes is not limited to an identifier, but?any expression may occur here" (presumably that's not Python's ancestor ABC, which I'm pretty sure used backticks for repr, but some other language with the same name) and goes on to show how you can build that in Haskell if you really want to? but I think that's an even worse idea for Python than for Haskell. Maybe attribute references would be OK, but anything beyond that, even slicing (to get functions out of a table) looks terrible: ? ? a `self.foo` b ? ? a `funcs['foo']` b Python 2.x's repr backticks allowed spaces inside the ticks. For operator syntax this would look terrible? but it does make parsing easier, and there's no reason to actually _ban_ it, just strongly discourage it. What should the precedence and associativity be? In Haskell, it's customizable?which is impossible in Python, where functions are defined at runtime but calls are parsed at call time?but defaults to left-associative and highest-precedence. In Python, I think it would be more readable as coming between comparisons and bitwise ops. In grammar terms: ? ? infix_expr ::= or_expr | or_expr "`" identifier "`" infix_expr ? ? comparison ::= infix_expr ( comparison_operator infix_expr ) * That grammar could easily be directly evaluated into a Call node in the AST, or it could have a different node (mainly because I'm curious whether MacroPy could do something interesting with it?), like: ? ? InfixCall(left=Num(n=1), op=Name(id='foo', ctx=Load()), right=Num(n=2)) Either way, it ultimately just compiles to the normal function-call bytecode, so?there's no need for any change to the interpreter. (That does mean that "1 `abs` 2" would raise a normal "TypeError: abs expected at most 1 arguments, got 2" instead of a more specific "TypeError: abs cannot be used as an infix operator", but I don't think that's a problem.) This theoretically could be expanded from operators to augmented assignment? but it shouldn't be; anyone who wants to write this needs to be kicked in the head: ? ? a `func`= b From rymg19 at gmail.com Fri Feb 21 23:09:47 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Fri, 21 Feb 2014 16:09:47 -0600 Subject: [Python-ideas] The Return of Implicit Lambda (Re: Allowing breaks in generator expressions by overloading the while keyword) In-Reply-To: <1393020176.19093.YahooMailNeo@web181002.mail.ne1.yahoo.com> References: <20140221102403.GV3684@ando> <1393014891.79562.YahooMailNeo@web181003.mail.ne1.yahoo.com> <1393020176.19093.YahooMailNeo@web181002.mail.ne1.yahoo.com> Message-ID: On Fri, Feb 21, 2014 at 4:02 PM, Andrew Barnert wrote: > From: Ryan Gonzalez > Sent: Friday, February 21, 2014 1:36 PM > > > >On Fri, Feb 21, 2014 at 2:34 PM, Andrew Barnert > wrote: > > > >From: Nick Coghlan > >> > >>Sent: Friday, February 21, 2014 4:18 AM > >> > >> > >>> On 21 February 2014 20:24, Steven D'Aprano > >>> wrote: > >>>> > >>>> but alas both Nick Coglan and (if I recall correctly) Guido have > ruled > >>>> that Python won't get this, so until the Revolution comes, it isn't > >>>> going to happen. > >>> > >>> It's not that it can't happen - it's that someone would need to > >>> build > >>> a case of a similar calibre to the one Chris Angelico is currently > >>> putting together for except expressions in PEP 463 :) > >>> > >>> However, it's a *much* bigger challenge in this case, as > >>> itertools.takewhile exists, whereas there's currently no way to do > >>> exception handling as part of a larger expression without major > >>> contortions. > >>> > >>> Re-reading Andrew's post at > >>> > http://stupidpythonideas.blogspot.com.au/2013/07/syntactic-takewhile.html, > >>> I'm actually more intrigued by Haskell's shorthand for lambda > >>> functions that consist of a single binary operator. > >>> > >>> Consider: > >>> > >>> isprime = all(n % p for p in takewhile((lambda p: p**2 < n), > >>> primes_seen)) > >>> > >>> Is there are a nicer way to write that? The Haskell equivalent is: > >>> > >>> (< n) . (** 2) > >>> > >> > >>> That's not very readable to most Python programmers > >> > >>That's really not a shorthand for lambda, it's a combination of other > features that make lambdas often unnecessary. I think the C++ boost::lambda > example is a better fit: > >> > >> _1 ** 2 < n > >> > >>> but what if you > >> > >>> could write something like: > >>> > >>> isprime = all(n % p for p in takewhile((: ? ** 2 < n), > primes_seen)) > >> > >>I like this. The ? is a lot better than the _ or _1 I suggested > (especially to people who do a lot of SQL programming in Python and already > use ? as an inline parameter), and using a colon makes it explicit without > adding too much visual noise. The only concern is whether using up one of > the handful of as-yet-unused ASCII symbols raises the bar a little too high... > >> > >>> This is somewhat similar to the implicit lambda proposal in > >> > >>> http://legacy.python.org/dev/peps/pep-0312/, but with the following > >>> two essential differences: > >>> > >>> 1. The parentheses would be required (as per generator expressions, > >>> and as is being discussed for except expressions) > >>> 2. By using a "?" token within the implicit lambda, you would create a > >>> lambda that takes a single argument. If there is no such token, then > >>> it would take no arguments. > >> > >>Would it be worth extending this to multiple arguments? I think it would > be handy, but I can't think of any way to make it look nearly as good as > the 0- and 1-arg forms. > >> > >>Obviously this should still be a one-arg lambda that just uses its > parameter repeatedly: > >> > >> : ? * x**2 + ? * x + ? > >> > >>Sticking a number on the ? looks terrible: > >> > >> : ?1 * x**2 + ?2 * x + ?3 > >> > >>Looking at the other DB-API 2.0 param styles, this is hideous because of > the way it uses colons twice: > >> > >> > >> : :1 * x**2 + :2 * x + :3 > >> > >>Something like this wouldn't be too bad, except that it already has a > much more useful meaning, set literals: > >> > >> : {1} * x**2 + {2} * x + {3} > >> > >>So really, the best I can come up with is the boost::lambda version I > already didn't like and you already dismissed without comment: > >> > >> : _1 * x**2 + _2 * x + _3 > >> > >>On the other hand, any expression this complex--even with just one > parameter--is already starting to look pretty bad as an implicit lambda, and > the downsides of an explicit lambda go down the longer it gets. So maybe > the best answer is: > >> > >> lambda a, b, c: a * x**2 + b * x + c > >> > >>Anyway, we can always start with just 0- and 1-argument forms and come > back to expand the idea later, right? > >> > >>> However, the examples in that PEP are no longer useful, as they have > >>> since been addressed by dedicated constructs (conditional expressions > >>> in PEP 308 and context managers in PEP 343). > >>> > >>> Note that I'm not saying this is a good idea. However, I am saying I > >>> think it is an idea that may be worth exploring further to see if it > >>> also addresses at least some of the use cases given in PEP's 403 and > >>> 3150 (especially in combination with PEP 463) by making custom key and > >>> predicate functions easier to define. > >> > >>I'll start digging up some examples; it is hard to predict how nice this > will look until we see it in a wide range of realistic code. > >>_______________________________________________ > >>Python-ideas mailing list > >>Python-ideas at python.org > >>https://mail.python.org/mailman/listinfo/python-ideas > >>Code of Conduct: http://python.org/psf/codeofconduct/ > > > >+1 for the question mark. However, there's the problem of multi-argument > lambdas. The ?(number) looks to much like Perl. What about something like ? > being an array of arguments? > > > Doesn't that just make it even _more_ like perl, where arguments are > always passed as an array? > > Anyway, presumably you mean that this: > > lambda *args: args[0] * x**2 + args[1] * x + args[2] > > ... could be written as: > > : ?[0] * x**2 + ?[1] * x + ?[2] > > > I think that's horribly unreadable--partly because of the use of *args in > the first place, and partly because ?[0] looks too... perlesque. > > I think the best answer is the one I suggested above--and which I think > Nick implicitly assumed because he's smart enough to get there directly: At > least initially, just do 0-argument and 1-argument lambdas. As long as you > leave the door open for extended syntaxes in a future proposal, we don't > need to figure them out yet. > The question mark still looks like Perl. What about a reserved word instead? -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From ericsnowcurrently at gmail.com Fri Feb 21 23:16:16 2014 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Fri, 21 Feb 2014 15:16:16 -0700 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: <20140218230110.GT4519@ando> <20140220220040.GL3684@ando> Message-ID: On Fri, Feb 21, 2014 at 2:56 AM, Peter Otten <__peter__ at web.de> wrote: > ????? wrote: > >> What is the "classic" use case for next() raising StopIteration, to be >> silently caught ? We need __next__ to do so in for loops, but when do we >> need it in the functional form? > > Pretty much every generator that treats the first item(s) specially, like > the one I gave above: > >>> def process_source(source): >>> it = iter(source) >>> first = next(it) >>> for item in it: >>> yield first * item > > Or these: > > http://docs.python.org/dev/library/itertools.html#itertools.accumulate > http://docs.python.org/dev/library/itertools.html#itertools.groupby > http://docs.python.org/dev/library/itertools.html#itertools.islice > ... > > The behaviour of next() is really a feature rather than a bug. It's also nice that you can pass it a default to avoid StopIteration in one-off iteration cases (like you cited above): first = next(it, 0) or generically: next((x for x in []), None) -eric From masklinn at masklinn.net Fri Feb 21 23:31:27 2014 From: masklinn at masklinn.net (Masklinn) Date: Fri, 21 Feb 2014 23:31:27 +0100 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: Message-ID: <5A29A7CC-2218-47CB-990A-8F9A48D63141@masklinn.net> On 2014-02-21, at 18:30 , Chris Angelico wrote: > So here's the proposal. Introduce a new operator to Python, just like > the dot operator but behaving differently when it returns a bound > method. We can possibly use ->, or maybe create a new operator that > currently makes no sense, like .. or .> or something. Its semantics > would be: As Yuri noted the concept exists, AFAIK it was introduced by smalltalk as "message cascading". Basically, the ability to send a sequence of messages to the same subject without having to repeatedly specify the subject. I believe Dart is the first language to have resurrected this feature so far. The cascading operator in smalltalk was `;` (the message-send operator is the space), so e.g. foo message ; message2: aParameter ; message3. would send all of message, message2:aParameter and message 3 to `foo`, in that specific order. In smalltalk, a cascade returns the result of the last message in the cascade. smalltalk provides a `yourself` operator to return the subject itself. Cascading is very commonly used to initialise collections as smalltalk was born at a time where literal high-level collections were not exactly a thing: aCollection := (OrderedCollection new) add: 1 ; add: 2 ; add: 3 ; youself. > 1) Look up the attribute following it on the object, exactly as per > the current . operator > 2) If the result is not a function, return it, exactly as per current. > 3) If it is a function, though, return a wrapper which, when called, > calls the inner function and then returns self. I could be wrong, but I'm pretty sure this is an over-complication when you look at it at the bytecode level: you can load the subject as many times as you've got attr accesses to do on it, or you could have an alternate attr access which puts TOS back. No need for a wrapper. > Effectively, x->y would be equivalent to chain(x.y): > > def chain(func): > def chainable(self, *args, **kwargs): > func(self, *args, **kwargs) > return self > return chainable > > Could be useful in a variety of contexts. > > Thoughts? No need for a wrapper. Where `a.b` compiles to LOAD_FAST a LOAD_ATTR b POP_TOP `a->b` would compile to LOAD_FAST a DUP_TOP LOAD_ATTR b POP_TOP at this point you've got an a left on the stack and can reuse it: `a->b()->c()->d()` would be LOAD_FAST a DUP_TOP LOAD_ATTR b CALL_FUNCTION POP_TOP DUP_TOP LOAD_ATTR c CALL_FUNCTION POP_TOP DUP_TOP LOAD_ATTR d CALL_FUNCTION POP_TOP The tail end of the cascade would be slightly more complex in that it would be: ROT_TWO POP_TOP so that the subject is discarded and the value of the last attribute/method is available on the stack (it may get popped as well if it's unused). Or maybe it would do nothing special and the cascade would yield (or pop) the subject unless closed by an attribute access or regular method call. That would avoid the requirement of a `yourself`-type method when initialising mutables, although the final irregularity may lack visibility. From greg.ewing at canterbury.ac.nz Fri Feb 21 23:33:20 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 22 Feb 2014 11:33:20 +1300 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: References: <20140221102403.GV3684@ando> Message-ID: <5307D430.5080608@canterbury.ac.nz> Nick Coghlan wrote: > isprime = all(n % p for p in takewhile((: ? ** 2 < n), primes_seen)) -1, this is far too cryptic and ugly to have a place in Python. I'd be +0 on adding a while clause to for-loops, in *both* comprehension and statement forms, so that the above could be written isprime = all(n % p for p in primes_seen while p ** 2 < n) Adding a corresponding clause to the for-statement would allow the semantic definition of list comprehensions in terms of the statement expansion to be preserved. However, this all hinges on whether there is enough use for the feature to be worth adding. I'm yet to be convinced of that. -- Greg From greg.ewing at canterbury.ac.nz Fri Feb 21 23:37:50 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 22 Feb 2014 11:37:50 +1300 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: References: <20140221102403.GV3684@ando> Message-ID: <5307D53E.80806@canterbury.ac.nz> Nick Coghlan wrote: > Oh, and under such a proposal, the generator expression: > > (x for x in seq) > > would be semantically equivalent to: > > (: yield x for x in ?)(seq) I don't follow that, since 'yield x for x in y' is not currently a legal expression. -- Greg From masklinn at masklinn.net Fri Feb 21 23:43:13 2014 From: masklinn at masklinn.net (Masklinn) Date: Fri, 21 Feb 2014 23:43:13 +0100 Subject: [Python-ideas] Method chaining notation In-Reply-To: <5307CC96.9020200@gmail.com> References: <5307CC96.9020200@gmail.com> Message-ID: <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> On 2014-02-21, at 23:00 , spir wrote: > Also, I don't find the idea of having a builtin construct for such hacks a good idea. Libs for which this may be practicle can return self --end of the story. That has two issues though: 1. it makes chainability a decision of the library author, the library user gets to have no preference. This means e.g. you can't create a tree of elements in ElementTree in a single expression (AFAIK Element does not take children parameters). With cascading, the user can "chain" a library whose author did not choose to support chaining (in fact with cascading no author would ever need to support chaining again). 2. where a return value can make sense (and be useful) the author *must* make a choice. No way to chain `dict.pop()` since it returns the popped value, even if `pop` was only used for its removal-with-shut-up properties. With cascading the user can have his cake and eat it: he gets the return value if he wants it, and can keep "chaining" if he does not care. From pyideas at rebertia.com Fri Feb 21 23:47:10 2014 From: pyideas at rebertia.com (Chris Rebert) Date: Fri, 21 Feb 2014 14:47:10 -0800 Subject: [Python-ideas] Infix functions In-Reply-To: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> Message-ID: On Fri, Feb 21, 2014 at 2:05 PM, Andrew Barnert wrote: > While we're discussing crazy ideas inspired by a combination of a long-abandoned PEP and Haskell idioms (see the implicit lambda thread), here's another: arbitrary infix operators: > > a `foo` b == foo(a, b) Prior discussion: https://mail.python.org/pipermail/python-ideas/2007-January/000050.html Which resulted in a new item in PEP 3099 ("Things that will Not Change in Python 3000"; http://legacy.python.org/dev/peps/pep-3099/ ): * No more backticks. Backticks (`) will no longer be used as shorthand for repr -- but that doesn't mean they are available for other uses. Even ignoring the backwards compatibility confusion, the character itself causes too many problems (in some fonts, on some keyboards, when typesetting a book, etc). I think people using suboptimal fonts and keyboard layouts should find better ones... Cheers, Chris From abarnert at yahoo.com Fri Feb 21 23:43:48 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 21 Feb 2014 14:43:48 -0800 (PST) Subject: [Python-ideas] Method chaining notation In-Reply-To: References: Message-ID: <1393022628.63093.YahooMailNeo@web181003.mail.ne1.yahoo.com> From: Chris Angelico Sent: Friday, February 21, 2014 9:30 AM > Yeah, I'm insane, opening another theory while I'm busily championing > a PEP. But it was while writing up the other PEP that I came up with a > possible syntax for this. > > In Python, as in most languages, method chaining requires the method > to return its own object. > > class Count: > ? ? def __init__(self): self.n = 0 > ? ? def inc(self): > ? ? ? ? self.n += 1 > ? ? ? ? return self > > dracula = Count() > dracula.inc().inc().inc() > print(dracula.n) > > It's common in languages like C++ to return *this by reference if > there's nothing else useful to return. It's convenient, it doesn't > cost anything much, and it allows method chaining. The Python > convention, on the other hand, is to return self only if there's a > very good reason to, and to return None any time there's mutation that > could plausibly return a new object of the same type (compare > list.sort() vs sorted()). Method chaining is therefore far less common > than it could be, with the result that, often, intermediate objects > need to be separately named and assigned to. I think that's intentional, as a way of discouraging (mutable) method chaining and similar idioms?and that Python code ultimately benefits from it. In Python, each statement generally mutates one thing one time. That makes it simpler to skim Python code and get an idea of what it does than code in languages like?C++ or JavaScript. On top of that, it's the lack of method chaining that means lines of Python code tend to be just about the right length, and don't need to be continued very often. If you break that, you lose most of the readability benefits of Python's whitespace-driven syntax. In JavaScript or Ruby, a function call is often a dozen lines long. Readable programs use indentation conventions for expressions, just as they do for block statements, but those expression indentation conventions do not map to indent tokens in Python (and indentation rules in Python-friendly editors) the same way the block statement conventions do. > I pulled up one file from > Lib/tkinter (happened to pick filedialog) and saw what's fairly > typical of Python GUI code: Tkinter has its own weird idioms that aren't necessarily representative of Python in general. And PyQt/PySide and PyGTK/GObject?have their own _different_ weird idioms. Partly this is because they're mappings to Python of idioms from Tcl, C++, and C (and/or Vala), respectively. But whatever the reason,?I'm not sure it's reasonable to call any of them typical. > So here's the proposal. Introduce a new operator to Python, just like > the dot operator but behaving differently when it returns a bound > method. We can possibly use ->, or maybe create a new operator that > currently makes no sense, like .. or .> or something. Its semantics > would be: > > 1) Look up the attribute following it on the object, exactly as per > the current . operator > 2) If the result is not a function, return it, exactly as per current. Why? Why not just use x.y for those cases, and make it a TypeError if you use x->y for a data attribute? It seems pretty misleading to "chain" through something that isn't a function call?especially since it doesn't actually chain in that case. > 3) If it is a function, though, return a wrapper which, when called, > calls the inner function and then returns self. For normal methods, the result will _not_ be a function, it will be a bound method. It will only be a function for classmethods, staticmethods, functions you've explicitly added to self after construction, and functions returned by __getattr__ or custom __getattribute__. And this isn't just nit-picking; it's something you can take advantage of: bound methods have a __self__, so your wrapper can just be: ? ? def wrap_method(method): ? ? ? ? @wraps(method) ? ? ? ? def wrapper(*args, **kwargs): ? ? ? ? ? ? method(*args, **kwargs) ? ? ? ? ? ? return method.__self__ ? ? ? ? return wrapper Or, alternatively, you've already got the self from the lookup, so you could just use that?in which case?you can even make it work on static and class methods if you want, although you don't have to if you don't want. And, depending on where you hook in to attribute lookup, you may be able to distinguish methods from data attributes before calling the descriptor's __get__, as explained toward the bottom, making this even simpler. > This can be done with an external wrapper, so it might be possible to > do this with MacroPy. It absolutely must be a compact notation, > though. > > This probably wouldn't interact at all with __getattr__ (because the > attribute has to already exist for this to work), Why? See below for details. > and definitely not > with __setattr__ or __delattr__ (mutations aren't affected). How it > interacts with __getattribute__ I'm not sure; whether it adds the > wrapper around any returned functions or applies only to something > that's looked up "the normal way" can be decided by ease of > implementation. > > Supposing this were done, using the -> token that currently is used > for annotations as part of 'def'. Here's how the PyGTK code would > look: > > import pygtk > pygtk.require('2.0') > import gtk > > def callback(widget, data): > ? ? print "Hello again - %s was pressed" % data > > def delete_event(widget, event, data=None): > ? ? gtk.main_quit() > ? ? return False > > window = (gtk.Window(gtk.WINDOW_TOPLEVEL) > ? ? ->set_title("Hello Buttons!") > ? ? ->connect("delete_event", delete_event) > ? ? ->set_border_width(10) > ? ? ->add(gtk.HBox(False, 0) > ? ? ? ? ->pack_start( > ? ? ? ? ? ? gtk.Button("Button 1")->connect("clicked", > callback, "button 1"), > ? ? ? ? ? ? True, True, 0) > ? ? ? ? ->pack_start( > ? ? ? ? ? ? gtk.Button("Button 1")->connect("clicked", > callback, "button 1"), > ? ? ? ? ? ? True, True, 0) > ? ? ) > ? ? ->show_all() > ) > > gtk.main() I personally think this looks terrible, and unpythonic, for exactly the reasons I suspected I would. I do not want to write?or, more importantly, read?15-line expressions in Python. Maybe that's just me. > Again, the structure of the code would match the structure of the > window. Unlike the Pike version, this one can even connect signals as > part of the method chaining. > > Effectively, x->y would be equivalent to chain(x.y): > > def chain(func): > ? ? def chainable(self, *args, **kwargs): > ? ? ? ? func(self, *args, **kwargs) > ? ? ? ? return self > ? ? return chain able With this definition, it definitely works with __getattr__, __getattribute__, instance attributes, etc., not just normal methods. The value of x.y is the result of x.__getattribute__('y'), which, unless you've overridden it, does something similar to this Python code (slightly simplified): ? ? try:?return x.__dict__['y'] ? ? except KeyError: pass ? ? for cls in type(x).__mro__: ? ? ? ? try: return cls.__dict__['y'].__get__(x) ? ? ? ? except KeyError: pass ? ? return x.__getattr__('y') By the time you get back x.y, you have no way of knowing whether it came from a normal method, a method in the instance dict, a __getattr__ call, or some funky custom stuff from __getattribute__. And I don't see why you have any reason to care, either. However, if you want the logic you suggested, as I mentioned earlier, you could implement x->y from scratch, which means you can hook just normal methods and nothing else, instead of switching on type or callability or something. For example: ? ? for cls in type(x).__mro__: ? ? ? ? try: ? ? ? ? ? ? descr = cls.__dict__['y'] ? ? ? ? except KeyError: ? ? ? ? ? ? pass ? ? ? ? else: ? ? ? ? ? ? if hasattr(descr, '__set__'): ? ? ? ? ? ? ? ? return descr.__get__(x) # data descriptor ? ? ? ? ? ? else: ? ? ? ? ? ? ? ? return wrap_method(descr.__get__(x)) # non-data descriptor > Could be useful in a variety of contexts. > > Thoughts? > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > From abarnert at yahoo.com Fri Feb 21 23:49:03 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 21 Feb 2014 14:49:03 -0800 (PST) Subject: [Python-ideas] The Return of Implicit Lambda (Re: Allowing breaks in generator expressions by overloading the while keyword) In-Reply-To: References: <20140221102403.GV3684@ando> <1393014891.79562.YahooMailNeo@web181003.mail.ne1.yahoo.com> <1393020176.19093.YahooMailNeo@web181002.mail.ne1.yahoo.com> Message-ID: <1393022943.95229.YahooMailNeo@web181002.mail.ne1.yahoo.com> From: Ryan Gonzalez Sent: Friday, February 21, 2014 2:09 PM >On Fri, Feb 21, 2014 at 4:02 PM, Andrew Barnert wrote: > >From: Ryan Gonzalez >>Sent: Friday, February 21, 2014 1:36 PM >> >> >> >>>On Fri, Feb 21, 2014 at 2:34 PM, Andrew Barnert wrote: >>> >>>From: Nick Coghlan >>>> >>>>Sent: Friday, February 21, 2014 4:18 AM >>>> >>>> >>>>> On 21 February 2014 20:24, Steven D'Aprano >>>>> wrote: >>>>>> >>>>>> ?but alas both Nick Coglan and (if I recall correctly) Guido have ruled >>>>>> ?that Python won't get this, so until the Revolution comes, it isn't >>>>>> ?going to happen. >>>>> >>>>> It's not that it can't happen - it's that someone would need to >>>>> build >>>>> a case of a similar calibre to the one Chris Angelico is currently >>>>> putting together for except expressions in PEP 463 :) >>>>> >>>>> However, it's a *much* bigger challenge in this case, as >>>>> itertools.takewhile exists, whereas there's currently no way to do >>>>> exception handling as part of a larger expression without major >>>>> contortions. >>>>> >>>>> Re-reading Andrew's post at >>>>> http://stupidpythonideas.blogspot.com.au/2013/07/syntactic-takewhile.html, >>>>> I'm actually more intrigued by Haskell's shorthand for lambda >>>>> functions that consist of a single binary operator. >>>>> >>>>> Consider: >>>>> >>>>> ? ? isprime = all(n % p for p in takewhile((lambda p: p**2 < n), >>>>> primes_seen)) >>>>> >>>>> Is there are a nicer way to write that? The Haskell equivalent is: >>>>> >>>>> ? ?(< n) . (** 2) >>>>>? >>>> >>>>> That's not very readable to most Python programmers >>>> >>>>That's really not a shorthand for lambda, it's a combination of other features that make lambdas often unnecessary. I think the C++ boost::lambda example is a better fit: >>>> >>>>? ? _1 ** 2 < n >>>> >>>>> but what if you >>>> >>>>> could write something like: >>>>> >>>>> ? ? isprime = all(n % p for p in takewhile((: ? ** 2 < n), primes_seen)) >>>> >>>>I like this. The ? is a lot better than the?_ or _1 I suggested (especially to people who do a lot of SQL programming in Python and already use ? as an inline parameter), and using a colon makes it explicit without adding too much visual noise. The only concern is whether using up one of the handful of as-yet-unused ASCII symbols raises the bar a little too high? >>>> >>>>> This is somewhat similar to the implicit lambda proposal in >>>> >>>>> http://legacy.python.org/dev/peps/pep-0312/, but with the following >>>>> two essential differences: >>>>> >>>>> 1. The parentheses would be required (as per generator expressions, >>>>> and as is being discussed for except expressions) >>>>> 2. By using a "?" token within the implicit lambda, you would create a >>>>> lambda that takes a single argument. If there is no such token, then >>>>> it would take no arguments. >>>> >>>>Would it be worth extending this to multiple arguments? I think it would be handy, but I can't think of any way to make it look nearly as good as the 0- and 1-arg forms. >>>> >>>>Obviously this should still be a one-arg lambda that just uses its parameter repeatedly: >>>> >>>>? ? : ? * x**2 + ? * x + ? >>>> >>>>Sticking a number on the ? looks terrible: >>>> >>>>? ? : ?1 * x**2 + ?2 * x + ?3 >>>> >>>>Looking at the other DB-API 2.0 param styles, this is hideous because of the way it uses colons twice: >>>> >>>> >>>>? ? : :1 * x**2 + :2 * x + :3 >>>> >>>>Something like this wouldn't be too bad, except that it already has a much more useful meaning, set literals: >>>> >>>>? ? : {1} * x**2 + {2} * x + {3} >>>> >>>>So really, the best I can come up with is the boost::lambda version I already didn't like and you already dismissed without comment: >>>> >>>>? ? : _1 * x**2 + _2 * x + _3 >>>> >>>>On the other hand, any expression this complex?even with just one parameter?is already starting to look pretty bad as an implicit lambda, and the downsides of an explicit lambda go down the longer it gets. So maybe the best answer is: >>>> >>>>? ? lambda a, b, c: a * x**2 + b * x + c >>>> >>>>Anyway, we can always start with just 0- and 1-argument forms and come back to expand the idea later, right? >>>> >>>>> However, the examples in that PEP are no longer useful, as they have >>>>> since been addressed by dedicated constructs (conditional expressions >>>>> in PEP 308 and context managers in PEP 343). >>>>> >>>>> Note that I'm not saying this is a good idea. However, I am saying I >>>>> think it is an idea that may be worth exploring further to see if it >>>>> also addresses at least some of the use cases given in PEP's 403 and >>>>> 3150 (especially in combination with PEP 463) by making custom key and >>>>> predicate functions easier to define. >>>> >>>>I'll start digging up some examples; it is hard to predict how nice this will look until we see it in a wide range of realistic code. >>>>_______________________________________________ >>>>Python-ideas mailing list >>>>Python-ideas at python.org >>>>https://mail.python.org/mailman/listinfo/python-ideas >>>>Code of Conduct: http://python.org/psf/codeofconduct/ >> >> >>>+1 for the question mark. However, there's the problem of multi-argument lambdas. The ?(number) looks to much like Perl. What about something like ? being an array of arguments? >> >> >>Doesn't that just make it even _more_ like perl, where arguments are always passed as an array? >> >>Anyway, presumably you mean that this: >> >>? ? lambda *args: args[0] * x**2 + args[1] * x + args[2] >> >>? could be written as: >> >>? ? : ?[0] * x**2 + ?[1] * x + ?[2] >> >> >>I think that's horribly unreadable?partly because of the use of *args in the first place, and partly because ?[0] looks too? perlesque. >> >>I think the best answer is the one I suggested above?and which I think Nick implicitly assumed because he's smart enough to get there directly: At least initially, just do 0-argument and 1-argument lambdas. As long as you leave the door open for extended syntaxes in a future proposal, we don't need to figure them out yet. >> >The question mark still looks like Perl. > > >What about a reserved word instead? I can't think of any existing keyword that (a) is unambiguous in that syntax, or (b) doesn't read like someone with Wernicke's aphasia substituting random words. But I suppose a _new_ keyword could work: ? ? : arg **2 < n And that seems like it could be extended to multiple arguments pretty nicely by adding a "keyword schema", or just a parse rule that matches "arg" digit * instead of "arg": ? ? : arg1 * x**2 + arg2 * x + arg3 However, I'm pretty sure "arg" is used in lots of real-life code, especially in this context: ? ? if __name__ == '__main__': ? ? ? ? import sys ? ? ? ? for arg in sys.argv[1:]: ? ? ? ? ? ? print(spam(arg)) Any suggestions for a better keyword? From python at mrabarnett.plus.com Fri Feb 21 23:54:05 2014 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 21 Feb 2014 22:54:05 +0000 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: References: <20140221102403.GV3684@ando> Message-ID: <5307D90D.9080002@mrabarnett.plus.com> On 2014-02-21 12:18, Nick Coghlan wrote: > On 21 February 2014 20:24, Steven D'Aprano wrote: >> >> but alas both Nick Coglan and (if I recall correctly) Guido have ruled >> that Python won't get this, so until the Revolution comes, it isn't >> going to happen. > > It's not that it can't happen - it's that someone would need to build > a case of a similar calibre to the one Chris Angelico is currently > putting together for except expressions in PEP 463 :) > > However, it's a *much* bigger challenge in this case, as > itertools.takewhile exists, whereas there's currently no way to do > exception handling as part of a larger expression without major > contortions. > > Re-reading Andrew's post at > http://stupidpythonideas.blogspot.com.au/2013/07/syntactic-takewhile.html, > I'm actually more intrigued by Haskell's shorthand for lambda > functions that consist of a single binary operator. > > Consider: > > isprime = all(n % p for p in takewhile((lambda p: p**2 < n), primes_seen)) > > Is there are a nicer way to write that? The Haskell equivalent is: > > (< n) . (** 2) > > That's not very readable to most Python programmers, but what if you > could write something like: > > isprime = all(n % p for p in takewhile((: ? ** 2 < n), primes_seen)) > [snip] C# has a lambda operator "=>". That would give: isprime = all(n % p for p in takewhile((x => x ** 2 < n), primes_seen)) From steve at pearwood.info Fri Feb 21 23:57:34 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 22 Feb 2014 09:57:34 +1100 Subject: [Python-ideas] Joining dicts again In-Reply-To: <3f21074da4e8e68b4863c1f312848a5b@draco.uberspace.de> References: <3f21074da4e8e68b4863c1f312848a5b@draco.uberspace.de> Message-ID: <20140221225733.GW3684@ando> On Fri, Feb 21, 2014 at 05:05:58PM +0100, Markus Unterwaditzer wrote: [...] > It seems that everybody misses the part of the OP where he states that > conflict resolution shouldn't happen via "one dict wins" but rather with > the "or"-operator. I certainly did :-( So we have *at least* four different ways to merge dictionaries a and b: # 1: a wins c = b.copy() c.update(a) # 2: b wins c = a.copy() c.update(b) # 3: choose a winner according to the `or` operator c = a.copy() for key, value in b.items(): if key in c: c[key] = c[key] or value else: c[key] = value # 4: keep both, in a list of 1 or 2 items c = {key:[value] for key, value in a.items()} for key, value in b.items(): if key in c and value != c[key][0]: c[key].append(value) else: c[key] = [value] The first three are special cases of a more general case, where you have a "decision function" that takes two values (one from dict a and the other from dict b) and decides which one to keep. Case 1 ("a always wins") would use `lambda x,y: x`, case 2 ("b wins") would use `lambda x,y: y` and case 3 would use operator.or_. The question is, why should any one of these be picked out as so obviously more useful than the others as to deserve being a dict method or operator support? -- Steven From greg.ewing at canterbury.ac.nz Fri Feb 21 23:58:40 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 22 Feb 2014 11:58:40 +1300 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: <20140221144551.19aed69e@fsol> References: <20140221102403.GV3684@ando> <53074E98.4020205@egenix.com> <88C967DE-DC83-4456-8B72-1FCA5577D381@masklinn.net> <20140221144551.19aed69e@fsol> Message-ID: <5307DA20.1090403@canterbury.ac.nz> Antoine Pitrou wrote: > But on the contrary, trying to compress control flow in a single line > breaks the visual structure expectation for Python code (in which most > control flow is introduced by a colon and a linefeed). We've had control flow in expressions for a long time. The 'and' and 'or' operators affect control flow, but nobody complains that they "break the visual flow". Quite the opposite: they're useful because they *maintain* the visual flow. They allow the code to be read declaratively without thinking in terms of control flow. The same thing applies to the if-expression, and comprehensions, and the proposed except-expression. They're attractive because they expand the range of code that can be written in a declarative rather than an imperative style. Often they allow something to be written on a single line that would otherwise require multiple lines, but that's not the only reason for using them. In fact, I would say that using them *just* to save space is often a bad idea -- if the code isn't easier to read when written declaratively, then you shouldn't be writing it that way, even if it is shorter. -- Greg From abarnert at yahoo.com Sat Feb 22 00:02:35 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 21 Feb 2014 15:02:35 -0800 (PST) Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: <5307D430.5080608@canterbury.ac.nz> References: <20140221102403.GV3684@ando> <5307D430.5080608@canterbury.ac.nz> Message-ID: <1393023755.30153.YahooMailNeo@web181001.mail.ne1.yahoo.com> First, given how often this comes up: Is it worth reopening PEP 3142 to at least add the additional considerations raised over the past 5 years, and maybe have Nick or someone write a?better rejection message than a link to Guido's "I didn't know there was a PEP for that. I hereby reject it."? From: Greg Ewing Sent: Friday, February 21, 2014 2:33 PM > Nick Coghlan wrote: >> ? ? isprime = all(n % p for p in takewhile((: ? ** 2 < n), primes_seen)) > > -1, this is far too cryptic and ugly to have a place in Python. > > I'd be +0 on adding a while clause to for-loops, in > *both* comprehension and statement forms, so that the > above could be written > > ? isprime = all(n % p for p in primes_seen while p ** 2 < n) > > Adding a corresponding clause to the for-statement > would allow the semantic definition of list comprehensions > in terms of the statement expansion to be preserved. As I mentioned somewhere else, all of the languages that have the equivalent of while in comprehensions do it this way: the while is not a comprehension clause in its own, following the usual nesting rules like for and if, it's a modifier to the for clause. (In those languages, that's _also_ how if in comprehensions works, but there's no reason Python couldn't have a hybrid design, where while works on the Clojure/Racket model and if on the Miranda/Haskell model.) So: ? ? (n % p for p in primes_seen while p ** 2 < n if p != 2) ? does not map to the nonsensical: ? ? for p in primes_seen: ? ? ? ? while p ** 2 < n: ? ? ? ? ? ? if p != 2: ? ? ? ? ? ? ? ? yield n % p ? but to the sensible (and now legal, under the proposal): ? ? for p in primes_seen while p ** 2 < n: ? ? ? ? if p != 2: ? ? ? ? ? ? yield n % p > However, this all hinges on whether there is enough > use for the feature to be worth adding. I'm yet to > be convinced of that. I think an off-hand comment Nick made the last time around is what convinced me we probably don't need this (and inspired the start of my blog post). Just as an if clause is?a syntactic way of writing filter, a while clause is a syntactic way of writing takewhile. If takewhile isn't even common enough to move from itertools to builtins, is it really common enough to move from a function to special syntax? From steve at pearwood.info Sat Feb 22 00:09:50 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 22 Feb 2014 10:09:50 +1100 Subject: [Python-ideas] is that expensive? In-Reply-To: References: Message-ID: <20140221230950.GX3684@ando> On Fri, Feb 21, 2014 at 06:37:58PM +0100, Liam Marsh wrote: > Hello everyone, > is it possible to create(or tell me its name) a command to evaluate compute > length of a command/procedure? > (in number of: -processor operations(for 32 and 64 bit > processors) > -RAM read and write operations > -hard disk write and read operations > -eventual graphic operations > ) > this may be difficult, but it ables users to optimise their programs. What should this is_this_expensive() of this function return? def func(): if random.random() < 0.5: return "done" else: while True: pass If you can write an algorithm for deciding: (1) what does "expensive" mean? (2) how do you calculate it in advance? then maybe somebody can create a Python function to perform it. But I suspect that it cannot be calculated in advance, there's no single meaningful measurement for "expense", and even if there was, it would be unhelpful and misleading for optimizing programs. Instead of trying to predict in advance how expensive a procedure will be, why don't you run the procedure and measure how expensive it actually is? See the profile module for ways to measure how much time is used by different parts of the code. -- Steven From solipsis at pitrou.net Sat Feb 22 00:10:10 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 22 Feb 2014 00:10:10 +0100 Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword References: <20140221102403.GV3684@ando> <53074E98.4020205@egenix.com> <88C967DE-DC83-4456-8B72-1FCA5577D381@masklinn.net> <20140221144551.19aed69e@fsol> <5307DA20.1090403@canterbury.ac.nz> Message-ID: <20140222001010.21ee4ced@fsol> On Sat, 22 Feb 2014 11:58:40 +1300 Greg Ewing wrote: > Antoine Pitrou wrote: > > But on the contrary, trying to compress control flow in a single line > > breaks the visual structure expectation for Python code (in which most > > control flow is introduced by a colon and a linefeed). > > We've had control flow in expressions for a long time. The > 'and' and 'or' operators affect control flow, but nobody > complains that they "break the visual flow". You can easily think of them as shortcutting operators. Doing the same with "break" isn't possible. Regards Antoine. From abarnert at yahoo.com Sat Feb 22 00:07:15 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 21 Feb 2014 15:07:15 -0800 (PST) Subject: [Python-ideas] Method chaining notation In-Reply-To: <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> Message-ID: <1393024035.64703.YahooMailNeo@web181001.mail.ne1.yahoo.com> From: Masklinn Sent: Friday, February 21, 2014 2:43 PM > On 2014-02-21, at 23:00 , spir wrote: >> Also, I don't find the idea of having a builtin construct for such > hacks a good idea. Libs for which this may be practicle can return self --end of > the story. > > That has two issues though: > > 1. it makes chainability a decision of the library author, the library > ? user gets to have no preference. This means e.g. you can't create > ? a tree of elements in ElementTree in a single expression (AFAIK > ? Element does not take children parameters). With cascading, the > ? user can "chain" a library whose author did not choose to support > ? chaining (in fact with cascading no author would ever need to > ? support chaining again). > > 2. where a return value can make sense (and be useful) the author > ? *must* make a choice. No way to chain `dict.pop()` since it > ? returns the popped value, even if `pop` was only used for its > ? removal-with-shut-up properties. With cascading the user can > ? have his cake and eat it: he gets the return value if he > ? wants it, and can keep "chaining" if he does not care. I think this is almost always a bad thing to do, and a feature that encourages/enables it is a bad feature just for that reason. If you just want to remove an element from a list or a dict, you use a del statement. There's no reason to call pop unless you want to result. Misusing pop to allow you to wedge a statement into an expression is exactly the same as any other misuse, akin to the "os.remove(path) except IOError: None" from the other thread. Look at it this way: If someone wrote this, would you congratulate him on his cleverness, or ask him to fix it before you waste time reviewing his code? ? ? [d.pop(key) for key in get_keys()] From abarnert at yahoo.com Sat Feb 22 00:13:53 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 21 Feb 2014 15:13:53 -0800 (PST) Subject: [Python-ideas] Infix functions In-Reply-To: References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> Message-ID: <1393024433.83604.YahooMailNeo@web181001.mail.ne1.yahoo.com> From: Chris Rebert Sent: Friday, February 21, 2014 2:47 PM > On Fri, Feb 21, 2014 at 2:05 PM, Andrew Barnert > wrote: >> While we're discussing crazy ideas inspired by a combination of a > long-abandoned PEP and Haskell idioms (see the implicit lambda thread), > here's another: arbitrary infix operators: >> >> ? ? a `foo` b == foo(a, b) > > Prior discussion: > https://mail.python.org/pipermail/python-ideas/2007-January/000050.html > > Which resulted in a new item in PEP 3099 ("Things that will Not Change > in Python 3000"; http://legacy.python.org/dev/peps/pep-3099/ ): > ? ? * No more backticks. > ? ? Backticks (`) will no longer be used as shorthand for repr -- but > that doesn't mean they are available for other uses. Even ignoring the > backwards compatibility confusion, the character itself causes too > many problems (in some fonts, on some keyboards, when typesetting a > book, etc). > > > I think people using suboptimal fonts and keyboard layouts should find > better ones... Thanks for pointing that out. OK, some other syntax then, there's no reason it has to be identical to Haskell. We can even go back to the original PEP's version: ? ? a @foo b = foo(a, b) ? or, more likely, come up with something better.?The important question here is whether there's something _worthy_ of new syntax; what that new syntax should be is just bikeshedding. The other main negative consideration from that thread came from mapping custom operators to custom magic methods. I explained in the initial post why I think that's a bad idea, and not necessary. Just map it to a plain old function call, and use generic dispatch if you need to. I think the mathematical example I gave shows how this can be more readable than normal function or method calls: ? ? a @cross b + c @cross (d @dot e) ? ? a.cross(b).add(c.cross(d.dot(e))) ? ? add(cross(a, b), cross(c, dot(d, e)) And the Haskell `catch` example shows how it _might_ be useful for other things, like higher-order function calls, but I haven't come up with a higher-order _Python_ function where it makes sense?and if there isn't one, the proposal is unnecessary. From steve at pearwood.info Sat Feb 22 00:19:24 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 22 Feb 2014 10:19:24 +1100 Subject: [Python-ideas] The Return of Implicit Lambda (Re: Allowing breaks in generator expressions by overloading the while keyword) In-Reply-To: References: <20140221102403.GV3684@ando> <1393014891.79562.YahooMailNeo@web181003.mail.ne1.yahoo.com> <1393020176.19093.YahooMailNeo@web181002.mail.ne1.yahoo.com> Message-ID: <20140221231924.GY3684@ando> Guys, I had to page through *eight pages* of quotes, at least six levels deep, before I found a two-line new comment. Please [snip] unnecessarily quoted text. Leave enough to establish context, every email doesn't need to be a full archive of the entire discussion to this point. On Fri, Feb 21, 2014 at 04:09:47PM -0600, Ryan Gonzalez wrote: [...] > The question mark still looks like Perl. > > What about a reserved word instead? We already have perfectly good syntax for lambdas. Making them even more concise is not important enough to use another keyword, with the consequences of preventing code from using that word in the future, and breaking existing code that uses that word. I like lambda. But trying to make them even shorter and more concise (or if you want to be cynical, more cryptic) is not on my agenda. -1 -- Steven From ncoghlan at gmail.com Sat Feb 22 00:28:00 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 22 Feb 2014 09:28:00 +1000 Subject: [Python-ideas] The Return of Implicit Lambda (Re: Allowing breaks in generator expressions by overloading the while keyword) In-Reply-To: <1393020176.19093.YahooMailNeo@web181002.mail.ne1.yahoo.com> References: <20140221102403.GV3684@ando> <1393014891.79562.YahooMailNeo@web181003.mail.ne1.yahoo.com> <1393020176.19093.YahooMailNeo@web181002.mail.ne1.yahoo.com> Message-ID: On 22 Feb 2014 08:02, "Andrew Barnert" wrote: > > I think the best answer is the one I suggested above--and which I think Nick implicitly assumed because he's smart enough to get there directly: At least initially, just do 0-argument and 1-argument lambdas. As long as you leave the door open for extended syntaxes in a future proposal, we don't need to figure them out yet. Yep, the implicit assumption was that the shorthand syntax could get away with handling just zero-argument functions (for simple callbacks and lazy evaluation) and one-positional-argument functions (for single object callbacks, sorting keys and filtering predicates). More complex cases don't need to be handled, as users have the option to revert to using an explicit lambda or a named function. This helps the proposal achieve one of useful heuristics to apply when judging a new piece of syntactic sugar: in the cases where it applies, it should be clearly superior to the existing alternatives. Once you get to 2+ arguments, it's hard to be clear without giving them names. Cheers, Nick. -------------- next part -------------- An HTML attachment was scrubbed... URL: From grosser.meister.morti at gmx.net Sat Feb 22 01:31:47 2014 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Sat, 22 Feb 2014 01:31:47 +0100 Subject: [Python-ideas] Joining dicts again In-Reply-To: References: Message-ID: <5307EFF3.6020809@gmx.net> Am 2014-02-21 10:40, schrieb haael at interia.pl: > > Hello > > I know this has been mangled thousand times, but let's do it once again. > > Why does Python not have a simple dict joining operator? > >>From what I read, it seems the biggest concern is: which value to pick up if both dicts have the same key. > a = {'x':1} > b = {'x':2} > c = a | b > print(c['x']) # 1 or 2? > > My proposal is: the value should be derermined as the result of the operation 'or'. The 'or' operator returns the first operand that evaluates to boolean True, or the last operand if all are False. > > So, provided we have 2 dicts 'a' and 'b' and c = a | b > > 1. If a key is not present in 'a' nor 'b', then it is not present in c. > 2. If a key is present in 'a' and not in 'b', then c[k] = a[k]. > 3. If a key is present in 'b' and not in 'a', then c[k] = b[k]. > 4. If a key is present both in 'a' and 'b', then c[k] = a[k] or b[k]. > I never had a case where this kind of conflict resolution made sense. Can you show us an example? > > We could also naturally define dict intersection operator using the operator 'and' to pick values. The key would have to be present in both operands. > > Forgive me I wasn't able to read all previous discussions. Was this proposed before? > > Thanks > haael From zuo at chopin.edu.pl Sat Feb 22 01:50:57 2014 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sat, 22 Feb 2014 01:50:57 +0100 Subject: [Python-ideas] Joining dicts again In-Reply-To: <20140221225733.GW3684@ando> References: <3f21074da4e8e68b4863c1f312848a5b@draco.uberspace.de> <20140221225733.GW3684@ando> Message-ID: <6ad8c08ed852328cd071bf2ddc474701@chopin.edu.pl> 21.02.2014 23:57, Steven D'Aprano wrote: > # 1: a wins > c = b.copy() > c.update(a) Sometimes I do: c = dict(b, **a) Cheers. *j From steve at pearwood.info Sat Feb 22 02:13:09 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 22 Feb 2014 12:13:09 +1100 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: Message-ID: <20140222011308.GZ3684@ando> On Sat, Feb 22, 2014 at 04:30:03AM +1100, Chris Angelico wrote: > Yeah, I'm insane, opening another theory while I'm busily championing > a PEP. Completely raving bonkers :-) > In Python, as in most languages, method chaining requires the method > to return its own object. [...] Rather than add syntactic support for this, I'd prefer a wrapper in the standard library. Syntax implies to me that chaining operators is, in some sense, a preferred idiom of the language, and although method chaining is sometimes useful, I don't think that it ought to be preferred. There's a difference between "Python supports this, use this syntax" and "You can do this in Python, import this library and call this function to make it happen". [...] > So here's the proposal. Introduce a new operator to Python, just like > the dot operator but behaving differently when it returns a bound > method. We can possibly use ->, or maybe create a new operator that > currently makes no sense, like .. or .> or something. Its semantics > would be: > > 1) Look up the attribute following it on the object, exactly as per > the current . operator > 2) If the result is not a function, return it, exactly as per current. > 3) If it is a function, though, return a wrapper which, when called, > calls the inner function and then returns self. When you say "a function", do you actually mean a function? What about other callables, such as methods (instance, class or static) or types? How does this interact with the descriptor protocol? > This can be done with an external wrapper, so it might be possible to > do this with MacroPy. It absolutely must be a compact notation, > though. Here's an alternative: add a `chained` wrapper to, oh, let's say functools (I don't think it needs to be a built-in). In my hierarchy of language preferredness, this suggests that while Python *supports* method chaining, it isn't *preferred*. (If it were preferred, it would happen by default, or there would be syntax for it.) If you want to chain methods, you can, but it's a slightly unusual thing to do, and the fact that you have to import a module and call a wrapper class should be sufficient discouragement for casual (ab)use. Inspired by the Ruby "tap" method, I wrote this proof-of-concept last year: https://mail.python.org/pipermail/python-list/2013-November/660892.html http://code.activestate.com/recipes/578770-method-chaining/ Of course, if the class author wants to support method chaining, they can always write the methods to return self :-) -- Steven From greg.ewing at canterbury.ac.nz Sat Feb 22 02:16:43 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 22 Feb 2014 14:16:43 +1300 Subject: [Python-ideas] Infix functions In-Reply-To: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> Message-ID: <5307FA7B.3050707@canterbury.ac.nz> Andrew Barnert wrote: > The Numeric community wanted this as a way of defining new mathematical >operators. ... you could solve the problem easily with infix functions: > > m `cross` n > m `dot` n For me that totally misses one of the main points of infix operators, which is that they're *concise*. There's little advantage in writing the above rather than cross(m, n) dot(m, n) -- Greg From haoyi.sg at gmail.com Sat Feb 22 02:33:02 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Fri, 21 Feb 2014 17:33:02 -0800 Subject: [Python-ideas] Method chaining notation In-Reply-To: <20140222011308.GZ3684@ando> References: <20140222011308.GZ3684@ando> Message-ID: > This can be done with an external wrapper, so it might be possible to do this with MacroPy. It absolutely must be a compact notation, though. Something like: merged = c[ my_dict.update(dict_a).update(dict_b) ] Desugaring into # top-level somewhere elsedef fresh_name(x, *args): y = x for op, a, kw in args: y = getattr(y, op)(*a, **kw) y = x if y is None else y return y merged = fresh_name(my_dict, ("update", [dict_a], {}), ("update", [dict_b], {})) could be doable pretty easily, and is flexible enough to make old-school chaining work while also letting things that return None do the right thing. If you don't want to use macros, an alternative could be doing something like: chain(my_dict).update(dict_a).update(dict_b).get Using a wrapper as Steven mentioned. On Fri, Feb 21, 2014 at 5:13 PM, Steven D'Aprano wrote: > On Sat, Feb 22, 2014 at 04:30:03AM +1100, Chris Angelico wrote: > > Yeah, I'm insane, opening another theory while I'm busily championing > > a PEP. > > Completely raving bonkers :-) > > > In Python, as in most languages, method chaining requires the method > > to return its own object. > [...] > > Rather than add syntactic support for this, I'd prefer a wrapper in the > standard library. Syntax implies to me that chaining operators is, in > some sense, a preferred idiom of the language, and although method > chaining is sometimes useful, I don't think that it ought to be > preferred. There's a difference between "Python supports this, use this > syntax" and "You can do this in Python, import this library and call > this function to make it happen". > > > [...] > > So here's the proposal. Introduce a new operator to Python, just like > > the dot operator but behaving differently when it returns a bound > > method. We can possibly use ->, or maybe create a new operator that > > currently makes no sense, like .. or .> or something. Its semantics > > would be: > > > > 1) Look up the attribute following it on the object, exactly as per > > the current . operator > > 2) If the result is not a function, return it, exactly as per current. > > 3) If it is a function, though, return a wrapper which, when called, > > calls the inner function and then returns self. > > When you say "a function", do you actually mean a function? What about > other callables, such as methods (instance, class or static) or types? > How does this interact with the descriptor protocol? > > > > This can be done with an external wrapper, so it might be possible to > > do this with MacroPy. It absolutely must be a compact notation, > > though. > > Here's an alternative: add a `chained` wrapper to, oh, let's say > functools (I don't think it needs to be a built-in). In my hierarchy of > language preferredness, this suggests that while Python *supports* > method chaining, it isn't *preferred*. (If it were preferred, it would > happen by default, or there would be syntax for it.) If you want to > chain methods, you can, but it's a slightly unusual thing to do, and the > fact that you have to import a module and call a wrapper class should be > sufficient discouragement for casual (ab)use. > > Inspired by the Ruby "tap" method, I wrote this proof-of-concept last > year: > > https://mail.python.org/pipermail/python-list/2013-November/660892.html > > http://code.activestate.com/recipes/578770-method-chaining/ > > Of course, if the class author wants to support method chaining, they > can always write the methods to return self :-) > > > -- > Steven > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From haoyi.sg at gmail.com Sat Feb 22 02:37:34 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Fri, 21 Feb 2014 17:37:34 -0800 Subject: [Python-ideas] Infix functions In-Reply-To: <5307FA7B.3050707@canterbury.ac.nz> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> Message-ID: > For me that totally misses one of the main points of > infix operators, which is that they're *concise*. There's > little advantage in writing the above rather than On the other hand, it does give you the other reason for infix operators, which is that the evaluation order of the things happening flow from left to right, rather than right-to-left or around in a spiral. On Fri, Feb 21, 2014 at 5:16 PM, Greg Ewing wrote: > Andrew Barnert wrote: > >> The Numeric community wanted this as a way of defining new mathematical >> operators. ... you could solve the problem easily with infix functions: >> >> >> m `cross` n >> m `dot` n >> > > For me that totally misses one of the main points of > infix operators, which is that they're *concise*. There's > little advantage in writing the above rather than > > cross(m, n) > dot(m, n) > > -- > Greg > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rymg19 at gmail.com Sat Feb 22 02:42:00 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Fri, 21 Feb 2014 19:42:00 -0600 Subject: [Python-ideas] Joining dicts again In-Reply-To: <6ad8c08ed852328cd071bf2ddc474701@chopin.edu.pl> References: <3f21074da4e8e68b4863c1f312848a5b@draco.uberspace.de> <20140221225733.GW3684@ando> <6ad8c08ed852328cd071bf2ddc474701@chopin.edu.pl> Message-ID: Which opens up the same issue I mentioned earlier: non-string keys don't work. On Fri, Feb 21, 2014 at 6:50 PM, Jan Kaliszewski wrote: > 21.02.2014 23:57, Steven D'Aprano wrote: > > # 1: a wins >> c = b.copy() >> c.update(a) >> > > Sometimes I do: > > c = dict(b, **a) > > Cheers. > *j > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From grosser.meister.morti at gmx.net Sat Feb 22 02:45:30 2014 From: grosser.meister.morti at gmx.net (=?UTF-8?B?TWF0aGlhcyBQYW56ZW5iw7Zjaw==?=) Date: Sat, 22 Feb 2014 02:45:30 +0100 Subject: [Python-ideas] Infix functions In-Reply-To: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> Message-ID: <5308013A.4070309@gmx.net> Am 2014-02-21 23:05, schrieb Andrew Barnert: > While we're discussing crazy ideas inspired by a combination of a long-abandoned PEP and Haskell idioms (see the implicit lambda thread), here's another: arbitrary infix operators: > > a `foo` b == foo(a, b) > If you really want to you could do this: class infix(object): __slots__ = '__call__', def __init__(self,func): self.__call__ = func def __ror__(self,arg1): return infix2(self.__call__, arg1) class infix2(object): __slots__ = 'func', 'arg1' def __init__(self,func,arg1): self.func = func self.arg1 = arg1 def __call__(*args,**kwargs): self, args = args[0], args[1:] return self.func(self.arg1,*args,**kwargs) def __or__(self,arg2): return self.func(self.arg1,arg2) @infix def foo(a,b): return a + b print "egg" |foo| "spam" From abarnert at yahoo.com Sat Feb 22 02:58:39 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 21 Feb 2014 17:58:39 -0800 (PST) Subject: [Python-ideas] Infix functions In-Reply-To: References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> Message-ID: <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> Apologies for the mess I'm likely to make trying to reply to a top-post? From Greg Ewing: >>?For me that totally misses one of the main points of >> infix operators, which is that they're *concise*. There's >> little advantage in writing the above rather than No, it's not about conciseness, it's about readability. I included longer examples in my original post, and repeated them again because people apparently missed them, but here's one of the examples yet again: ? ? a `cross` b + c `cross` (d `dot` e) vs. ? ? add(cross(a, b), cross(c, dot(d, e)) Sure, it's only one character shorter (or four characters using @cross instead of `cross`), but you can't tell me it's not more readable. The function-calling version is out of order, and has an extra level of nesting, and has a lot more syntactic noise. And look at my motivating example from Haskell: the infix `catch` is not any more _concise_ than the prefix version, but it puts things in the right order (the same order as the proposed exception expression for Python)?and that's why Haskell doesn't need an exception expression. (Well, that plus the ability to create functions from most expressions without lambda, but that's a separate issue.) Again, I'm not sure there are many such cases in Python (after all, 2-argument higher-order functions make up half of Haskell's library, while Python only has a handful of them?), but if there are, they'd benefit the same way. From Haoyi Li: >On the other hand, it does give you the other reason for infix operators, which is that the evaluation order of the things happening flow from left to right, rather than right-to-left or around in a spiral. Exactly. From rymg19 at gmail.com Sat Feb 22 03:02:58 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Fri, 21 Feb 2014 20:02:58 -0600 Subject: [Python-ideas] Infix functions In-Reply-To: <5308013A.4070309@gmx.net> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5308013A.4070309@gmx.net> Message-ID: You should contribute that to PyExt(https://github.com/kirbyfan64/pyext). I'd love to have it there. On Fri, Feb 21, 2014 at 7:45 PM, Mathias Panzenb?ck < grosser.meister.morti at gmx.net> wrote: > Am 2014-02-21 23:05, schrieb Andrew Barnert: > > While we're discussing crazy ideas inspired by a combination of a >> long-abandoned PEP and Haskell idioms (see the implicit lambda thread), >> here's another: arbitrary infix operators: >> >> a `foo` b == foo(a, b) >> >> > If you really want to you could do this: > > class infix(object): > __slots__ = '__call__', > > def __init__(self,func): > self.__call__ = func > > def __ror__(self,arg1): > return infix2(self.__call__, arg1) > > class infix2(object): > __slots__ = 'func', 'arg1' > > def __init__(self,func,arg1): > self.func = func > self.arg1 = arg1 > > def __call__(*args,**kwargs): > self, args = args[0], args[1:] > return self.func(self.arg1,*args,**kwargs) > > def __or__(self,arg2): > return self.func(self.arg1,arg2) > > @infix > def foo(a,b): > return a + b > > print "egg" |foo| "spam" > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Sat Feb 22 03:12:20 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 22 Feb 2014 15:12:20 +1300 Subject: [Python-ideas] Infix functions In-Reply-To: <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> Message-ID: <53080784.3040801@canterbury.ac.nz> Andrew Barnert wrote: > a `cross` b + c `cross` (d `dot` e) > > vs. > > add(cross(a, b), cross(c, dot(d, e)) > > Sure, it's only one character shorter (or four characters using @cross > instead of `cross`), but you can't tell me it's not more readable. It's *slightly* better, *maybe*, but it still looks like a mess to my eyes, compared to a % b + c % (d * e) There are several reasons for this version being more readable: * It's far more compact * It's not cluttered up with backticks * It leverages my existing intuition about the relative precedences of the operators involved Precedence is a big problem. You're assuming that backtick operators would have precedence higher than '+'. What if I want to define my own addition-like operator? One precedence isn't going to fit all. -- Greg From antony.lee at berkeley.edu Sat Feb 22 03:13:04 2014 From: antony.lee at berkeley.edu (Antony Lee) Date: Fri, 21 Feb 2014 18:13:04 -0800 Subject: [Python-ideas] Make MappingProxyType picklable Message-ID: Currently, mappingproxies are not picklable, which (among other things) lead to signature objects being non picklable. Could this be easily fixed, or is there a stronger reason for not allowing that pickling? If mappingproxies cannot be made picklable, perhaps signatures can implement __getnewargs__ to allow pickling? Antony -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Sat Feb 22 03:16:23 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 21 Feb 2014 18:16:23 -0800 (PST) Subject: [Python-ideas] Infix functions In-Reply-To: <5308013A.4070309@gmx.net> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5308013A.4070309@gmx.net> Message-ID: <1393035383.58231.YahooMailNeo@web181005.mail.ne1.yahoo.com> From: Mathias Panzenb?ck Sent: Friday, February 21, 2014 5:45 PM >Am 2014-02-21 23:05, schrieb Andrew Barnert: >> While we're discussing crazy ideas inspired by a combination of a long-abandoned PEP and Haskell idioms (see the implicit lambda thread), here's another: arbitrary infix operators: >> >>? ? ? a `foo` b == foo(a, b) >> > >If you really want to you could do this: This is basically the same trick you use in Haskell to allow arbitrary expressions rather than just identifiers to be used infix. Which I don't think you actually want to allow. Also, it requires you to declare certain functions as infix-able rather than just use any functions arbitrarily. Of course that does allow you to give the infix version a different name than the prefix version, like bar = infix(foo), or even bar = infix(partial(Spam(2).long_method_name, 'eggs')), which could I suppose be useful. One question about the implementation: >??? class infix2(object): >??? ??? __slots__ = 'func', 'arg1' >??? >??? ??? def __init__(self,func,arg1): >??? ??? ??? self.func = func >??? ??? ??? self.arg1 = arg1 >??? >??? ??? def __call__(*args,**kwargs): >??? ??? ??? self, args = args[0], args[1:] >??? ??? ??? return self.func(self.arg1,*args,**kwargs) First, why not just: ? ? def __call__(self, *args, **kwargs): ? ? ? ? return self.func(self.arg1, *args, **kwargs) More importantly, I assume this is to allow auto-partialing, so "spam" |foo is a normal function of one argument that you can pass around? That doesn't actually work the same way as Haskell operators, and it looks pretty weird in Python too, and I can't think of when you'd want it, but it is clever. From grosser.meister.morti at gmx.net Sat Feb 22 03:34:58 2014 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Sat, 22 Feb 2014 03:34:58 +0100 Subject: [Python-ideas] Infix functions In-Reply-To: <1393035383.58231.YahooMailNeo@web181005.mail.ne1.yahoo.com> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5308013A.4070309@gmx.net> <1393035383.58231.YahooMailNeo@web181005.mail.ne1.yahoo.com> Message-ID: <53080CD2.6020100@gmx.net> Am 2014-02-22 03:16, schrieb Andrew Barnert: > From: Mathias Panzenb?ck > Sent: Friday, February 21, 2014 5:45 PM > > >> Am 2014-02-21 23:05, schrieb Andrew Barnert: >>> While we're discussing crazy ideas inspired by a combination of a long-abandoned PEP and Haskell idioms (see the implicit lambda thread), here's another: arbitrary infix operators: >>> >>> a `foo` b == foo(a, b) >>> >> >> If you really want to you could do this: > > > This is basically the same trick you use in Haskell to allow arbitrary expressions rather than just identifiers to be used infix. Which I don't think you actually want to allow. Also, it requires you to declare certain functions as infix-able rather than just use any functions arbitrarily. Of course that does allow you to give the infix version a different name than the prefix version, like bar = infix(foo), or even bar = infix(partial(Spam(2).long_method_name, 'eggs')), which could I suppose be useful. > > One question about the implementation: > >> class infix2(object): >> __slots__ = 'func', 'arg1' >> >> def __init__(self,func,arg1): >> self.func = func >> self.arg1 = arg1 >> >> def __call__(*args,**kwargs): >> self, args = args[0], args[1:] >> return self.func(self.arg1,*args,**kwargs) > > > First, why not just: > > def __call__(self, *args, **kwargs): > return self.func(self.arg1, *args, **kwargs) > Because then you can never pass a keyword argument named "self". Exotic case, but still. > More importantly, I assume this is to allow auto-partialing, so "spam" |foo is a normal function of one argument that you can pass around? That doesn't actually work the same way as Haskell operators, and it looks pretty weird in Python too, and I can't think of when you'd want it, but it is clever. > Yeah, I just thought maybe one would like to have that. No special thoughts there. Also this is not my idea. I saw such code somewhere and wrote this down from memory. It could even be that I saw this on this mailing list. Btw. bugfix for the infix class: class infix(object): __slots__ = 'func', def __init__(self,func): self.func = func def __ror__(self,arg1): return infix2(self.func, arg1) def __call__(*args,**kwargs): self, args = args[0], args[1:] return self.func(*args,**kwargs) From abarnert at yahoo.com Sat Feb 22 03:45:09 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 21 Feb 2014 18:45:09 -0800 (PST) Subject: [Python-ideas] Infix functions In-Reply-To: <53080784.3040801@canterbury.ac.nz> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> Message-ID: <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> From: Greg Ewing Sent: Friday, February 21, 2014 6:12 PM > Andrew Barnert wrote: >> ? ? a `cross` b + c `cross` (d `dot` e) >> >> vs. >> >> ? ? add(cross(a, b), cross(c, dot(d, e)) >> >> Sure, it's only one character shorter (or four characters using @cross >> instead? of `cross`), but you can't tell me it's not more readable. > > It's *slightly* better, *maybe*, but it still looks > like a mess to my eyes, compared to > > ? a % b + c % (d * e) > > There are several reasons for this version being more > readable: > > * It's far more compact > * It's not cluttered up with backticks > * It leverages my existing intuition about the relative > ? precedences of the operators involved Sure, having an infinite number of easily-distinguishable operators that happen to have the precedence that fits your use case would be create. But we have a very limited set of operators. If you want element-wise multiplication, div, trued, and mod, there aren't any operators left with the right precedence for cross-multiplication, dot-product, and back-division. And even if that weren't an issue, using % to mean mod sometimes and cross-product other times is bound to be confusing to readers. So, the perfect option doesn't exist. Backtick operators always let you write things in order, sometimes but not always let you avoid parentheses, reduce visual noise but don't dispel it completely, and do little or nothing for conciseness. They're not perfect, but they are an improvement. > Precedence is a big problem. You're assuming that backtick > operators would have precedence higher than '+'. What if > I want to define my own addition-like operator? One > precedence isn't going to fit all. This is another thing I already discussed this in my initial email, and I don't want to repeat myself again. They should all have the same precedence,?precisely _because_ they all have the same "visual weight", and no in-built intuitions. Try writing up some examples mixing it with different operators, and I think that, no matter what the name is, they all look like they should bind the same way: ? ? 1 * 2 `plus` 3 * 4 == plus(1*2, 3*4), not 1 * plus(2, 3) * 4 ? ? 1 + 2 `times` 3 + 4 == times(1+2, 3+4), not 1 + times(2, 3) + 4 ? or ? ? ? 1 < 2 `plus` 3 == 1 < plus(2, 3), not plus(1<2, 3) ? ? 1 < 2 `times` 3 == 1 < times(2, 3) You can argue about what exactly the precedence should be, but whatever it is, I'm convinced it should be fixed. (In Haskell, there's a default, but it's customizable. And people do create little one-letter functions and infix them with custom precedence and convince themselves that `p` looks like it belongs at a different level than `g`, but I think that's silly in Haskell, and would be even more so in Python.) From yselivanov.ml at gmail.com Sat Feb 22 04:26:43 2014 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Fri, 21 Feb 2014 22:26:43 -0500 Subject: [Python-ideas] Make MappingProxyType picklable In-Reply-To: References: Message-ID: <530818F3.6030602@gmail.com> Antony, On 2/21/2014, 9:13 PM, Antony Lee wrote: > Currently, mappingproxies are not picklable, which (among other things) > lead to signature objects being non picklable. Could this be easily fixed, > or is there a stronger reason for not allowing that pickling? > If mappingproxies cannot be made picklable, perhaps signatures can > implement __getnewargs__ to allow pickling? > Antony > Good catch. I think we'll make Signatures picklable in 3.5 (http://bugs.python.org/issue20726) It can be implemented using __reduce__ and __setstate__ methods. We can't use __getnewargs__ as we have keyword-only arguments in Parameter and Signature classes (but maybe we'll be able to use __getnewargs_ex__ in 3.5) Do you have any other reasons to make mappingproxies picklable? Yury From greg.ewing at canterbury.ac.nz Sat Feb 22 04:28:00 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 22 Feb 2014 16:28:00 +1300 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: Message-ID: <53081940.3010505@canterbury.ac.nz> Chris Angelico wrote: > In Python, as in most languages, method chaining requires the method > to return its own object. I don't think Python has much need for method chaining. Most uses of it I've seen in other languages are better addressed in other ways in Python. E.g. a pattern often used in Java for initialising objects: b = new BlockStone().setHardness(1.5F) .setResistance(10.0F).setStepSound(soundTypePiston) .setBlockName("stone").setBlockTextureName("stone") is expressed much more Pythonically as b = BlockStone(hardness = 1.5, resistance = 10.0, step_sound = sound_type_piston, name = "stone", texture_name = "stone") > window = gtk.Window(gtk.WINDOW_TOPLEVEL) > window.set_title("Hello Buttons!") > window.connect("delete_event", delete_event) > window.set_border_width(10) > box1 = gtk.HBox(False, 0) > window.add(box1) > button1 = gtk.Button("Button 1") > button1.connect("clicked", callback, "button 1") > box1.pack_start(button1, True, True, 0) > button2 = gtk.Button("Button 2") > button2.connect("clicked", callback, "button 2") > box1.pack_start(button2, True, True, 0) > window.show_all() I think this is a symptom of bad API design. A more Pythonic way to write that would be window = Window(style = TOPLEVEL, title = "Hello Buttons!", on_delete = delete_event, border_width = 10, HBox(False, 0, [ Button("Button 1", on_clicked = callback), Button("Button 2", on_clicked = callback), ]) ) -- Greg From rosuav at gmail.com Sat Feb 22 05:31:35 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 22 Feb 2014 15:31:35 +1100 Subject: [Python-ideas] Method chaining notation In-Reply-To: <20140222011308.GZ3684@ando> References: <20140222011308.GZ3684@ando> Message-ID: On Sat, Feb 22, 2014 at 12:13 PM, Steven D'Aprano wrote: > On Sat, Feb 22, 2014 at 04:30:03AM +1100, Chris Angelico wrote: >> So here's the proposal. Introduce a new operator to Python, just like >> the dot operator but behaving differently when it returns a bound >> method. We can possibly use ->, or maybe create a new operator that >> currently makes no sense, like .. or .> or something. Its semantics >> would be: >> >> 1) Look up the attribute following it on the object, exactly as per >> the current . operator >> 2) If the result is not a function, return it, exactly as per current. >> 3) If it is a function, though, return a wrapper which, when called, >> calls the inner function and then returns self. > > When you say "a function", do you actually mean a function? What about > other callables, such as methods (instance, class or static) or types? > How does this interact with the descriptor protocol? Well, I mean anything that can become a bound method. At some point, a lookup is done and it returns a function (in the normal case), but I guess presumably any callable will do at that point. I'm looking at hooking in at the point where it becomes bound. A bound method is a function wrapper that provides a 'self' argument. A chaining bound method would be a function wrapper that provides a 'self' argument, and then returns it. > Here's an alternative: add a `chained` wrapper to, oh, let's say > functools (I don't think it needs to be a built-in). In my hierarchy of > language preferredness, this suggests that while Python *supports* > method chaining, it isn't *preferred*. (If it were preferred, it would > happen by default, or there would be syntax for it.) If you want to > chain methods, you can, but it's a slightly unusual thing to do, and the > fact that you have to import a module and call a wrapper class should be > sufficient discouragement for casual (ab)use. Downside of that is that it's really verbose. This sort of thing is useful only if it's short. It's like doing a filtered iteration: for spam in can if tasty: eat(spam) for spam in filter(lambda x: tasty, can): eat(spam) Using filter() is so verbose that there has been and will continue to be a stream of requests for a "filtered for loop" syntax. Having to pass everything through a named wrapper is too wordy to be useful. ChrisA From rosuav at gmail.com Sat Feb 22 05:26:05 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 22 Feb 2014 15:26:05 +1100 Subject: [Python-ideas] Method chaining notation In-Reply-To: <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> Message-ID: On Sat, Feb 22, 2014 at 9:43 AM, Masklinn wrote: > On 2014-02-21, at 23:00 , spir wrote: >> Also, I don't find the idea of having a builtin construct for such hacks a good idea. Libs for which this may be practicle can return self --end of the story. > > That has two issues though: > > 1. it makes chainability a decision of the library author, the library > user gets to have no preference. This means e.g. you can't create > a tree of elements in ElementTree in a single expression (AFAIK > Element does not take children parameters). With cascading, the > user can "chain" a library whose author did not choose to support > chaining (in fact with cascading no author would ever need to > support chaining again). Right. That's the main point behind this: it gives the *caller* the choice of whether to chain or not. That's really the whole benefit, right there. ChrisA From rosuav at gmail.com Sat Feb 22 05:40:59 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 22 Feb 2014 15:40:59 +1100 Subject: [Python-ideas] Method chaining notation In-Reply-To: <5A29A7CC-2218-47CB-990A-8F9A48D63141@masklinn.net> References: <5A29A7CC-2218-47CB-990A-8F9A48D63141@masklinn.net> Message-ID: On Sat, Feb 22, 2014 at 9:31 AM, Masklinn wrote: > On 2014-02-21, at 18:30 , Chris Angelico wrote: >> So here's the proposal. Introduce a new operator to Python, just like >> the dot operator but behaving differently when it returns a bound >> method. We can possibly use ->, or maybe create a new operator that >> currently makes no sense, like .. or .> or something. Its semantics >> would be: > > As Yuri noted the concept exists, AFAIK it was introduced by smalltalk > as "message cascading". Basically, the ability to send a sequence of > messages to the same subject without having to repeatedly specify the > subject. I believe Dart is the first language to have resurrected this > feature so far. Cascading is what I'm looking for here, yes. As noted in the Wiki page Yuri linked to, chaining-with-return-self enables cascading. Consider this to be a proposal to add method cascading to Python. Also, since Dart uses .., that's a good reason to use .. here too. >> 1) Look up the attribute following it on the object, exactly as per >> the current . operator >> 2) If the result is not a function, return it, exactly as per current. >> 3) If it is a function, though, return a wrapper which, when called, >> calls the inner function and then returns self. > > I could be wrong, but I'm pretty sure this is an over-complication > when you look at it at the bytecode level: you can load the subject > as many times as you've got attr accesses to do on it, or you could > have an alternate attr access which puts TOS back. No need for a > wrapper. That would be a job for the peephole optimizer. What happens if you do this: func = x..y # more code func().z It can't just leave x on the stack, but it has to have the same semantics. But I agree, the DUP_TOP form would be excellent for the common case: > No need for a wrapper. Where `a.b` compiles to > > LOAD_FAST a > LOAD_ATTR b > POP_TOP > > `a->b` would compile to > > LOAD_FAST a > DUP_TOP > LOAD_ATTR b > POP_TOP > > at this point you've got an a left on the stack > and can reuse it: > > `a->b()->c()->d()` would be > > LOAD_FAST a > > DUP_TOP > LOAD_ATTR b > CALL_FUNCTION > POP_TOP > > DUP_TOP > LOAD_ATTR c > CALL_FUNCTION > POP_TOP > > DUP_TOP > LOAD_ATTR d > CALL_FUNCTION > POP_TOP > > Or maybe it would do nothing special and the cascade would yield (or > pop) the subject unless closed by an attribute access or regular method > call. That would avoid the requirement of a `yourself`-type method when > initialising mutables, although the final irregularity may lack > visibility. Yes, it would do that. If you use .. everywhere, then the end result of the whole expression should be the original object. In Pike GTK, where most methods return themselves, I can do this: object window = GTK2.Window(0) ->set_title("Title!") ->add(some_object) ->show_all(); The return value from show_all() is the original window. With explicit method cascading, I could either capture the return value of the last function call by choosing _not_ to use cascading there, or I could capture the original object by continuing the cascade. (In the case of GUI work like this, I'd default to cascading, if I were not using the result of the expression. It'd mean that adding or deleting lines of code wouldn't risk changing anything - it's like permitting a trailing comma in a tuple/list.) ChrisA From stephen at xemacs.org Sat Feb 22 05:50:43 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 22 Feb 2014 13:50:43 +0900 Subject: [Python-ideas] The Return of Implicit Lambda (Re: Allowing breaks in generator expressions by overloading the while keyword) In-Reply-To: <20140221231924.GY3684@ando> References: <20140221102403.GV3684@ando> <1393014891.79562.YahooMailNeo@web181003.mail.ne1.yahoo.com> <1393020176.19093.YahooMailNeo@web181002.mail.ne1.yahoo.com> <20140221231924.GY3684@ando> Message-ID: <87vbw7vjh8.fsf@uwakimon.sk.tsukuba.ac.jp> Steven D'Aprano writes: > Guys, I had to page through *eight pages* of quotes, at least six levels > deep, before I found a two-line new comment. Indeed, if you're going to top-post, please do it at the *top*. From stephen at xemacs.org Sat Feb 22 05:57:07 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 22 Feb 2014 13:57:07 +0900 Subject: [Python-ideas] Infix functions In-Reply-To: References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> Message-ID: <87txbrvj6k.fsf@uwakimon.sk.tsukuba.ac.jp> Chris Rebert writes: > I think people using suboptimal fonts and keyboard layouts should find > better ones... Replacement is not yet an option for people with suboptimal eyes and fingers. From ericsnowcurrently at gmail.com Sat Feb 22 06:14:58 2014 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Fri, 21 Feb 2014 22:14:58 -0700 Subject: [Python-ideas] getitem() (was Re: [Python-Dev] PEP 463: Exception-catching expressions) Message-ID: (in response to https://mail.python.org/pipermail/python-dev/2014-February/132648.html) On Fri, Feb 21, 2014 at 9:25 PM, Yury Selivanov wrote: > On 2/21/2014, 10:42 PM, Chris Angelico wrote: >> That fixes it for the list. Square brackets notation works for any >> sequence. Are you proposing adding magic to the language so that any >> class that defines a __getitem__ automatically has a .get() with these >> semantics? Or, conversely, that defining __getitem__ requires that you >> also explicitly define get()? > > > No, I proposed to consider adding 'list.get()' and > 'collections.abc.Sequence.get()' (or a better name), > if catching IndexError is such a popular pattern. > > We can also fix stdlib. Authors of the python libraries/ > frameworks will likely follow. > > No magic in the language, no requirements on __getitem__, > obviously. We could also add a companion to getattr: getitem(). def getitem(seq, index, default=_NOT_SET): try: return seq[index] except IndexError: if default is _NOT_SET: raise return default Of course, one might expect getitem() to work for mappings too (or we'd have 2 distinct functions for mappings and sequences). Having getitem() work for both would not be too hard. Keep in mind that I still think that the PEP 463 syntax is worth adding more than any getitem() function. However, I think the getitem() route would be better than adding another method to sequences (including list). -eric From steve at pearwood.info Sat Feb 22 06:16:27 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 22 Feb 2014 16:16:27 +1100 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <20140222011308.GZ3684@ando> Message-ID: <20140222051627.GF3684@ando> On Sat, Feb 22, 2014 at 03:31:35PM +1100, Chris Angelico wrote: > > Here's an alternative: add a `chained` wrapper to, oh, let's say > > functools (I don't think it needs to be a built-in). In my hierarchy of > > language preferredness, this suggests that while Python *supports* > > method chaining, it isn't *preferred*. (If it were preferred, it would > > happen by default, or there would be syntax for it.) If you want to > > chain methods, you can, but it's a slightly unusual thing to do, and the > > fact that you have to import a module and call a wrapper class should be > > sufficient discouragement for casual (ab)use. > > Downside of that is that it's really verbose. This sort of thing is > useful only if it's short. It's like doing a filtered iteration: It's not that long: chained() only adds nine characters, and if you're worried about that, call it chain() instead (seven). With your syntax, every dot lookup takes two chars instead of one, so it only takes eight method calls before my syntax is shorter than yours. # unrealistically short method names just so the example fits on one line obj->f()->g()->h()->i()->j()->k()->l()->m() chain(obj).f().g().h().i().j().k().l().m() In practice, I would expect that you would want to split the chain across multiple lines, just for readability, and if you're approaching a chain ten methods long, your code probably needs a rethink. > > for spam in can if tasty: > eat(spam) > > for spam in filter(lambda x: tasty, can): > eat(spam) The number of characters isn't really relevent there. If that was written: for spam in f(len, can): eat(spam) which is shorter than "for spam in can if len(spam)", people would still want the filtered syntax. The problem is to think in terms of higher-order functions, and that doesn't come easily to most people. > Using filter() is so verbose that there has been and will continue to > be a stream of requests for a "filtered for loop" syntax. Having to > pass everything through a named wrapper is too wordy to be useful. Not so -- if the alternative is: obj.f() obj.g() obj.h() obj.i() obj.j() obj.k() obj.l() obj.m() my version with chain() or chained() or even enable_chained_methods() wins hands down. Given that Guido dislikes chaining methods, I don't think he'll want to make it too easy to chain methods :-) -- Steven From haoyi.sg at gmail.com Sat Feb 22 06:18:24 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Fri, 21 Feb 2014 21:18:24 -0800 Subject: [Python-ideas] Infix functions In-Reply-To: <87txbrvj6k.fsf@uwakimon.sk.tsukuba.ac.jp> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <87txbrvj6k.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: I like infix functions, but am no fan of the backtick syntax. Of all the things that backticks mean to people: quotes, finger-quotes, code-snippets, interpolate, whatever, "make me infix" is not one of them. Hard to type on a keyboard is almost a non-reason in comparison. I know Haskell does it, but... not that many people have used Haskell. Another idea would be to go the Scala path and turn *methods* into infix operators, by letting you call *point_a.add(point_b)* as *point_a add point_b* or *point_a @add point_b *or some other special sigil. This is semantically different from using infix functions because your infix operators now do not clutter up your namespace, since they would be properties of the LHS object. Not sure if it's "Pythonic", whatever that means, but it's a neat idea that has found a lot of success in the Scala world and I think is superior to the various other mechanisms for infix-functions/operator-overloading that I see in Python/Ruby/C#/F#/etc. On Fri, Feb 21, 2014 at 8:57 PM, Stephen J. Turnbull wrote: > Chris Rebert writes: > > > I think people using suboptimal fonts and keyboard layouts should find > > better ones... > > Replacement is not yet an option for people with suboptimal eyes and > fingers. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Sat Feb 22 06:37:13 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 22 Feb 2014 16:37:13 +1100 Subject: [Python-ideas] Method chaining notation In-Reply-To: <20140222051627.GF3684@ando> References: <20140222011308.GZ3684@ando> <20140222051627.GF3684@ando> Message-ID: On Sat, Feb 22, 2014 at 4:16 PM, Steven D'Aprano wrote: > It's not that long: chained() only adds nine characters, and > if you're worried about that, call it chain() instead (seven). With your > syntax, every dot lookup takes two chars instead of one, so it only > takes eight method calls before my syntax is shorter than yours. > > # unrealistically short method names just so the example fits on one line > obj->f()->g()->h()->i()->j()->k()->l()->m() > chain(obj).f().g().h().i().j().k().l().m() > > In practice, I would expect that you would want to split the chain > across multiple lines, just for readability, and if you're approaching a > chain ten methods long, your code probably needs a rethink. Oh, I see. I thought I'd need to call chain() at each step along the way, in which case it really would be too long. > Not so -- if the alternative is: > > obj.f() > obj.g() > obj.h() > obj.i() > obj.j() > obj.k() > obj.l() > obj.m() > > my version with chain() or chained() or even enable_chained_methods() > wins hands down. It certainly beats that version, yeah! ChrisA From rosuav at gmail.com Sat Feb 22 06:38:54 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 22 Feb 2014 16:38:54 +1100 Subject: [Python-ideas] getitem() (was Re: [Python-Dev] PEP 463: Exception-catching expressions) In-Reply-To: References: Message-ID: On Sat, Feb 22, 2014 at 4:14 PM, Eric Snow wrote: > def getitem(seq, index, default=_NOT_SET): > try: > return seq[index] > except IndexError: > if default is _NOT_SET: > raise > return default > > Of course, one might expect getitem() to work for mappings too (or > we'd have 2 distinct functions for mappings and sequences). Having > getitem() work for both would not be too hard. Very easy. Just catch LookupError or (IndexError, KeyError) instead of IndexError. ChrisA From antony.lee at berkeley.edu Sat Feb 22 07:03:50 2014 From: antony.lee at berkeley.edu (Antony Lee) Date: Fri, 21 Feb 2014 22:03:50 -0800 Subject: [Python-ideas] Make MappingProxyType picklable In-Reply-To: <530818F3.6030602@gmail.com> References: <530818F3.6030602@gmail.com> Message-ID: Not really, although if there is no technical limitation (and if there is one, what is it?), it seems preferable to make mappingproxies picklable rather than use ad-hoc fixes for Signature. Thanks for the quick reply. 2014-02-21 19:26 GMT-08:00 Yury Selivanov : > Antony, > > > On 2/21/2014, 9:13 PM, Antony Lee wrote: > >> Currently, mappingproxies are not picklable, which (among other things) >> lead to signature objects being non picklable. Could this be easily >> fixed, >> or is there a stronger reason for not allowing that pickling? >> If mappingproxies cannot be made picklable, perhaps signatures can >> implement __getnewargs__ to allow pickling? >> Antony >> >> Good catch. I think we'll make Signatures picklable in 3.5 > (http://bugs.python.org/issue20726) It can be implemented > using __reduce__ and __setstate__ methods. We can't use > __getnewargs__ as we have keyword-only arguments in > Parameter and Signature classes (but maybe we'll be able > to use __getnewargs_ex__ in 3.5) > > Do you have any other reasons to make mappingproxies > picklable? > > Yury > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Sat Feb 22 10:04:20 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sat, 22 Feb 2014 01:04:20 -0800 Subject: [Python-ideas] Infix functions In-Reply-To: References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <87txbrvj6k.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Feb 21, 2014, at 21:18, Haoyi Li wrote: > I like infix functions, but am no fan of the backtick syntax. Of all the things that backticks mean to people: quotes, finger-quotes, code-snippets, interpolate, whatever, "make me infix" is not one of them. Hard to type on a keyboard is almost a non-reason in comparison. I know Haskell does it, but... not that many people have used Haskell. > > Another idea would be to go the Scala path and turn methods into infix operators, by letting you call point_a.add(point_b) as point_a add point_b or point_a @add point_b or some other special sigil. This is semantically different from using infix functions because your infix operators now do not clutter up your namespace, since they would be properties of the LHS object. Not sure if it's "Pythonic", whatever that means, but it's a neat idea that has found a lot of success in the Scala world and I think is superior to the various other mechanisms for infix-functions/operator-overloading that I see in Python/Ruby/C#/F#/etc. As I said in the initial message, this is exactly what the Numeric guys suggested in the PEP back in the 2.1 days: "a @foo b" resolves to something like "a.__foo__(b)". They also wanted it to fall back to b.__rfoo__(a)". By "something like" I do not mean "exactly". The original proposal relied on the fact that quoted method names, with non-identifier symbols in them, would replace double underscore names, so you'd have "@foo" and "r at foo" (and today's __add__ would be "+"). Anyway, I think functions make more sense than methods here, but I already explained why and don't want to repeat it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From denis.spir at gmail.com Sat Feb 22 11:23:27 2014 From: denis.spir at gmail.com (spir) Date: Sat, 22 Feb 2014 11:23:27 +0100 Subject: [Python-ideas] Method chaining notation In-Reply-To: <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> Message-ID: <53087A9F.7050100@gmail.com> On 02/21/2014 11:43 PM, Masklinn wrote: > On 2014-02-21, at 23:00 , spir wrote: >> Also, I don't find the idea of having a builtin construct for such hacks a good idea. Libs for which this may be practicle can return self --end of the story. > > That has two issues though: > > 1. it makes chainability a decision of the library author, the library > user gets to have no preference. This means e.g. you can't create > a tree of elements in ElementTree in a single expression (AFAIK > Element does not take children parameters). With cascading, the > user can "chain" a library whose author did not choose to support > chaining (in fact with cascading no author would ever need to > support chaining again). I agree with you, here... > 2. where a return value can make sense (and be useful) the author > *must* make a choice. No way to chain `dict.pop()` since it > returns the popped value, even if `pop` was only used for its > removal-with-shut-up properties. With cascading the user can > have his cake and eat it: he gets the return value if he > wants it, and can keep "chaining" if he does not care. ... not there (if I understand you well; not quite 100% sure). In fact, I find this point rather a counter-argument, something to avoid (again, if I understand). What I mean is that executing given methods should have consistent effect; also, you should use the right method for the right task: if you don't want a stack's top item _and_ have it removed, then don't use 'pop', otherwise you are misleading readers (including yourself maybe, later) (note 'pop' is just a convenience utility for this very case; we could just read and remove in 2 steps). d From ncoghlan at gmail.com Sat Feb 22 15:24:41 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 23 Feb 2014 00:24:41 +1000 Subject: [Python-ideas] The Return of Implicit Lambda (Re: Allowing breaks in generator expressions by overloading the while keyword) In-Reply-To: <87vbw7vjh8.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20140221102403.GV3684@ando> <1393014891.79562.YahooMailNeo@web181003.mail.ne1.yahoo.com> <1393020176.19093.YahooMailNeo@web181002.mail.ne1.yahoo.com> <20140221231924.GY3684@ando> <87vbw7vjh8.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On 22 Feb 2014 14:51, "Stephen J. Turnbull" wrote: > > Steven D'Aprano writes: > > > Guys, I had to page through *eight pages* of quotes, at least six levels > > deep, before I found a two-line new comment. > > Indeed, if you're going to top-post, please do it at the *top*. FWIW, when I do it, it's mostly due to a combination of the Gmail UI hiding quoted text by default and the phone interface making deleting large blocks of text annoying. Agreed it needs to be kept under control, but it's unfortunately easy to slip up and overquote given the default behaviour of at least Gmail and likely other mobile clients :P Cheers, Nick. -------------- next part -------------- An HTML attachment was scrubbed... URL: From masklinn at masklinn.net Sat Feb 22 15:42:07 2014 From: masklinn at masklinn.net (Masklinn) Date: Sat, 22 Feb 2014 15:42:07 +0100 Subject: [Python-ideas] Method chaining notation In-Reply-To: <53087A9F.7050100@gmail.com> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> Message-ID: <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> On 2014-02-22, at 11:23 , spir wrote: > >> 2. where a return value can make sense (and be useful) the author >> *must* make a choice. No way to chain `dict.pop()` since it >> returns the popped value, even if `pop` was only used for its >> removal-with-shut-up properties. With cascading the user can >> have his cake and eat it: he gets the return value if he >> wants it, and can keep "chaining" if he does not care. > > ... not there (if I understand you well; not quite 100% sure). In fact, I find this point rather a counter-argument, something to avoid (again, if I understand). What I mean is that executing given methods should have consistent effect Executing the method always has the same effect on its subject. That it may not be used for the same purpose is a different issue and common: a[k] = v can be used to add a new (k, v) pair or to update a key to a new value (in fact Erlang's new map construct makes the difference and provides for an update-only version). Even more so for values returned by mutating methods, as far as I no there is no rule that they must be used if the method is only called for its side effects. > also, you should use the right method for the right task: if you don't want a stack's top item _and_ have it removed, then don't use 'pop', otherwise you are misleading readers (including yourself maybe, later) (note 'pop' is just a convenience utility for this very case; we could just read and remove in 2 steps). Not only is `dict.pop` an expression (which del is not, and thus del can't be used in some contexts where pop would be useable, e.g. in a lambda) but dict.pop can also handle a non-existent value at the key, doing so with `del` requires adding a supplementary conditional. dict.pop's convenience oft makes it the right method for the right task, even if the case is "remove the key a from the dict if it's there". But if you don't like the pop example there are others. The sibling thread "Joining dicts again" for instance: with cascading it's a non-problem, you can just cascade `update` calls on a base dict and you get a fully correct (as opposed to dict(a, **b) which is only correct if all of b's keys are strings) single-expression union. From oscar.j.benjamin at gmail.com Sat Feb 22 15:49:21 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sat, 22 Feb 2014 14:49:21 +0000 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: <20140218230110.GT4519@ando> <20140220220040.GL3684@ando> Message-ID: On 21 February 2014 18:00, Peter Otten <__peter__ at web.de> wrote: > Oscar Benjamin wrote: > > I think we constantly have to deal with libraries that do almost but not > exactly what we want them to do. If you look at the code it is clear that > the author made a conscious design decision > > @property > def fieldnames(self): > if self._fieldnames is None: > try: > self._fieldnames = next(self.reader) > except StopIteration: > pass > self.line_num = self.reader.line_num > return self._fieldnames > > totally unrelated to for loops catching StopIterations. I was aware of the code. If you look at the commit that made it that way then you can see that the previous implementation was a bare next. It's not clear to me that the behaviour was a design decision or an implementation accident that was propagated for backwards compatibility: $ hg blame Lib/csv.py | grep StopIteration 44735: except StopIteration: $ hg log -p -r 44735 Lib/csv.py --- a/Lib/csv.py Sat Aug 09 12:47:13 2008 +0000 +++ b/Lib/csv.py Sat Aug 09 19:44:22 2008 +0000 @@ -68,7 +68,7 @@ class DictReader: def __init__(self, f, fieldnames=None, restkey=None, restval=None, dialect="excel", *args, **kwds): - self.fieldnames = fieldnames # list of keys for the dict + self._fieldnames = fieldnames # list of keys for the dict self.restkey = restkey # key to catch long rows self.restval = restval # default value for short rows self.reader = reader(f, dialect, *args, **kwds) @@ -78,11 +78,25 @@ def __iter__(self): return self + @property + def fieldnames(self): + if self._fieldnames is None: + try: + self._fieldnames = next(self.reader) + except StopIteration: + pass + self.line_num = self.reader.line_num + return self._fieldnames + + @fieldnames.setter + def fieldnames(self, value): + self._fieldnames = value + def __next__(self): + if self.line_num == 0: + # Used only for its side effect. + self.fieldnames row = next(self.reader) - if self.fieldnames is None: - self.fieldnames = row - row = next(self.reader) self.line_num = self.reader.line_num Peter wrote: > > I fail to see the fundamental difference between next(...) and > sequence[...]. There are edge cases you have to consider, and you add > comments where you expect them to help your readers to understand your > intentions. The difference is that it's not common practice to catch and ignore IndexError around large blocks of code e.g.: try: do_loads_of_stuff() except IndexError: pass However that is in effect what happens for StopIteration since a typical program will have loads of places where it gets silently caught. The difference is that StopIteration is a particularly innocuous error to leak. Cheers, Oscar From alan.cristh at gmail.com Sat Feb 22 16:44:40 2014 From: alan.cristh at gmail.com (Alan Cristhian Ruiz) Date: Sat, 22 Feb 2014 12:44:40 -0300 Subject: [Python-ideas] Method chaining notation In-Reply-To: <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> Message-ID: <5308C5E8.40103@gmail.com> What is wrong with the current sintax?: 'abcd'\ .upper()\ .lower()\ .title() From rosuav at gmail.com Sat Feb 22 16:54:26 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 23 Feb 2014 02:54:26 +1100 Subject: [Python-ideas] Method chaining notation In-Reply-To: <5308C5E8.40103@gmail.com> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> Message-ID: On Sun, Feb 23, 2014 at 2:44 AM, Alan Cristhian Ruiz wrote: > What is wrong with the current sintax?: > > 'abcd'\ > .upper()\ > .lower()\ > .title() It doesn't have each method operate on the original object. It's not easy to see with strings, but try this: list_of_numbers = [1,2] list_of_numbers.append(3) list_of_numbers.append(4) list_of_numbers.append(5) Now write that without repeating list_of_numbers all the way down the line. ChrisA From haoyi.sg at gmail.com Sat Feb 22 16:59:41 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Sat, 22 Feb 2014 07:59:41 -0800 Subject: [Python-ideas] Infix functions In-Reply-To: References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <87txbrvj6k.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: > As I said in the initial message, this is exactly what the Numeric guys suggested in the PEP back in the 2.1 days Ah I missed that point, sorry for the noise On Sat, Feb 22, 2014 at 1:04 AM, Andrew Barnert wrote: > On Feb 21, 2014, at 21:18, Haoyi Li wrote: > > I like infix functions, but am no fan of the backtick syntax. Of all the > things that backticks mean to people: quotes, finger-quotes, code-snippets, > interpolate, whatever, "make me infix" is not one of them. Hard to type on > a keyboard is almost a non-reason in comparison. I know Haskell does it, > but... not that many people have used Haskell. > > Another idea would be to go the Scala path and turn *methods* into infix > operators, by letting you call *point_a.add(point_b)* as *point_a add > point_b* or *point_a @add point_b *or some other special sigil. This is > semantically different from using infix functions because your infix > operators now do not clutter up your namespace, since they would be > properties of the LHS object. Not sure if it's "Pythonic", whatever that > means, but it's a neat idea that has found a lot of success in the Scala > world and I think is superior to the various other mechanisms for > infix-functions/operator-overloading that I see in Python/Ruby/C#/F#/etc. > > > As I said in the initial message, this is exactly what the Numeric guys > suggested in the PEP back in the 2.1 days: "a @foo b" resolves to something > like "a.__foo__(b)". They also wanted it to fall back to b.__rfoo__(a)". > > By "something like" I do not mean "exactly". The original proposal relied > on the fact that quoted method names, with non-identifier symbols in them, > would replace double underscore names, so you'd have "@foo" and "r at foo" > (and today's __add__ would be "+"). > > Anyway, I think functions make more sense than methods here, but I already > explained why and don't want to repeat it. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From __peter__ at web.de Sat Feb 22 17:51:32 2014 From: __peter__ at web.de (Peter Otten) Date: Sat, 22 Feb 2014 17:51:32 +0100 Subject: [Python-ideas] Infix functions References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5308013A.4070309@gmx.net> <1393035383.58231.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080CD2.6020100@gmx.net> Message-ID: Mathias Panzenb?ck wrote: > Also this is not my idea. I saw such code somewhere and wrote this down > from memory. It could even be that I saw this on this mailing list. The first time I saw this was at http://code.activestate.com/recipes/384122-infix-operators/ From elazarg at gmail.com Sat Feb 22 17:52:00 2014 From: elazarg at gmail.com (=?UTF-8?B?15DXnNei15bXqA==?=) Date: Sat, 22 Feb 2014 18:52:00 +0200 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: <20140218230110.GT4519@ando> <20140220220040.GL3684@ando> Message-ID: If an iterators were indexable, with non-decreasing order only, then first(it) would become a simple it[0]. with exactly the same semantics, except the possibilty of raising a (visible) IndexError on an exhausted iterator, instead of . By "non decreasing" I mean the ability to do something like this: it = iter('abcd') for i in range(4): print(it[i]) Of course, it will force the iterator to have additional memory. I am sure this was suggested before, probably many times (and obviously rejected), but not exactly in this specific context. So, maybe allowing only it[0], or maybe generator[0], would be nice (even if it's just an ugly special case"). ---- Elazar 2014-02-22 16:49 GMT+02:00 Oscar Benjamin : > > On 21 February 2014 18:00, Peter Otten <__peter__ at web.de> wrote: > > Oscar Benjamin wrote: > > > > I think we constantly have to deal with libraries that do almost but not > > exactly what we want them to do. If you look at the code it is clear that > > the author made a conscious design decision > > > > @property > > def fieldnames(self): > > if self._fieldnames is None: > > try: > > self._fieldnames = next(self.reader) > > except StopIteration: > > pass > > self.line_num = self.reader.line_num > > return self._fieldnames > > > > totally unrelated to for loops catching StopIterations. > > I was aware of the code. If you look at the commit that made it that > way then you can see that the previous implementation was a bare next. > It's not clear to me that the behaviour was a design decision or an > implementation accident that was propagated for backwards > compatibility: > > $ hg blame Lib/csv.py | grep StopIteration > 44735: except StopIteration: > $ hg log -p -r 44735 Lib/csv.py > > --- a/Lib/csv.py Sat Aug 09 12:47:13 2008 +0000 > +++ b/Lib/csv.py Sat Aug 09 19:44:22 2008 +0000 > @@ -68,7 +68,7 @@ > class DictReader: > def __init__(self, f, fieldnames=None, restkey=None, restval=None, > dialect="excel", *args, **kwds): > - self.fieldnames = fieldnames # list of keys for the dict > + self._fieldnames = fieldnames # list of keys for the dict > self.restkey = restkey # key to catch long rows > self.restval = restval # default value for short rows > self.reader = reader(f, dialect, *args, **kwds) > @@ -78,11 +78,25 @@ > def __iter__(self): > return self > > + @property > + def fieldnames(self): > + if self._fieldnames is None: > + try: > + self._fieldnames = next(self.reader) > + except StopIteration: > + pass > + self.line_num = self.reader.line_num > + return self._fieldnames > + > + @fieldnames.setter > + def fieldnames(self, value): > + self._fieldnames = value > + > def __next__(self): > + if self.line_num == 0: > + # Used only for its side effect. > + self.fieldnames > row = next(self.reader) > - if self.fieldnames is None: > - self.fieldnames = row > - row = next(self.reader) > self.line_num = self.reader.line_num > > > > Peter wrote: > > > > I fail to see the fundamental difference between next(...) and > > sequence[...]. There are edge cases you have to consider, and you add > > comments where you expect them to help your readers to understand your > > intentions. > > The difference is that it's not common practice to catch and ignore > IndexError around large blocks of code e.g.: > > try: > do_loads_of_stuff() > except IndexError: > pass > > However that is in effect what happens for StopIteration since a > typical program will have loads of places where it gets silently > caught. The difference is that StopIteration is a particularly > innocuous error to leak. > > > Cheers, > Oscar > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Sat Feb 22 18:00:59 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 23 Feb 2014 04:00:59 +1100 Subject: [Python-ideas] Function to return first(or last) true value from list In-Reply-To: References: <20140218230110.GT4519@ando> <20140220220040.GL3684@ando> Message-ID: On Sun, Feb 23, 2014 at 3:52 AM, ????? wrote: > If an iterators were indexable, with non-decreasing order only, then > first(it) would become a simple it[0]. with exactly the same semantics, > except the possibilty of raising a (visible) IndexError on an exhausted > iterator, instead of . > > By "non decreasing" I mean the ability to do something like this: > > it = iter('abcd') > for i in range(4): > print(it[i]) > > Of course, it will force the iterator to have additional memory. > I am sure this was suggested before, probably many times (and obviously > rejected), but not exactly in this specific context. > So, maybe allowing only it[0], or maybe generator[0], would be nice (even if > it's just an ugly special case"). You can get that with itertools.tee(), or list(). ChrisA From hannu at krosing.net Sat Feb 22 20:34:51 2014 From: hannu at krosing.net (Hannu Krosing) Date: Sat, 22 Feb 2014 20:34:51 +0100 Subject: [Python-ideas] Infix functions In-Reply-To: <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> Message-ID: <5308FBDB.6030709@krosing.net> On 02/22/2014 03:45 AM, Andrew Barnert wrote: > From: Greg Ewing > > Sent: Friday, February 21, 2014 6:12 PM > > >> Andrew Barnert wrote: >>> a `cross` b + c `cross` (d `dot` e) >>> >>> vs. >>> >>> add(cross(a, b), cross(c, dot(d, e)) >>> >>> Sure, it's only one character shorter (or four characters using @cross >>> instead of `cross`), but you can't tell me it's not more readable. >> It's *slightly* better, *maybe*, but it still looks >> like a mess to my eyes, compared to >> >> a % b + c % (d * e) >> >> There are several reasons for this version being more >> readable: >> >> * It's far more compact >> * It's not cluttered up with backticks >> * It leverages my existing intuition about the relative >> precedences of the operators involved > Sure, having an infinite number of easily-distinguishable operators that happen to have the precedence that fits your use case would be create. But we have a very limited set of operators. If you want element-wise multiplication, div, trued, and mod, there aren't any operators left with the right precedence for cross-multiplication, dot-product, and back-division. And even if that weren't an issue, using % to mean mod sometimes and cross-product other times is bound to be confusing to readers. Why not use multiple "operator characters" for user-defined infix operators, like postgreSQL does ? a *% b ++ c *% (d *. e) We could even define that the characters in the combined operator define precendece so the above could be written as a *% b ++ c *% d *. e where *% has higher precedence than *. because of (* is higher than %) and *. has higher precedence than *% because (. is gher than %) Cheers Hannu > > So, the perfect option doesn't exist. > > Backtick operators always let you write things in order, sometimes but not always let you avoid parentheses, reduce visual noise but don't dispel it completely, and do little or nothing for conciseness. They're not perfect, but they are an improvement. > >> Precedence is a big problem. You're assuming that backtick >> operators would have precedence higher than '+'. What if >> I want to define my own addition-like operator? One >> precedence isn't going to fit all. > > This is another thing I already discussed this in my initial email, and I don't want to repeat myself again. > > They should all have the same precedence, precisely _because_ they all have the same "visual weight", and no in-built intuitions. Try writing up some examples mixing it with different operators, and I think that, no matter what the name is, they all look like they should bind the same way: > > > 1 * 2 `plus` 3 * 4 == plus(1*2, 3*4), not 1 * plus(2, 3) * 4 > 1 + 2 `times` 3 + 4 == times(1+2, 3+4), not 1 + times(2, 3) + 4 > > ? or ? > > 1 < 2 `plus` 3 == 1 < plus(2, 3), not plus(1<2, 3) > > 1 < 2 `times` 3 == 1 < times(2, 3) > > You can argue about what exactly the precedence should be, but whatever it is, I'm convinced it should be fixed. > > (In Haskell, there's a default, but it's customizable. And people do create little one-letter functions and infix them with custom precedence and convince themselves that `p` looks like it belongs at a different level than `g`, but I think that's silly in Haskell, and would be even more so in Python.) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From rymg19 at gmail.com Sat Feb 22 21:38:32 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Sat, 22 Feb 2014 14:38:32 -0600 Subject: [Python-ideas] The Return of Implicit Lambda (Re: Allowing breaks in generator expressions by overloading the while keyword) In-Reply-To: References: <20140221102403.GV3684@ando> <1393014891.79562.YahooMailNeo@web181003.mail.ne1.yahoo.com> <1393020176.19093.YahooMailNeo@web181002.mail.ne1.yahoo.com> <20140221231924.GY3684@ando> <87vbw7vjh8.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Sat, Feb 22, 2014 at 8:24 AM, Nick Coghlan wrote: > > On 22 Feb 2014 14:51, "Stephen J. Turnbull" wrote: > > > > Steven D'Aprano writes: > > > > > Guys, I had to page through *eight pages* of quotes, at least six > levels > > > deep, before I found a two-line new comment. > > > > Indeed, if you're going to top-post, please do it at the *top*. > > FWIW, when I do it, it's mostly due to a combination of the Gmail UI > hiding quoted text by default and the phone interface making deleting large > blocks of text annoying. > > Agreed it needs to be kept under control, but it's unfortunately easy to > slip up and overquote given the default behaviour of at least Gmail and > likely other mobile clients :P > > Cheers, > Nick. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > Same thing here. I've been trying to bottom-post ever since someone here requested I do so, but it takes practice. -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From denis.spir at gmail.com Sat Feb 22 22:05:17 2014 From: denis.spir at gmail.com (spir) Date: Sat, 22 Feb 2014 22:05:17 +0100 Subject: [Python-ideas] Method chaining notation In-Reply-To: <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> Message-ID: <5309110D.6040103@gmail.com> On 02/22/2014 03:42 PM, Masklinn wrote: > Executing the method always has the same effect on its subject. That it > may not be used for the same purpose is a different issue and common: > a[k] = v can be used to add a new (k, v) pair or to update a key to a > new value (in fact Erlang's new map construct makes the difference and > provides for an update-only version). You are right! and this is a weakness of python in my view. But you could as well have chosen plain assignment, couldn't you? works the same way: n = 1 # symbol creation / definition n = 2 # symbol change / redefinition [Note, as a side-point, that if def / redef signs were different, there would be no ambiguity around local vs global scope. The ambiguity lies in fact in that the compiler cannot know if one wants to create a local symbol or change a global one. As for locally creating a global symbol, this should just not exist ;-); symbols live in their creation scope.] d From abarnert at yahoo.com Sat Feb 22 23:59:42 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sat, 22 Feb 2014 14:59:42 -0800 (PST) Subject: [Python-ideas] Infix functions In-Reply-To: <5308FBDB.6030709@krosing.net> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> Message-ID: <1393109982.6210.YahooMailNeo@web181002.mail.ne1.yahoo.com> From: Hannu Krosing Sent: Saturday, February 22, 2014 11:34 AM > On 02/22/2014 03:45 AM, Andrew Barnert wrote: >> Sure, having an infinite number of easily-distinguishable operators that? >> happen to have the precedence that fits your use case would be create. But we >> have a very limited set of operators. If you want element-wise multiplication, >> div, trued, and mod, there aren't any operators left with the right >> precedence for cross-multiplication, dot-product, and back-division. And even if >> that weren't an issue, using % to mean mod sometimes and cross-product other >> times is bound to be confusing to readers. > Why not use multiple "operator characters" for user-defined infix > operators, like postgreSQL does ? > > a *% b ++ c *% (d *. e) First, I'm not sure why everyone is focusing on the mathematical examples. As I said front he start,?I think NumPy and friends have mostly solved the problem of doing math readably in Python, and the only reason custom operators would likely be useful would be if we had _other_ use cases, probably (but not necessarily) akin to the higher-order functions people frequently inline in Haskell. Of course in Haskell, many of those _are_ written as symbols, but I doubt anyone wants that for Python. I could see "spam `compose` eggs" in Python, but not "spam . eggs". Anyway, sticking with the mathematical cases, Haskell does what you're proposing. And it?was discussed for Python Numeric in the 2.1 days. I don't think PEP 225 covers why they rejected the idea in favor of just doubling the number of operators by adding a special ~ prefix, but there are two major problems. First, none of those sequences has any inherent meaning. I can guess what @cross means. If you just double the set of operators by allowing a ~ prefix, I can guess that ~* is a variant on multiplication (I do have to know whether you're using ~* for element-wise and * for object-wise, Matlab-style, or vice-versa, R-style, but that's just one piece of information per project). If you have a whole suite of free-form operators made up of symbol strings, I have absolutely no idea what *% means. Maybe you know what you intended there, but would you know after three weeks of working on a different project? This is a serious problem with Haskell; I can't read someone's code unless I have his project's operator-defining stub open so I know whether he used <. for sorting, compose-composing, backward folding, or something I never even heard of that I have to work through. Second,?many such sequences are ambiguous. What does 3*.2 mean? Is it your custom *. operator, or a normal * multiplying 3 and .2? It's not just the dot; + and - are unary prefix operators, = is a different kind of syntax, etc. You could use whitespace to distinguish, so 2 %+ 3 is op%+(2, 3) but 2%+3 is mod(2, +3), but that's a pretty big change to the way the parser works. Also, there's a reason the "spaces around binary operators" rule is in the PEP 8 style guide rather than the language: usually it makes your code more readable to follow the guideline, but sometimes it makes it less readable. > We could even define that the characters in the combined operator define > precendece so the > above could be written as > > a *% b ++ c *% d *. e > > where *% has higher precedence than *. because of (* is higher than %) > and *. has higher > precedence than *% because (. is gher than %) Well, * and % are actually the same precedence in Python, and . isn't an operator so it doesn't have a precedence, but I get your point. This has been discussed for Haskell a few times; see?the thread starting http://www.haskell.org/pipermail/haskell-cafe/2006-October/018884.html for one example. I don't know that anyone ever thought it through for Python. It might work. It would certainly encourage people to use operators starting with * only to use multiplication-like things instead of whatever-they-want, since it's going to get multiplication-like precedence and associativity. It might?be worth working through some detailed examples. From abarnert at yahoo.com Sat Feb 22 23:59:36 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sat, 22 Feb 2014 14:59:36 -0800 (PST) Subject: [Python-ideas] Infix functions In-Reply-To: <5308FBDB.6030709@krosing.net> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> Message-ID: <1393109976.79100.YahooMailNeo@web181001.mail.ne1.yahoo.com> From: Hannu Krosing Sent: Saturday, February 22, 2014 11:34 AM > On 02/22/2014 03:45 AM, Andrew Barnert wrote: >> Sure, having an infinite number of easily-distinguishable operators that? >> happen to have the precedence that fits your use case would be create. But we >> have a very limited set of operators. If you want element-wise multiplication, >> div, trued, and mod, there aren't any operators left with the right >> precedence for cross-multiplication, dot-product, and back-division. And even if >> that weren't an issue, using % to mean mod sometimes and cross-product other >> times is bound to be confusing to readers. > Why not use multiple "operator characters" for user-defined infix > operators, like postgreSQL does ? > > a *% b ++ c *% (d *. e) First, I'm not sure why everyone is focusing on the mathematical examples. As I said front he start,?I think NumPy and friends have mostly solved the problem of doing math readably in Python, and the only reason custom operators would likely be useful would be if we had _other_ use cases, probably (but not necessarily) akin to the higher-order functions people frequently inline in Haskell. Of course in Haskell, many of those _are_ written as symbols, but I doubt anyone wants that for Python. I could see "spam `compose` eggs" in Python, but not "spam . eggs". Anyway, sticking with the mathematical cases, Haskell does what you're proposing. And it?was discussed for Python Numeric in the 2.1 days. I don't think PEP 225 covers why they rejected the idea in favor of just doubling the number of operators by adding a special ~ prefix, but there are two major problems. First, none of those sequences has any inherent meaning. I can guess what @cross means. If you just double the set of operators by allowing a ~ prefix, I can guess that ~* is a variant on multiplication (I do have to know whether you're using ~* for element-wise and * for object-wise, Matlab-style, or vice-versa, R-style, but that's just one piece of information per project). If you have a whole suite of free-form operators made up of symbol strings, I have absolutely no idea what *% means. Maybe you know what you intended there, but would you know after three weeks of working on a different project? This is a serious problem with Haskell; I can't read someone's code unless I have his project's operator-defining stub open so I know whether he used <. for sorting, compose-composing, backward folding, or something I never even heard of that I have to work through. Second,?many such sequences are ambiguous. What does 3*.2 mean? Is it your custom *. operator, or a normal * multiplying 3 and .2? It's not just the dot; + and - are unary prefix operators, = is a different kind of syntax, etc. You could use whitespace to distinguish, so 2 %+ 3 is op%+(2, 3) but 2%+3 is mod(2, +3), but that's a pretty big change to the way the parser works. Also, there's a reason the "spaces around binary operators" rule is in the PEP 8 style guide rather than the language: usually it makes your code more readable to follow the guideline, but sometimes it makes it less readable. > We could even define that the characters in the combined operator define > precendece so the > above could be written as > > a *% b ++ c *% d *. e > > where *% has higher precedence than *. because of (* is higher than %) > and *. has higher > precedence than *% because (. is gher than %) Well, * and % are actually the same precedence in Python, and . isn't an operator so it doesn't have a precedence, but I get your point. This has been discussed for Haskell a few times; see?the thread starting http://www.haskell.org/pipermail/haskell-cafe/2006-October/018884.html for one example. I don't know that anyone ever thought it through for Python. It might work. It would certainly encourage people to use operators starting with * only to use multiplication-like things instead of whatever-they-want, since it's going to get multiplication-like precedence and associativity. It might?be worth working through some detailed examples. From steve at pearwood.info Sun Feb 23 02:23:06 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 23 Feb 2014 12:23:06 +1100 Subject: [Python-ideas] Infix functions In-Reply-To: <5308FBDB.6030709@krosing.net> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> Message-ID: <20140223012306.GH3684@ando> On Sat, Feb 22, 2014 at 08:34:51PM +0100, Hannu Krosing wrote: > Why not use multiple "operator characters" for user-defined infix > operators, like postgreSQL does ? > > a *% b ++ c *% (d *. e) How would that work in Python? Give an example (a toy or pretend example is fine) for how I would define an operator ^& for ints. > We could even define that the characters in the combined operator define > precendece so the > above could be written as Who is "we"? The programmer of the module, or the Python core devs? In other words, are the precedences set once, in the language, or in each module that uses them? If I import an object x from one module with precedences set one way, and an object y from another module with precedences set another way, and try to use them in the same expression, whose precedences win? An unrelated thought: there are currently eight ASCII symbols used for non-comparison operators in Python: ~%^&*-+/ (did I miss any?). Of those, three can be used as unary operators ~-+ and so should not be allowed in the second position. So with eight symbols available in the first position, and five in the second, that gives us 40 new infix operators. But one of them ** is already used, so 39. To my mind, 39 arbitrary infix operators is both too many and too few. Too many, because, really, who needs 39 new infix operators? Who will remember them all? But too few, because it means that when you want a new operator, you have to force it into one of those 39 (48 if you can re-use one of existing operators, 52 if you include comparison operators > >= < <= but not equality and inequality). For example, I'd like to use ? and ? for set union and intersection, but instead those two operations get mapped to & and | and I can never remember which is which. I'd like to use operators such as ? ?. But so long as Python operators are ASCII only, that cannot happen. But then, I have a mathematics background. I'm having trouble thinking of examples outside of maths where I would want to define infix operators at all, instead of just using a method or function. -- Steven From ncoghlan at gmail.com Sun Feb 23 03:25:09 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 23 Feb 2014 12:25:09 +1000 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> Message-ID: On 23 February 2014 01:54, Chris Angelico wrote: > On Sun, Feb 23, 2014 at 2:44 AM, Alan Cristhian Ruiz > wrote: >> What is wrong with the current sintax?: >> >> 'abcd'\ >> .upper()\ >> .lower()\ >> .title() > > It doesn't have each method operate on the original object. It's not > easy to see with strings, but try this: > > list_of_numbers = [1,2] > list_of_numbers.append(3) > list_of_numbers.append(4) > list_of_numbers.append(5) > > Now write that without repeating list_of_numbers all the way down the line. The thing is, this *isn't an accident*, it's a deliberate choice in the library design to distinguish between data flow pipelines that transform data without side effects, and repeated mutation of a single object (although reading Guido's earlier reply, that may be me retrofitting an explicit rationale onto Guido's personal preference). Mutation and transformation are critically different operations, and most requests for method chaining amount to "I want to use something that looks like a chained transformation to apply multiple mutating operations to the same object". The response from the core developers to that request is almost certainly always going to be "No", because it's a fundamentally bad idea to blur that distinction: you should be able to tell *at a glance* whether an operation is mutating an existing object or creating a new one (this is actually one of the problems with the iterator model: for iterators, rather than iterables, the "__iter__ returns self" implementation means that iteration becomes an operation with side effects, which can be surprising at times, usually because the iterator shows up as unexpectedly empty later on). Compare: seq = get_data() seq.sort() seq = sorted(get_data()) Now, compare that with the proposed syntax as applied to the first operation: seq = []->extend(get_data())->sort() That *looks* like it should be a data transformation pipeline, but it's not - each step in the chain is mutating the original object, rather than creating a new one. That's a critical *problem* with the idea, not a desirable feature. There are a few good responses to this: 1. Design your APIs as transformation APIs that avoid in-place operations with side effects. This is a really good choice, as stateless transformations are one of the key virtues of functional programming, and if a problem can be handled that way without breaking the reader's brain, *do it*. Profiling later on may reveal the need to use more efficient in-place operations, but externally stateless APIs are still a great starting point that are less likely to degenerate into an unmaintainable stateful mess over time (you can maintain temporary state *internally*, but from the API users' perspective, things should look like they're stateless). 2. Provide a clean "specification" API, that allows a complex object structure to be built from something simpler (see, for example, logging.dictConfig(), or the various declarative approaches to defining user interfaces, or the Python 3 open(), which can create multilayered IO stacks on behalf of the user) 3. If the core API is based on mutation, but there's a clean and fast copying mechanism, consider adding a transformation API around it that trades speed (due to the extra object copies) for clarity (due to the lack of reliance on side effects). There's also a somewhat hacky workaround that can be surprisingly effective in improving readability when working with tree structures: abuse context managers to make the indentation structure match the data manipulation structure. @contextmanager def make(obj): yield obj with make(gtk.Window(gtk.WINDOW_TOPLEVEL)) as window: window.set_title("Hello Buttons!") window.connect("delete_event", delete_event) window.set_border_width(10) with make(gtk.HBox(False, 0)) as box1: window.add(box1) with make(gtk.Button("Button 1")) as button1: button1.connect("clicked", callback, "button 1") box1.pack_start(button1, True, True, 0) with make(gtk.Button("Button 2")) as button2: button2.connect("clicked", callback, "button 2") box1.pack_start(button2, True, True, 0) window.show_all() Although even judicious use of vertical whitespace and comments can often be enough to provide a significant improvement: # Make the main window window = gtk.Window(gtk.WINDOW_TOPLEVEL) window.set_title("Hello Buttons!") window.connect("delete_event", delete_event) window.set_border_width(10) # Make the box and add the buttons box1 = gtk.HBox(False, 0) window.add(box1) # Add Button 1 button1 = gtk.Button("Button 1") button1.connect("clicked", callback, "button 1") box1.pack_start(button1, True, True, 0) # Add Button 2 button2 = gtk.Button("Button 2") button2.connect("clicked", callback, "button 2") box1.pack_start(button2, True, True, 0) # And now we're done window.show_all() And adding a short internal helper function makes it even clearer: # Make the main window window = gtk.Window(gtk.WINDOW_TOPLEVEL) window.set_title("Hello Buttons!") window.connect("delete_event", delete_event) window.set_border_width(10) # Make the box and add the buttons box1 = gtk.HBox(False, 0) window.add(box1) def add_button(box, label, callback_arg): button = gtk.Button(label) button.connect("clicked", callback, callback_arg) box.pack_start(button, True, True, 0) add_button(box, "Button 1", "button 1") add_button(box, "Button 2", "button 2") # And now we're done window.show_all() It's easy to write code that looks terrible - but to make the case for a syntax change, you can't use code that looks terrible as a rationale, when there are existing ways to refactor that code that make it substantially easier to read. It's only when the code is *still* hard to read after it has been refactored to be as beautiful as is currently possible that a case for new syntactic sugar can be made. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From rosuav at gmail.com Sun Feb 23 03:48:48 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 23 Feb 2014 13:48:48 +1100 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> Message-ID: On Sun, Feb 23, 2014 at 1:25 PM, Nick Coghlan wrote: > it's a fundamentally bad idea to blur that distinction: you should be > able to tell *at a glance* whether an operation is mutating an > existing object or creating a new one... > > Compare: > > seq = get_data() > seq.sort() > > seq = sorted(get_data()) > > Now, compare that with the proposed syntax as applied to the first operation: > > seq = []->extend(get_data())->sort() > > That *looks* like it should be a data transformation pipeline, but > it's not - each step in the chain is mutating the original object, > rather than creating a new one. That's a critical *problem* with the > idea, not a desirable feature. Except that it doesn't. The idea of using a different operator is that it should clearly be mutating the original object. It really IS obvious, at a glance, that it's going to be returning the existing object, because that operator means it will always be. I believe that naming things that don't matter is a bad idea. We don't write code like this: five = 5 two = 2 print("ten is",five*two) because the intermediate values are completely insignificant. It's much better to leave them unnamed. (Okay, they're trivial here, but suppose those were function calls.) In a GTK-based layout, you'll end up creating a whole lot of invisible widgets whose sole purpose is to control the layout of other widgets. In a complex window, you might easily have dozens of those. (Same happens in Tkinter, from what I gather, but I haven't much looked into that.) Naming those widgets doesn't improve readability - in fact, it damages it, because you're left wondering which insignificant box is which. Leaving them unnamed and just part of a single expression emphasizes their insignificance. ChrisA From bruce at leapyear.org Sun Feb 23 04:17:50 2014 From: bruce at leapyear.org (Bruce Leban) Date: Sat, 22 Feb 2014 19:17:50 -0800 Subject: [Python-ideas] Infix functions In-Reply-To: <20140223012306.GH3684@ando> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> Message-ID: On Sat, Feb 22, 2014 at 5:23 PM, Steven D'Aprano wrote: > For example, I'd like to use > ? and ? for set union and intersection, but instead those two operations > get mapped to & and | and I can never remember which is which. I'd like > to use operators such as ? ?. But so long as Python operators are ASCII > only, that cannot happen. > What if Python didn't restrict you to ASCII operators? Here's specifics of how that could work: (1) An extended operator is a *single* math symbol character, e.g., ? or ?or ? but not ??. The set of allowed characters needs to be defined carefully. It could be any Unicode S-class character with the math property. Extended operators can be combined with an equal sign to do in-place updating, e.g., A ?= B. (2) Extended operators can be declared like this. class sample: def '?'(self, rhs): return self.value.intersection(sample.get_set(rhs)) def 'r?'(self, lhs): """Reversed operator.""" return sample.get_set(lhs).intersection(self.value) def 'i?'(self, rhs): self.value.intersection_update(sample.get_set(rhs)) @staticmethod def get_set(self): try: value = rhs.value except AttributeError: value = rhs return set(value) I'm not sure about enclosing the operator character in quotes. C++ and C# use an operator keyword, while Ruby doesn't mark it at all. Python, of course, uses __oper__ functions for overloading built-in operators. Writing __?__ would complicate the tokenizer as it would have to recognize this exact syntax and treat it as an identifier while not allowing math symbol characters in other uses in an identifier. And I probably need to write sample['?'] not sample.?. Without the quotes or something along those lines, these look too similar: def U(a, b): pass def ?(a, b): pass (3) All extended operators have the same precedence, between comparisons and bitwise or. Perhaps expressions mixing different extended operators always require parentheses, i.e., (A ? B ? C) would be a syntax error. To my mind this is way more readable than something like *%+& which looks like someone cursing. :-) --- Bruce Learn how hackers think: http://j.mp/gruyere-security -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Sun Feb 23 06:06:35 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sat, 22 Feb 2014 21:06:35 -0800 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> Message-ID: On Feb 22, 2014, at 18:48, Chris Angelico wrote: > On Sun, Feb 23, 2014 at 1:25 PM, Nick Coghlan wrote: >> That *looks* like it should be a data transformation pipeline, but >> it's not - each step in the chain is mutating the original object, >> rather than creating a new one. That's a critical *problem* with the >> idea, not a desirable feature. > > Except that it doesn't. The idea of using a different operator is that > it should clearly be mutating the original object. It really IS > obvious, at a glance, that it's going to be returning the existing > object, because that operator means it will always be. The difference between the look of nested statements and giant expressions in Python is much larger than the difference between the look of . and ->. One structure means you're doing imperative stuff, mutating one value on each line. The other means you're doing declarative stuff, transforming objects into new temporary objects. That distinction is huge, and the fact that it's immediately visible in Python i's one of the strengths of Python over most other "multi-paradigm" languages. > I believe that naming things that don't matter is a bad idea. We don't > write code like this: > > five = 5 > two = 2 > print("ten is",five*two) But in real life code, this would be something like rows * columns, and even if rows and columns are constant, they're constants you might want to change in a future version of the code, so you _would_ name them. And if, as you say, they're actually function calls, not constants, I think most people would write: rows = consoleobj.getparam('ROWS') cols = consoleobj.getparam('COLS') cells = rows * cols ... rather than try to cram it all in one line. > because the intermediate values are completely insignificant. It's > much better to leave them unnamed. (Okay, they're trivial here, but > suppose those were function calls.) In a GTK-based layout, you'll end > up creating a whole lot of invisible widgets whose sole purpose is to > control the layout of other widgets. In a complex window, you might > easily have dozens of those. (Same happens in Tkinter, from what I > gather, but I haven't much looked into that.) Naming those widgets > doesn't improve readability - in fact, it damages it, because you're > left wondering which insignificant box is which. Leaving them unnamed > and just part of a single expression emphasizes their insignificance. All you're arguing here is that PyGtk is badly designed, or that Gtk is not a good match for Python, so you have to write wrappers. There's no reason the wrapper has to be fluent instead of declarative. From abarnert at yahoo.com Sun Feb 23 06:09:45 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sat, 22 Feb 2014 21:09:45 -0800 Subject: [Python-ideas] Infix functions In-Reply-To: <20140223012306.GH3684@ando> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> Message-ID: <906F3EE4-AE77-4773-83C1-4425D9584E3A@yahoo.com> On Feb 22, 2014, at 17:23, Steven D'Aprano wrote: >> We could even define that the characters in the combined operator define >> precendece so the >> above could be written as > > Who is "we"? The programmer of the module, or the Python core devs? In > other words, are the precedences set once, in the language, or in each > module that uses them? His point was that if we make precedence programmatic and rule-based, there is no "we", just the fixed and documented set of rules that automatically give you the precedence for any symbol-string operator. Which solves the problem you're asking about. From haoyi.sg at gmail.com Sun Feb 23 06:27:22 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Sat, 22 Feb 2014 21:27:22 -0800 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> Message-ID: > The other means you're doing declarative stuff, transforming objects into new temporary objects. That distinction is huge I guess that's where people disagree. I think the distinction is not huge. Whether imperatively constructing something or "declaratively" (???) doing transformations, the *meaning* of the code is the same: start from some *foo* and do *some stuff *on *foo *in sequence until the *foo *is what I want. Whether it's implemented using mutation or allocation/garbage-collection is an implementation detail that clouds our view of a higher-level semantic: initializing an object with some stuff. In fact, this distinction is so meaningless that many languages/runtimes will turn one into the other as an optimization, because the semantics are exactly the same. On Sat, Feb 22, 2014 at 9:06 PM, Andrew Barnert wrote: > On Feb 22, 2014, at 18:48, Chris Angelico wrote: > > > On Sun, Feb 23, 2014 at 1:25 PM, Nick Coghlan > wrote: > >> That *looks* like it should be a data transformation pipeline, but > >> it's not - each step in the chain is mutating the original object, > >> rather than creating a new one. That's a critical *problem* with the > >> idea, not a desirable feature. > > > > Except that it doesn't. The idea of using a different operator is that > > it should clearly be mutating the original object. It really IS > > obvious, at a glance, that it's going to be returning the existing > > object, because that operator means it will always be. > > The difference between the look of nested statements and giant expressions > in Python is much larger than the difference between the look of . and ->. > One structure means you're doing imperative stuff, mutating one value on > each line. The other means you're doing declarative stuff, transforming > objects into new temporary objects. That distinction is huge, and the fact > that it's immediately visible in Python i's one of the strengths of Python > over most other "multi-paradigm" languages. > > > I believe that naming things that don't matter is a bad idea. We don't > > write code like this: > > > > five = 5 > > two = 2 > > print("ten is",five*two) > > But in real life code, this would be something like rows * columns, and > even if rows and columns are constant, they're constants you might want to > change in a future version of the code, so you _would_ name them. > > And if, as you say, they're actually function calls, not constants, I > think most people would write: > > rows = consoleobj.getparam('ROWS') > cols = consoleobj.getparam('COLS') > cells = rows * cols > > ... rather than try to cram it all in one line. > > > because the intermediate values are completely insignificant. It's > > much better to leave them unnamed. (Okay, they're trivial here, but > > suppose those were function calls.) In a GTK-based layout, you'll end > > up creating a whole lot of invisible widgets whose sole purpose is to > > control the layout of other widgets. In a complex window, you might > > easily have dozens of those. (Same happens in Tkinter, from what I > > gather, but I haven't much looked into that.) Naming those widgets > > doesn't improve readability - in fact, it damages it, because you're > > left wondering which insignificant box is which. Leaving them unnamed > > and just part of a single expression emphasizes their insignificance. > > All you're arguing here is that PyGtk is badly designed, or that Gtk is > not a good match for Python, so you have to write wrappers. There's no > reason the wrapper has to be fluent instead of declarative. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Sun Feb 23 06:30:42 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sat, 22 Feb 2014 21:30:42 -0800 Subject: [Python-ideas] Infix functions In-Reply-To: References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> Message-ID: <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> On Feb 22, 2014, at 19:17, Bruce Leban wrote: > On Sat, Feb 22, 2014 at 5:23 PM, Steven D'Aprano wrote: >> For example, I'd like to use >> ? and ? for set union and intersection, but instead those two operations >> get mapped to & and | and I can never remember which is which. I'd like >> to use operators such as ? ?. But so long as Python operators are ASCII >> only, that cannot happen. > > What if Python didn't restrict you to ASCII operators? That was something the PEP 225 (might have got it wrong from memory; I'm on my phone) envisioned as a future solution, in the far off day that Python and source editors could handle Unicode. And we have pretty much reached that day. So I'm not sure why I didn't pick up on this; thanks for bringing it up! However, there is still a big problem typing these symbols. Unicode hasn't meant a resurgence in APL, after all... A handful of them are available as opt-sequences on Mac, but most aren't. And on other platforms, where you have to use alt+keypad, none of them are typeable. Of course powerful enough editors can easily take care of this problem, but is it acceptable to say "if you want to use fancy operators, you have to use emacs, vi, or some not-yet-released version of Eclipse or PyCharm"? (Maybe it is; this is a real question, not rhetorical...) And it's not just editing code; if I want to import a module at the interactive prompt in my terminal or IDLE and call one of these functions, if I want to write a code snippet in an email or a StackOverflow answer, etc., I need an input method that works there (or I need to copy and paste from emacs). Finally, this is even _more_ specific to mathematical use cases. If I want to inline a higher-order function, there will likely be no obvious mathematical symbol for most of them. > Here's specifics of how that could work: > > (1) An extended operator is a single math symbol character, e.g., ? or ? or ? but not ??. The set of allowed characters needs to be defined carefully. It could be any Unicode S-class character with the math property. Extended operators can be combined with an equal sign to do in-place updating, e.g., A ?= B. That seems like a good first cut at the rule. > (2) Extended operators can be declared like this. > > class sample: > def '?'(self, rhs): > return self.value.intersection(sample.get_set(rhs)) That's the same syntax the PEP envisioned. They also wanted to change __add__ to '+', which probably seemed more reasonable in the early 2.x days than after 20 years of history. > And I probably need to write sample['?'] not sample.?. This part I don't like. In Python, attribute access and keyed access are different; namespaces are not dictionaries and vice-versa. And it argues against the quoted operator syntax (unless you want to allow quotes for attribute access...). But I'm not sure what the right answer is. > (3) All extended operators have the same precedence, between comparisons and bitwise or. I definitely agree with a fixed precedence (and associativity) that they all share. That specific choice is the same precedence I suggested for `operators`. But I did that based on writing a bunch of expressions mixing operators and seeing what looked most "natural". I'd have to look at the same examples with Unicode operators to see if it still looks right. > Perhaps expressions mixing different extended operators always require parentheses, i.e., (A ? B ? C) would be a syntax error. With backtick operators this doesn't seem necessary, but for Unicode symbols, I think you're right. The natural way to read that is for intersection to bind more tightly, which is not what the rule would do. So a SyntaxError is better than misleading code. > To my mind this is way more readable than something like *%+& which looks like someone cursing. :-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From cs at zip.com.au Sun Feb 23 06:36:28 2014 From: cs at zip.com.au (Cameron Simpson) Date: Sun, 23 Feb 2014 16:36:28 +1100 Subject: [Python-ideas] is that expensive? In-Reply-To: References: Message-ID: <20140223053628.GA9588@cskk.homeip.net> On 21Feb2014 18:37, Liam Marsh wrote: > is it possible to create(or tell me its name) a command to evaluate compute > length of a command/procedure? > (in number of: -processor operations(for 32 and 64 bit > processors) > -RAM read and write operations > -hard disk write and read operations > -eventual graphic operations > ) > this may be difficult, but it ables users to optimise their programs. By inspection of the code, without running the procedure? In general, probably not; I think that's equivalent to the halting problem and that is known to not have a universal solution. I think your question needs refining. Please provide more context. A skillful human can look at a function and, often, evaluate its performance in terms of its input. Not always, it depends on the inputs, and it also depends on what aspects of the operations are themselves expensive. Not all functions will terminate for all inputs, either (back to the halting problem again), so how expensive will you call such a function? Even supposing you have a suitable function (one you can inspect and decide on how it will behave), your choices of measurement are all a little vague and variable: For each of your 4 items above, they are highly dependent on the system architecture: different processors have different instruction sets, different compilers have different optimisation possibilities (and those possibilities depend on the specific CPU, too), RAM read and write operations depend on both the CPU and the memory architecture, hard disk write and read operations depend on both the language I/O library support and the OS buffering systems, and the cost of a "hard disk write and read operation" depends also on the hard disc hardware (SSD? on-disc buffering? RAID? ...) and graphics operations are also very variable. Yes, a human can often look at a human-written function and give an opinion about its cost. Cheers, -- Cameron Simpson Say it with flowers - Give her a triffid. - Russell Sparkes, russell at cerberus.bhpese.oz.au From bruce at leapyear.org Sun Feb 23 08:17:54 2014 From: bruce at leapyear.org (Bruce Leban) Date: Sat, 22 Feb 2014 23:17:54 -0800 Subject: [Python-ideas] Infix functions In-Reply-To: <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> Message-ID: On Sat, Feb 22, 2014 at 9:30 PM, Andrew Barnert wrote: > On Feb 22, 2014, at 19:17, Bruce Leban wrote: > > However, there is still a big problem typing these symbols. Unicode hasn't > meant a resurgence in APL, after all... A handful of them are available as > opt-sequences on Mac, but most aren't. And on other platforms, where you > have to use alt+keypad, none of them are typeable. Of course powerful > enough editors can easily take care of this problem, but is it acceptable > to say "if you want to use fancy operators, you have to use emacs, vi, or > some not-yet-released version of Eclipse or PyCharm"? (Maybe it is; this is > a real question, not rhetorical...) And it's not just editing code; if I > want to import a module at the interactive prompt in my terminal or > IDLE and call one of these functions, if I want to write a code snippet in > an email or a StackOverflow answer, etc., I need an input method that works > there (or I need to copy and paste from emacs). > These are legitimate questions. I would hate to see substitute n-graphs like in C: https://en.wikipedia.org/wiki/Digraphs_and_trigraphs. And having this feature and then having everyone write A <> B instead doesn't have much of an advantage. On the other hand, I haven't seen any replies from people saying they can't read these characters. So the real issue might be the inconvenience of typing. I wonder if the people who would use them are going to be typing them anyway in other contexts. Finally, this is even _more_ specific to mathematical use cases. If I want > to inline a higher-order function, there will likely be no obvious > mathematical symbol for most of them. > I'm not so sure about that. Check out this list of 949 Unicode math symbols. http://www.fileformat.info/info/unicode/category/Sm/list.htm And depending on the definition, there could be even more than that. While some of these have strong meanings (like variations on standard mathematical operators, integral signs, etc.) a lot seem more adaptable. > And I probably need to write sample['?'] not sample.?. > > This part I don't like. In Python, attribute access and keyed access are > different; namespaces are not dictionaries and vice-versa. > > And it argues against the quoted operator syntax (unless you want to allow > quotes for attribute access...). > Oops. Of course I meant getattr(sample, '?'). I've been switch back and forth a lot between Python and Javascript lately and it's easy to get that confused. Yes, that's not as attractive as sample.__add__ but do we write this that often? --- Bruce Learn how hackers think: http://j.mp/gruyere-security -------------- next part -------------- An HTML attachment was scrubbed... URL: From edk141 at gmail.com Sun Feb 23 12:13:45 2014 From: edk141 at gmail.com (Ed Kellett) Date: Sun, 23 Feb 2014 11:13:45 +0000 Subject: [Python-ideas] with expression In-Reply-To: References: Message-ID: On 20 February 2014 18:50, Yann Kaiser wrote: > ... some uses I've thought of: # current: try: with open('foo.txt', 'r') as f: thing = f.read() except IOError: thing = 'default' # except-expression: with open('foo.txt', 'r') as f: thing = f.read() except IOError: 'default' # with-expression: thing = f.read() with open('foo.txt', 'r') as f, default('default', IOError) ######### # current: try: thing = operation_may_fail() except: thing = 42 # except-expression: thing = operation_may_fail() except Exception: 42 # with-expression: thing = operation_may_fail() with computed_default(lambda: 42) ######### # current: # assuming one of these functions takes a very long time, and the other # blocks a thread you don't want to be blocking for long with some_lock: x = do_unsafe_thing() with another_lock: y = other_thing() frobnicate(x, y) # except-expression: # (no change) # with-expression: frobnicate((do_unsafe_thing() with some_lock), (other_thing() with another_lock)) ######### # current: try: os.unlink('/sbin/init') except OSError: pass # -- or -- with suppress(OSError): os.unlink('/sbin/init') # except-expression: os.unlink('/sbin/init') except OSError: None # (PEP 463 says you should use a with statement) # with-expression: os.unlink('/sbin/init') with suppress(OSError) From denis.spir at gmail.com Sun Feb 23 12:29:01 2014 From: denis.spir at gmail.com (spir) Date: Sun, 23 Feb 2014 12:29:01 +0100 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> Message-ID: <5309DB7D.5090209@gmail.com> On 02/23/2014 03:25 AM, Nick Coghlan wrote: > >> list_of_numbers = [1,2] >> list_of_numbers.append(3) >> list_of_numbers.append(4) >> list_of_numbers.append(5) As a side-note: There is no need in python for such constructs as proposed. As noted previously, most of them happen at init time (or more generally at object conctruction time), as in your example, and python proposes better constructs for that. >> Now write that without repeating list_of_numbers all the way down the line. > The thing is, this *isn't an accident*, it's a deliberate choice in > the library design to distinguish between data flow pipelines that > transform data without side effects, and repeated mutation of a single > object (although reading Guido's earlier reply, that may be me > retrofitting an explicit rationale onto Guido's personal preference). > > Mutation and transformation are critically different operations, [...] I approve all what you say (if i understand right). Except that your choice of terms is rather misleading: mutation and transformation are just synonyms. What you mean (if i understand right) is mutation (of an existing piece of data) vs creation (of a new, and different, piece of data). d From denis.spir at gmail.com Sun Feb 23 12:38:54 2014 From: denis.spir at gmail.com (spir) Date: Sun, 23 Feb 2014 12:38:54 +0100 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> Message-ID: <5309DDCE.5020008@gmail.com> On 02/23/2014 06:06 AM, Andrew Barnert wrote: > All you're arguing here is that PyGtk is badly designed, or that Gtk is not a good match for Python, so you have to write wrappers. There's no reason the wrapper has to be fluent instead of declarative. That's what I was about to argue. I don't understand why the python wrapper does not let you construct widgets in one go, with all their "equipment", since it's trivial and standard style in python (except, certainly, for adding sub-widgets to containers, as the sub-widgets exist and need to defnied by themselves). d From p.f.moore at gmail.com Sun Feb 23 12:48:20 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 23 Feb 2014 11:48:20 +0000 Subject: [Python-ideas] Infix functions In-Reply-To: References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> Message-ID: On 23 February 2014 07:17, Bruce Leban wrote: > On the other hand, I haven't seen any replies from people saying they can't > read these characters. So the real issue might be the inconvenience of > typing. I wonder if the people who would use them are going to be typing > them anyway in other contexts. Some things that immediately strike me: 1. The Windows console is not Unicode-friendly and displaying non-ASCII source code is still problematic. 2. Encoding detection is *not* a solved problem, and it's certainly not universally implemented in editors. It may look like it on Unix systems where the system encoding is generally UTF8 these days, but on Windows the system codepage is typically a 256-character one and working with anything different (which you'd need to for this proposal) means remembering to specify explicit encodings quite frequently. 3. Tools like grep may or may not work with extended characters - they don't on Windows, but I don't know how much of the fault for that lies with the Windows console. From what I've seen of the grep sources they might but they rely on char *argv which implies that the C runtime support for setting up argv might be relevant here (and on Windows that does *not* handle extended characters) The general theme here is obviously "Windows" so maybe that's really what I'm saying. But it does mean that Andrew Barnert's comment "in the far off day that Python and source editors could handle Unicode. And we have pretty much reached that day." may not be true unless you're willing to ignore Windows users. Paul From denis.spir at gmail.com Sun Feb 23 12:51:25 2014 From: denis.spir at gmail.com (spir) Date: Sun, 23 Feb 2014 12:51:25 +0100 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> Message-ID: <5309E0BD.8060204@gmail.com> On 02/23/2014 06:27 AM, Haoyi Li wrote: >> >The other means you're doing declarative stuff, transforming objects into > new temporary objects. That distinction is huge > > I guess that's where people disagree. I think the distinction is not huge. > Whether imperatively constructing something or "declaratively" (???) doing > transformations, the *meaning* of the code is the same: start from some > *foo* and do *some stuff *on *foo *in sequence until the *foo *is what I > want. > > Whether it's implemented using mutation or allocation/garbage-collection is > an implementation detail that clouds our view of a higher-level semantic: > initializing an object with some stuff. In fact, this distinction is so > meaningless that many languages/runtimes will turn one into the other as an > optimization, because the semantics are exactly the same. It's not the same if you have references to some of the objects involved; in python via "symbolic assignments", assignments which right side is a symbol, as in "b=a". This is the whole point, I guess, and one core difficulty of complex system modelisation. Such references exist in the model being expressed (as in, Bob is Ann's boyfriend, and the football team goal keeper, and the manager of the shop around the corner, and...); or else your code is wrong (you're making up inexistent refs to things, as artifact of your coding style). The 2 methods above are not semantically equivalent. In the mutative case, you're talking about one single *thing* (the representation of one thing in the model, correctly "reified" in code). This thing may be multiply ref'ed, because it is a thing and as such has multiple *aspects* or *roles* (or competences?). In the functional case, you making up new objects at every step, thus if there were ref's they would be broken. We should only use such a style for non-things, meaning for plain data (information) *about* things. (This style is not appropriate for GUI widgets, which conceptually are things, and pretty often ref'ed. Instead widgets should be created on one go.) d From haael at interia.pl Sun Feb 23 14:31:27 2014 From: haael at interia.pl (haael at interia.pl) Date: Sun, 23 Feb 2014 14:31:27 +0100 Subject: [Python-ideas] Joining dicts again In-Reply-To: <20140221225733.GW3684@ando> References: <3f21074da4e8e68b4863c1f312848a5b@draco.uberspace.de> <20140221225733.GW3684@ando> Message-ID: @Steven D'Aprano > So we have *at least* four different ways to merge dictionaries a and b: > > # 1: a wins > c = b.copy() > c.update(a) > > > # 2: b wins > c = a.copy() > c.update(b) > > > # 3: choose a winner according to the `or` operator > c = a.copy() > for key, value in b.items(): > if key in c: > c[key] = c[key] or value > else: > c[key] = value > > > # 4: keep both, in a list of 1 or 2 items > c = {key:[value] for key, value in a.items()} > for key, value in b.items(): > if key in c and value != c[key][0]: > c[key].append(value) > else: > c[key] = [value] > > > The first three are special cases of a more general case, where > you have a "decision function" that takes two values (one from dict a > and the other from dict b) and decides which one to keep. Case 1 ("a > always wins") would use `lambda x,y: x`, case 2 ("b wins") would use > `lambda x,y: y` and case 3 would use operator.or_. > > The question is, why should any one of these be picked out as so > obviously more useful than the others as to deserve being a dict method > or operator support? > > Steven All solutions provided by you are not one-liners. Every requires at least 2 lines of code and is an imperative-style code block, instead of a simple expression. I would really like to have a simple dict joining _expression_ that can be inserted everywhere I just need. fun(dict_arg=(dict_a | dict_b)) fun(**(dict_a | dict_b)) I personally have no problem with any of your code samples being promoted to an operator. My proposal is philosophically exactly the same as dropping the "print" _statement_ and replacing it with "print" _function_. It just merges more nicely with the rest of Python code. Really, I would even be happy if we had at least a dict _method_ that returns the updated dict: {'a':1}.update({'b':2}) # returns {'a':1, 'b':2} @Mathias Panzenb?ck > I never had a case where this kind of conflict resolution made sense. Can > you show us an example? Consider this example once again: fun(**(dict_a | dict_b)) Function default parameters are often null-like values. I don't say that they always are, but if is common in the programming practice. The null values usually evaluate to boolean False in Python when converted. Now consider the following: Let's have a dict with some function's default parameters: {'a':None, 'b':[]} Now let's have two separate code blocks that fill those arguments with some computed values. def funA(p): p['a'] = 1 return p def funB(p): p[b] = [1, 2, 3] return p Again, this pattern is not uncommon in the programming practice. We often have some code blocks that try to fill as much parameters as possible, but not every of them. Finally, we want to merge the dicts returned by these functions and provide them to the function of our interest. Basically, we want something like that: dict_default = {'a':None, 'b':[]} dict_a = funA(dict_default.copy()) dict_b = funB(dict_default.copy()) dict_param = merge_using_or_resolution(dict_a, dict_b) fun(**dict_param) Quite a lot of code, as you see. It also involves a lot of copying. The function 'merge_using_or_resolution' also needs at least one dict copy. If we had a dict joining operator, we could simply write: dict_default = {'a':None, 'b':[]} fun(**(funA(dict_default) | funB(dict_default)) No copies, no temporary variables, the functions funA and funB would also become one-liners, no namespace and memory pollution. Also, this code could be better optimized, unlike the previous example. As you see, there are many benefits of having dict-joining operator. I want to stress it once again. My examples are quite abstract, but the code patterns are actually quite common. Thanks haael From steve at pearwood.info Sun Feb 23 15:08:58 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 24 Feb 2014 01:08:58 +1100 Subject: [Python-ideas] Joining dicts again In-Reply-To: References: <3f21074da4e8e68b4863c1f312848a5b@draco.uberspace.de> <20140221225733.GW3684@ando> Message-ID: <20140223140858.GK3684@ando> On Sun, Feb 23, 2014 at 02:31:27PM +0100, haael at interia.pl wrote: > > @Steven D'Aprano > > > So we have *at least* four different ways to merge dictionaries a and b: [...] > > The question is, why should any one of these be picked out as so > > obviously more useful than the others as to deserve being a dict method > > or operator support? Please note the question. That is a critical question. > I would really like to have a simple dict joining _expression_ that > can be inserted everywhere I just need. But which one? There are at least four "simple dict joining" operations. Which one should be turned into a method, and which ones should be left out? It's easy to say that you want a dict operator | to merge dictionaries, but you still have to explain why one merge function should be promoted to an operator, and the others three (or more) merge functions miss out. Then, when people complain that *their* choice got left out, you can explain why their use-case is not important enough to be an operator, but yours is. In the meantime, you can add a two or three line function to your module, and use that. An easy solution to a simple problem. -- Steven From steve at pearwood.info Sun Feb 23 15:20:53 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 24 Feb 2014 01:20:53 +1100 Subject: [Python-ideas] Joining dicts again In-Reply-To: References: <3f21074da4e8e68b4863c1f312848a5b@draco.uberspace.de> <20140221225733.GW3684@ando> Message-ID: <20140223142053.GL3684@ando> On Sun, Feb 23, 2014 at 02:31:27PM +0100, haael at interia.pl wrote: > @Mathias Panzenb?ck > > > I never had a case where this kind of conflict resolution made sense. Can > > you show us an example? [...] > Now consider the following: > > Let's have a dict with some function's default parameters: > {'a':None, 'b':[]} Seems a bit contrived, but okay. > Now let's have two separate code blocks that fill those arguments with some computed values. > > def funA(p): > p['a'] = 1 > return p > > def funB(p): > p[b] = [1, 2, 3] > return p I notice that both of these mutate the dictionary in place. This is important later on. > Again, this pattern is not uncommon in the programming practice. We > often have some code blocks that try to fill as much parameters as > possible, but not every of them. I can't say that I've ever done this, but let's continue. > Finally, we want to merge the dicts returned by these functions and > provide them to the function of our interest. > > Basically, we want something like that: > > dict_default = {'a':None, 'b':[]} > dict_a = funA(dict_default.copy()) > dict_b = funB(dict_default.copy()) > dict_param = merge_using_or_resolution(dict_a, dict_b) > fun(**dict_param) I see that you are making dict_a and dict_b mutated copies. I don't understand why you don't just work with a single dict, instead of making copies all over the place. And since both funA and funB return the dict, you can chain them. This is so such simpler than the way you wrote it, and avoids all the unnecessary copying: dict_param = {'a':None, 'b':[]} fun(**(funA(funB(dict_param)))) Even if you decide that you want to keep the defaults untouched (in case you need them later), that makes only a single copy: dict_defaults = {'a':None, 'b':[]} fun(**(funA(funB(dict_defaults.copy())))) -- Steven From rosuav at gmail.com Sun Feb 23 15:21:47 2014 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 24 Feb 2014 01:21:47 +1100 Subject: [Python-ideas] Joining dicts again In-Reply-To: References: <3f21074da4e8e68b4863c1f312848a5b@draco.uberspace.de> <20140221225733.GW3684@ando> Message-ID: On Mon, Feb 24, 2014 at 12:31 AM, wrote: > dict_default = {'a':None, 'b':[]} > fun(**(funA(dict_default) | funB(dict_default)) > All you need is a shorter name for your function and the job's done. Since it'll be local to your module anyway, it doesn't need the long and verbose name. Here's Steven's #3, implemented as a function: def merge(a, b): "Merge two dicts using the 'or' operator c = a.copy() for key, value in b.items(): if key in c: c[key] = c[key] or value else: c[key] = value Or you could make it shorter thus: def merge(a, b): "Merge two dicts using the 'or' operator c = a.copy() for key, value in b.items(): c[key] = c.get(key) or value Then your code is: dict_default = {'a':None, 'b':[]} fun(**merge(funA(dict_default), funB(dict_default))) That's it! ChrisA From rosuav at gmail.com Sun Feb 23 15:23:23 2014 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 24 Feb 2014 01:23:23 +1100 Subject: [Python-ideas] Joining dicts again In-Reply-To: <20140223142053.GL3684@ando> References: <3f21074da4e8e68b4863c1f312848a5b@draco.uberspace.de> <20140221225733.GW3684@ando> <20140223142053.GL3684@ando> Message-ID: On Mon, Feb 24, 2014 at 1:20 AM, Steven D'Aprano wrote: >> Let's have a dict with some function's default parameters: >> {'a':None, 'b':[]} > > Seems a bit contrived, but okay. I hesitate to overly optimize contrived examples. Usually results in missing one critical part of what we weren't told and being useless. :| ChrisA From breamoreboy at yahoo.co.uk Sun Feb 23 15:46:56 2014 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 23 Feb 2014 14:46:56 +0000 Subject: [Python-ideas] Joining dicts again In-Reply-To: References: <3f21074da4e8e68b4863c1f312848a5b@draco.uberspace.de> <20140221225733.GW3684@ando> Message-ID: On 23/02/2014 13:31, haael at interia.pl wrote: > > @Steven D'Aprano > >> So we have *at least* four different ways to merge dictionaries a and b: >> >> # 1: a wins >> c = b.copy() >> c.update(a) >> >> >> # 2: b wins >> c = a.copy() >> c.update(b) >> >> >> # 3: choose a winner according to the `or` operator >> c = a.copy() >> for key, value in b.items(): >> if key in c: >> c[key] = c[key] or value >> else: >> c[key] = value >> >> >> # 4: keep both, in a list of 1 or 2 items >> c = {key:[value] for key, value in a.items()} >> for key, value in b.items(): >> if key in c and value != c[key][0]: >> c[key].append(value) >> else: >> c[key] = [value] >> >> >> The first three are special cases of a more general case, where >> you have a "decision function" that takes two values (one from dict a >> and the other from dict b) and decides which one to keep. Case 1 ("a >> always wins") would use `lambda x,y: x`, case 2 ("b wins") would use >> `lambda x,y: y` and case 3 would use operator.or_. >> >> The question is, why should any one of these be picked out as so >> obviously more useful than the others as to deserve being a dict method >> or operator support? >> >> Steven > > > > All solutions provided by you are not one-liners. Every requires at least 2 lines of code and is an imperative-style code block, instead of a simple expression. > Excellent as it makes the code easier to read. Let's leave things like that. > I would really like to have a simple dict joining _expression_ that can be inserted everywhere I just need. > I wouldn't. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com From alexander.belopolsky at gmail.com Sun Feb 23 16:50:35 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 23 Feb 2014 10:50:35 -0500 Subject: [Python-ideas] Infix functions In-Reply-To: References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> Message-ID: On Sat, Feb 22, 2014 at 10:17 PM, Bruce Leban wrote: > I'm not sure about enclosing the operator character in quotes. C++ and C# > use an operator keyword, while Ruby doesn't mark it at all. Python, of > course, uses __oper__ functions for overloading built-in operators. > Writing __?__ would complicate the tokenizer as it would have to > recognize this exact syntax and treat it as an identifier while not > allowing math symbol characters in other uses in an identifier. > What about __intersection__, __union__, etc. It looks like all symbols of interest have relatively short one-two word names in Unicode that can me mechanically converted to dunder method names. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron3200 at gmail.com Sun Feb 23 19:24:32 2014 From: ron3200 at gmail.com (Ron Adam) Date: Sun, 23 Feb 2014 12:24:32 -0600 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> Message-ID: On 02/22/2014 08:48 PM, Chris Angelico wrote: > On Sun, Feb 23, 2014 at 1:25 PM, Nick Coghlan wrote: >> >it's a fundamentally bad idea to blur that distinction: you should be >> >able to tell*at a glance* whether an operation is mutating an >> >existing object or creating a new one... >> > >> >Compare: >> > >> > seq = get_data() >> > seq.sort() >> > >> > seq = sorted(get_data()) >> > >> >Now, compare that with the proposed syntax as applied to the first operation: >> > >> > seq = []->extend(get_data())->sort() >> > >> >That*looks* like it should be a data transformation pipeline, but >> >it's not - each step in the chain is mutating the original object, >> >rather than creating a new one. That's a critical*problem* with the >> >idea, not a desirable feature. > Except that it doesn't. The idea of using a different operator is that > it should clearly be mutating the original object. It really IS > obvious, at a glance, that it's going to be returning the existing > object, because that operator means it will always be. I agree with nick, this looks like a transformation chain. Each step transforming the "new" result of the previous step. seq = []->extend(get_data())->sort() To make it pythonic ... The operator you want is one for an in place method call. If we apply the "+=" pattern for the '__iadd__' method call syntax, to the more the general '.' method syntax, we get ".=", the in place method call syntax. seq = [] seq .= extend(get_data()) # In place method call. In place method calls seem quite reasonable to me. And then to get the rest of the way there, allow chained "in place" method calls. seq = [] .= extend(get_data()) .= sort() Which should be a separate pep from the ".=" enhancement. BTW... allowing ".=" could mean a class could have one __iget_method__ attribute instead of multiple __ixxxx___ methods. (Or something like that.) Cheers, Ron From anikom15 at gmail.com Sun Feb 23 21:24:58 2014 From: anikom15 at gmail.com (=?ISO-8859-1?Q?Westley_Mart=EDnez?=) Date: Sun, 23 Feb 2014 12:24:58 -0800 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> Message-ID: On Feb 23, 2014 10:25 AM, "Ron Adam" wrote: > I agree with nick, this looks like a transformation chain. Each step transforming the "new" result of the previous step. > > > seq = []->extend(get_data())->sort() > > > > To make it pythonic ... > > > The operator you want is one for an in place method call. If we apply the "+=" pattern for the '__iadd__' method call syntax, to the more the general '.' method syntax, we get ".=", the in place method call syntax. > > seq = [] > seq .= extend(get_data()) # In place method call. > > In place method calls seem quite reasonable to me. > > > > And then to get the rest of the way there, allow chained "in place" method calls. > > seq = [] .= extend(get_data()) .= sort() > > Which should be a separate pep from the ".=" enhancement. > > > > BTW... allowing ".=" could mean a class could have one __iget_method__ attribute instead of multiple __ixxxx___ methods. (Or something like that.) > > Cheers, > Ron I like this syntax. It easy to tell what exactly is getting mutated. -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Sun Feb 23 21:47:46 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sun, 23 Feb 2014 12:47:46 -0800 Subject: [Python-ideas] Infix functions In-Reply-To: References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> Message-ID: <8AB414A2-5156-4DAB-AC7C-47A6A283CED4@yahoo.com> On Feb 23, 2014, at 3:48, Paul Moore wrote: > ... Windows > The general theme here is obviously "Windows" so maybe that's really > what I'm saying. But it does mean that Andrew Barnert's comment "in > the far off day that Python and source editors could handle Unicode. > And we have pretty much reached that day." may not be true unless > you're willing to ignore Windows users. You're right, I misspoke imprecisely. We've reached the day where it's _in sight_, but now where it's usable fact. But that still means we can evaluate the idea a lot better now. Even if Windows 9 were UTF-8-centric, or Python 4 used nothing but the UTF-16 interfaces in Windows and were PowerShell-friendly, there would still be all the problems I raised (like entry methods--not just for editors, but for the interactive interpreter, web sites and mail clients where you want to post code samples, etc.), and I think it's an open but maybe answerable question whether those problems are acceptable as a tradeoff for the benefits (as opposed to 20 years ago, where you could really only answer by guessing what a Unicode-friendly platform would look like). Of course this is me talking about Bruce's idea. I don't want to put words in his mouth. And I also don't want people to forget that _my_ idea was using infix functions for _non-mathematical_ cases, and this is only a minor sideline to me. From abarnert at yahoo.com Sun Feb 23 21:58:13 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sun, 23 Feb 2014 12:58:13 -0800 Subject: [Python-ideas] Method chaining notation In-Reply-To: <5309E0BD.8060204@gmail.com> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <5309E0BD.8060204@gmail.com> Message-ID: <62247C4E-FBA2-4F14-99BD-0B2EBE755795@yahoo.com> On Feb 23, 2014, at 3:51, spir wrote: > The 2 methods above are not semantically equivalent. In the mutative case, you're talking about one single *thing* (the representation of one thing in the model, correctly "reified" in code). This thing may be multiply ref'ed, because it is a thing and as such has multiple *aspects* or *roles* (or competences?). In the functional case, you making up new objects at every step, thus if there were ref's they would be broken. We should only use such a style for non-things, meaning for plain data (information) *about* things. > > (This style is not appropriate for GUI widgets, which conceptually are things, and pretty often ref'ed. Instead widgets should be created on one go.) But put that together with your other reply: > That's what I was about to argue. I don't understand why the python wrapper does not let you construct widgets in one go, with all their "equipment", since it's trivial and standard style in python (except, certainly, for adding sub-widgets to containers, as the sub-widgets exist and need to defnied by themselves). The cleanest way to do this would then be to build the initializer(s) declaratively, then initialize the (mutable) object in one go, right? As for why PyGtk doesn't work that way, there are two reasons. First, it's deliberately intended to make Python Gtk code, C Gtk code, Vala Gtk code, etc. look as similar as possible. Gtk has a language-agnostic idiom that takes precedence over each language's idioms. This means you only have to write examples, detailed docs, etc. once, instead of once for every language with bindings. Second, from a practical point of view, it allow PyGtk to be a very thin wrapper around Gtk. And if you want to know why Gtk itself wasn't designed to be more Pythonic... Well, it wasn't designed for Python. It was originally designed for C, and then updated for C and Vala. So it has C/Vala-focused idioms. That's the same reason Qt and Wx have C++ idioms, Tk has Tcl idioms, WinForms has C#/VB idioms, Cocoa has ObjC idioms, etc. Unfortunately, none of the major GUI libraries was designed primarily with Python in mind, so we have to adapt. From nathan at cmu.edu Sun Feb 23 22:06:11 2014 From: nathan at cmu.edu (Nathan Schneider) Date: Sun, 23 Feb 2014 16:06:11 -0500 Subject: [Python-ideas] Infix functions In-Reply-To: <1393109982.6210.YahooMailNeo@web181002.mail.ne1.yahoo.com> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <1393109982.6210.YahooMailNeo@web181002.mail.ne1.yahoo.com> Message-ID: On Sat, Feb 22, 2014 at 5:59 PM, Andrew Barnert wrote: > From: Hannu Krosing > > Sent: Saturday, February 22, 2014 11:34 AM > > > > On 02/22/2014 03:45 AM, Andrew Barnert wrote: > >> Sure, having an infinite number of easily-distinguishable operators > that > > >> happen to have the precedence that fits your use case would be create. > But we > >> have a very limited set of operators. If you want element-wise > multiplication, > >> div, trued, and mod, there aren't any operators left with the right > >> precedence for cross-multiplication, dot-product, and back-division. > And even if > >> that weren't an issue, using % to mean mod sometimes and cross-product > other > >> times is bound to be confusing to readers. > > > > Why not use multiple "operator characters" for user-defined infix > > operators, like postgreSQL does ? > > > > a *% b ++ c *% (d *. e) > > First, I'm not sure why everyone is focusing on the mathematical examples. I have not seen a compelling use case for infix expressions other than mathematical operators which currently require a method, thereby forcing an order that does not match what we are used to writing. As has been argued, conciseness is a virtue for the readability of complex mathematical expressions. Allowing Unicode symbols for operators would be nice in an ideal world, but it seems like in practice it would create a lot of pain for very little gain. So I think the best compromise would be to allow current binary operators to be augmented with a special character, such as ~ or : or `, such that (a) new ambiguities would not be introduced into the grammar, and (b) the precedence and dunder method name would be determined by the current operator. Whether the operator has been "modified" with the special character could be passed as an argument to the dunder method. > I don't think PEP 225 covers why they rejected the idea in favor of just > doubling the number of operators by adding a special ~ prefix, but there > are two major problems. > > > First, none of those sequences has any inherent meaning. I can guess what > @cross means. If you just double the set of operators by allowing a ~ > prefix, I can guess that ~* is a variant on multiplication (I do have to > know whether you're using ~* for element-wise and * for object-wise, > Matlab-style, or vice-versa, R-style, but that's just one piece of > information per project). Using the special modifier character would tell the user that they have to refer to the library documentation to interpret it. In fact, I think builtins should be prohibited from supporting any of the modified operators, so as to avoid establishing a default interpretation for any of them. But presumably, most of the important use cases (e.g., cross product) would have a conceptually related current operator (*), and would be frequent enough that it would be worth the reader's time to look it up. For rarer cases, there is little harm in just providing a method. > If you have a whole suite of free-form operators made up of symbol > strings, I have absolutely no idea what *% means. > Agreed, which is why I would be in favor of restricting ourselves to a single modifier character that is not already an operator. Nathan -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Sun Feb 23 22:16:18 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sun, 23 Feb 2014 13:16:18 -0800 (PST) Subject: [Python-ideas] Allowing breaks in generator expressions by overloading the while keyword In-Reply-To: References: <20140221102403.GV3684@ando> <53074E98.4020205@egenix.com> Message-ID: <1393190178.19057.YahooMailNeo@web181002.mail.ne1.yahoo.com> From: Nick Coghlan Sent: Friday, February 21, 2014 6:17 AM > The reason I keep beating my head against this particular wall (cf. > PEP 403's @in clauses and PEP 3150's given suites) is that my personal > goal for Python is that it should be a tool that lets people express > what they are thinking clearly and relatively concisely. > > As far as I have been able to tell, the persistent requests for > "multi-line lambdas", cleaner lambda syntax, etc, are because Python > doesn't currently make it easy to express a lot of operations that > involve higher order manipulation of "one shot" callables? - closures > or custom functions where you *don't* want to re-use them, but Python > still forces you to pull them out and name them. I think PEP 403 is my > best current description of the mental speed bump involved: > http://www.python.org/dev/peps/pep-0403/ There are actually two different?in fact, nearly opposite?problems that get lumped together. The first problem is that you often want to use _expressions_ as values, and you can't. The second problem is that you sometimes want to build complicated functions in-line. Solving that does nothing to help the first problem (as JavaScript proves), and I'm not sure it's really as big a problem as people think (there's a reason experienced JavaScript programmers start using Deferreds/Promises?borrowed from Python?to escape the "callback hell" their language allows), but I won't get into that here. The fact that both of these manifest as "problems with lambda" is really just a coincidence?the first is a problem with lambda because the only way we have to simulate passing around expressions is by wrapping them in functions; the second is a problem with lambda because without statements inside expressions it's the only way to define a function inline. Anyway, the first problem is why we have comprehensions, and why people want more out of them.?What can comprehensions do that map and filter can't? Maybe they're a little faster, maybe they let you write long sequences of map and filter in order instead of mutated into prefix order, but nobody cares about either of those enough to add new syntax.?The real benefit is that they let you map an expression over an iterable, or filter an iterable with an expression, without wrapping it upon a function. Compare: ? ? [x**2 for x in seq] ? ? map(lambda x: x**2, seq) The second one forces the reader to think through an extra abstraction that isn't relevant to the problem. And people want the same thing more generally. Most novices don't think of callbacks as higher-order functions, which is why they write this: ? ? b = Button("button 10", callback=onclick(10)) ? instead of one of these: ? ? b = Button("button 10", callback=lambda: onclick(10)) ? ? b = Button("button 10", callback=partial(onlick, 10)) But really, why should they have to? All they want to say is "when I click the button, evaluate the expression onclick(10)". Why?is that so hard? It's not really about improving lambda, but about removing the need for it. Surely this isn't any easier to explain: ? ? b = Button("button 10", callback=:onclick(10)) Even if it isn't a complete solution, there is of course an obvious practical benefit in readability. If you see a chain of widget declarations, and they all end with something like "callback=:onclick(10)", you only really have to process that ":" once for the whole chain, and then it fades into the background as an unimportant detail in understanding the real point of the code. So I love the idea. And, so far, Python has been adding practical partial solutions. As you (Nick) pointed out,?comprehensions, decorators, with statements, conditional and except expressions, etc. are all ways to provide new contexts where expressions can be used without wrapping them in functions. There are probably a few others worth covering, and at that point maybe we'll have handled 90% of the cases people need, and they can deal with the last 10%. While that may sound clunky, I'm not sure it's really a problem. Python isn't a perfect language, but it's a very good language, and that's why we all use it. Of course if there's a general solution, that would be great. But I'm not sure there is. Lisp-style quoting and macros, or Haskell-style higher-order-functioning, _might_ be general solutions?but even if they are, I'm not sure they're solutions that fit into Python. (Still, for the first two, everyone really should play with MacroPy and get a feel for what it can do and how Pythonic it can feel?) I wrote this up in more detail, with a lot more digressions, as a blog post, but blogger.com seems to be having problems, so the world is spared my rambling. :) From greg.ewing at canterbury.ac.nz Sun Feb 23 22:45:20 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 24 Feb 2014 10:45:20 +1300 Subject: [Python-ideas] [Python-Dev] Tangent on class level scoping rules In-Reply-To: References: <5307E7E8.6060001@canterbury.ac.nz> Message-ID: <530A6BF0.3050704@canterbury.ac.nz> Nick Coghlan wrote: > Dealing with references from nested closures is the hard part. I think that could be handled by creating new cells for the inner variables each time the inner scope is entered. -- Greg From abarnert at yahoo.com Sun Feb 23 22:59:25 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sun, 23 Feb 2014 13:59:25 -0800 (PST) Subject: [Python-ideas] Infix functions In-Reply-To: References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <1393109982.6210.YahooMailNeo@web181002.mail.ne1.yahoo.com> Message-ID: <1393192765.94079.YahooMailNeo@web181003.mail.ne1.yahoo.com> From: Nathan Schneider Sent: Sunday, February 23, 2014 1:06 PM >On Sat, Feb 22, 2014 at 5:59 PM, Andrew Barnert wrote: > >From: Hannu Krosing >> >>Sent: Saturday, February 22, 2014 11:34 AM >> >> >> >>> On 02/22/2014 03:45 AM, Andrew Barnert wrote: >> >>>> ?Sure, having an infinite number of easily-distinguishable operators that? >> >>>> happen to have the precedence that fits your use case would be create. But we >>>> have a very limited set of operators. If you want element-wise multiplication, >>>> div, trued, and mod, there aren't any operators left with the right >>>> precedence for cross-multiplication, dot-product, and back-division. And even if >>>> that weren't an issue, using % to mean mod sometimes and cross-product other >>>> times is bound to be confusing to readers. >> >> >>> Why not use multiple "operator characters" for user-defined infix >>> operators, like postgreSQL does ? >>> >>> a *% b ++ c *% (d *. e) >> >>First, I'm not sure why everyone is focusing on the mathematical examples. > >I have not seen a compelling use case for infix expressions other than mathematical operators which currently require a method, thereby forcing an order that does not match what we are used to writing. As has been argued, conciseness is a virtue for the readability of complex mathematical expressions. For the mathematical case, are you a heavy user of NumPy, SymPy, or other such libraries who finds them unreadable because of this problem? Because my understanding is that most such people don't think it's a serious problem in their real work, which is why PEP 225 hasn't been taken up again. I believe the big trick is thinking of even vectors as multi-dimensional: Multiply two row-vectors and you get element-wise multiplication; multiply a row-vector and a column-vector and you get cross-product. Beyond the mathematical case, let me try to explain again. Why do people want an except expression, when you could just write it as a function? Because there are two problems with this: ? ? catch(lambda: 1/n, ZeroDivisionError, lambda e: nan) First, you have to manually "delay" all the expressions by wrapping them in lambdas. Second, everything reads out of order?the important bit here is 1/n, not the fact that it's catching an error. That's why people want to write this: ? ? 1/n except ZeroDivisionError as e: nan But what if we had general solutions to both problems? Using Nick's short lambda syntax (actually just PEP 312 in this case) and backticks for infix operators, even though those may not be the best solutions: ? ? :1/n `catch` (ZeroDivisionError, :nan) Perfect? No. Good enough that we wouldn't feel compelled to add new syntax to improve it? Maybe. Obviously, if these two features together only solved one use case, and imperfectly at that, they wouldn't be worth it. But?consider the with expression someone recently proposed: ? ? data = f.read() with open(path) as f (Apologies for using a silly example; I can't remember what the supporting use cases were.)?You can write that as a function, but it looks like this: ? ? data = withcontext(lambda f: f.read(), open(path)) But with short lambdas and infix functions, it's a lot better: ? ? data = :?.read() `withcontext` open(path) Are there other "missing features" in Python that could be worked around with short lambdas and inline functions, instead of having to add new syntax for each one? (Could we have avoided some of the new features added in the past decade?) This is the part I'm not sure of. It may be that, at least for the kinds of programs people write today, Python has almost all of the constructions almost anyone needs, and we're better off filling in the last one or two gaps than looking for a general solution. As a side note, Clojure uses a different trick in a similar way. For example, list comprehensions are plain old functions, just as in Racket and other Lisp descendants?but its Smalltalk-style infix parameter names make it look a lot more like Python/Haskell/Miranda's special syntax: ? ? [p**2 for p in primes while p<1000 if p%2] ? ? (for [p primes :when (odd? p) :while (p<1000)] p**2) And the same trick could work for a catch function to make it even nicer, and it could have solved ternary expressions like the conditional: ? ? x = :1/n :catch ZeroDivisionError :then :NaN ? ? x = :n :then :1/n :orelse :NaN ? but I have absolutely no idea how to fit that idea into Python. But back to your take on the mathematical case: >So I think the best compromise would be to allow current binary operators to be augmented with a special character, such as ~ or : or `, such that (a) new ambiguities would not be introduced into the grammar, and (b) the precedence and dunder method name would be determined by the current operator. Whether the operator has been "modified" with the special character could be passed as an argument to the dunder method. This is nearly identical to PEP 225, except that you're passing a "tilde-prefixed" flag to the existing dunder methods instead of doubling the number of dunder methods. I think in general it's better to have separate functions for separate purposes than to add flag parameters, but I can see the argument that this is a special case: likely all of the operators would treat that flag the same way, so it could just be a matter of putting, say, "if tilde: self = self.lift_and_transpose()" at the start of each method, or doing the equivalent with a decorator on each method, or dynamically applying that decorator to all of the methods, or ? >Using the special modifier character would tell the user that they have to refer to the library documentation to interpret it. In fact, I think builtins should be prohibited from supporting any of the modified operators, so as to avoid establishing a default interpretation for any of them. Yes, PEP 225 has an extensive argument for why this is the best interpretation. (The short version is: Matlab vs. R. But there's more to it; it's worth reading.) >>If you have a whole suite of free-form operators made up of symbol strings, I have absolutely no idea what *% means. >Agreed, which is why I would be in favor of restricting ourselves to a single modifier character that is not already an operator. Unfortunately, ~ _is_ already an operator, and can in fact be combined with the unary + and - operators: ? ? >>> ~3 ? ? -4 ? ? >>> ~-3 ? ? 2 From ncoghlan at gmail.com Sun Feb 23 23:15:10 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 24 Feb 2014 08:15:10 +1000 Subject: [Python-ideas] [Python-Dev] Tangent on class level scoping rules In-Reply-To: <530A6BF0.3050704@canterbury.ac.nz> References: <5307E7E8.6060001@canterbury.ac.nz> <530A6BF0.3050704@canterbury.ac.nz> Message-ID: On 24 Feb 2014 07:46, "Greg Ewing" wrote: > > Nick Coghlan wrote: > >> Dealing with references from nested closures is the hard part. > > > I think that could be handled by creating new cells > for the inner variables each time the inner scope > is entered. Yes, it's not hard * in theory*, it's just hard *in practice*, given the current design of CPython's compiler - my perspective is informed by trying to do it for comprehensions and eventually deciding "meh, I'll just use a real closure so they behave *exactly* like generator expressions in a constructor call and I can stop worrying about it". There's also the fact that even after getting it to work, you have to *document* it in the language reference as a new kind of scope, because it doesn't behave the same way as any of the existing ones. Sure, it only differs from a full closure in a few obscure edge cases, but it still differs. Cheers, Nick. > > -- > Greg > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Sun Feb 23 23:53:46 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 24 Feb 2014 11:53:46 +1300 Subject: [Python-ideas] Infix functions In-Reply-To: References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> Message-ID: <530A7BFA.5040700@canterbury.ac.nz> Bruce Leban wrote: > (2) Extended operators can be declared like this. > > class sample: > def '?'(self, rhs): > return self.value.intersection(sample.get_set(rhs)) Or def __INTERSECTION__(self, rhs): ... def __rINTERSECTION__(self, rhs): ... (i.e. use the Unicode character names.) -- Greg From nicholas.cole at gmail.com Mon Feb 24 00:14:17 2014 From: nicholas.cole at gmail.com (Nicholas Cole) Date: Sun, 23 Feb 2014 23:14:17 +0000 Subject: [Python-ideas] Infix functions In-Reply-To: <1393024433.83604.YahooMailNeo@web181001.mail.ne1.yahoo.com> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <1393024433.83604.YahooMailNeo@web181001.mail.ne1.yahoo.com> Message-ID: On Friday, 21 February 2014, Andrew Barnert wrote: > From: Chris Rebert > > > Sent: Friday, February 21, 2014 2:47 PM > > > > On Fri, Feb 21, 2014 at 2:05 PM, Andrew Barnert > > > > wrote: > >> While we're discussing crazy ideas inspired by a combination of a > > long-abandoned PEP and Haskell idioms (see the implicit lambda thread), > > here's another: arbitrary infix operators: > >> > >> a `foo` b == foo(a, b) > > > > Prior discussion: > > https://mail.python.org/pipermail/python-ideas/2007-January/000050.html > > > > Which resulted in a new item in PEP 3099 ("Things that will Not Change > > in Python 3000"; http://legacy.python.org/dev/peps/pep-3099/ ): > > * No more backticks. > > Backticks (`) will no longer be used as shorthand for repr -- but > > that doesn't mean they are available for other uses. Even ignoring the > > backwards compatibility confusion, the character itself causes too > > many problems (in some fonts, on some keyboards, when typesetting a > > book, etc). > > > > > > I think people using suboptimal fonts and keyboard layouts should find > > better ones... > > > Thanks for pointing that out. > > OK, some other syntax then, there's no reason it has to be identical to > Haskell. We can even go back to the original PEP's version: > > a @foo b = foo(a, b) Please, no! Everything on this list for the past few days seems to have been more and more compact syntax for things that may or may not have a use. I'm in favour of adding new things to the language. I really like the except expressions pep. I look at it and I think, "I might have imagined you could already do that, and it will make a lot of code more readable." I think that is a great test for whether something should be added! But function any class decorators seem to have created a wave of suggestions for non-intuitive syntax based around funny characters and notations. It is for the BDFL to say whether these are Pythonic, but I hope none of them turn out to be. I'm comfortable with a language that is very slightly more verbose than it could be. Python at version 3.4 is very readable. Please let's keep it that way! I'm not getting at anyone - or any particular proposal - but there have been a spate of similar things! N. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From breamoreboy at yahoo.co.uk Mon Feb 24 00:21:46 2014 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 23 Feb 2014 23:21:46 +0000 Subject: [Python-ideas] Infix functions In-Reply-To: References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <1393024433.83604.YahooMailNeo@web181001.mail.ne1.yahoo.com> Message-ID: On 23/02/2014 23:14, Nicholas Cole wrote: > > > On Friday, 21 February 2014, Andrew Barnert > > wrote: > > From: Chris Rebert > > > Sent: Friday, February 21, 2014 2:47 PM > > > > On Fri, Feb 21, 2014 at 2:05 PM, Andrew Barnert > > > > wrote: > >> While we're discussing crazy ideas inspired by a combination of a > > long-abandoned PEP and Haskell idioms (see the implicit lambda > thread), > > here's another: arbitrary infix operators: > >> > >> a `foo` b == foo(a, b) > > > > Prior discussion: > > > https://mail.python.org/pipermail/python-ideas/2007-January/000050.html > > > > Which resulted in a new item in PEP 3099 ("Things that will Not > Change > > in Python 3000"; http://legacy.python.org/dev/peps/pep-3099/ ): > > * No more backticks. > > Backticks (`) will no longer be used as shorthand for repr -- but > > that doesn't mean they are available for other uses. Even > ignoring the > > backwards compatibility confusion, the character itself causes too > > many problems (in some fonts, on some keyboards, when typesetting a > > book, etc). > > > > > > I think people using suboptimal fonts and keyboard layouts should > find > > better ones... > > > Thanks for pointing that out. > > OK, some other syntax then, there's no reason it has to be identical > to Haskell. We can even go back to the original PEP's version: > > a @foo b = foo(a, b) > > > Please, no! > > Everything on this list for the past few days seems to have been more > and more compact syntax for things that may or may not have a use. > > I'm in favour of adding new things to the language. I really like the > except expressions pep. I look at it and I think, "I might have > imagined you could already do that, and it will make a lot of code more > readable." I think that is a great test for whether something should be > added! > > But function any class decorators seem to have created a wave of > suggestions for non-intuitive syntax based around funny characters and > notations. It is for the BDFL to say whether these are Pythonic, but I > hope none of them turn out to be. > > I'm comfortable with a language that is very slightly more verbose than > it could be. Python at version 3.4 is very readable. Please let's keep > it that way! > > I'm not getting at anyone - or any particular proposal - but there have > been a spate of similar things! > > N. > Massive +1 from me as you match my sentiments entirely. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com From stephen at xemacs.org Mon Feb 24 00:31:22 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 24 Feb 2014 08:31:22 +0900 Subject: [Python-ideas] Method chaining notation In-Reply-To: <5309DDCE.5020008@gmail.com> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <5309DDCE.5020008@gmail.com> Message-ID: <87d2idv22d.fsf@uwakimon.sk.tsukuba.ac.jp> spir writes: > On 02/23/2014 06:06 AM, Andrew Barnert wrote: > > All you're arguing here is that PyGtk is badly designed, or that > > Gtk is not a good match for Python, so you have to write > > wrappers. There's no reason the wrapper has to be fluent instead > > of declarative. > That's what I was about to argue. I don't understand why the python > wrapper does not let you construct widgets in one go, *Because* it's a *wrapper*, which leverages the gobject-introspection FFI. (gobject-introspection is an export-oriented FFI, rather than an import-oriented FFI like ctypes.) > with all their "equipment", since it's trivial and standard style > in python (except, certainly, for adding sub-widgets to containers, > as the sub-widgets exist and need to defnied by themselves). I don't think you need to except subwidgets. They could be defined recursively by including calls to their constructors in the "description" of the parent. It might be tricky to find a pleasant way to express placement in containers with flexible placement disciplines, but for single-child, column, row, and grid widgets I don't see a problem at all. Now, that would be nice for initialization, but the GTK+ v3 API is very dynamic and insanely complicated. I think it would take a huge amount of work to do this at all well, and I suspect that your program would benefit only from beautification of initialization -- everything else would still look the same. (Of course many programs don't need a dynamic UI; I suppose that would be a benefit.) In any case, I don't find a "chained" API for something like GTK any more attractive than the repetition of object being mutated. They're both quite ugly, an accurate reflection of the underlying library which tried to be a better Xt, but ended up equally messy, compounded by being a lot bigger (and more poorly documented). From stephen at xemacs.org Mon Feb 24 00:52:21 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 24 Feb 2014 08:52:21 +0900 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> Message-ID: <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> Ron Adam writes: > The operator you want is one for an in place method call. > seq = [] .= extend(get_data()) .= sort() That looks like anything but Python to me. If I really thought of that as a single operation, I'd do something like class ScarfNSort(list): def __init__(self): self.extend(get_data()) self.sort() seq = ScarfNSort() If it doesn't deserve a class definition, then the repeated references to 'seq' wouldn't bother me. N.B. '.=' shouldn't be called "in-place": 'sort' and 'extend' are already in-place. The word would be "chain," "cascade," or similar. From greg.ewing at canterbury.ac.nz Mon Feb 24 01:36:27 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 24 Feb 2014 13:36:27 +1300 Subject: [Python-ideas] Method chaining notation In-Reply-To: <62247C4E-FBA2-4F14-99BD-0B2EBE755795@yahoo.com> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <5309E0BD.8060204@gmail.com> <62247C4E-FBA2-4F14-99BD-0B2EBE755795@yahoo.com> Message-ID: <530A940B.8090108@canterbury.ac.nz> Andrew Barnert wrote: > Unfortunately, none of the major GUI libraries was > designed primarily with Python in mind, so we have to adapt. Yes, but the adaptation can be in the form of wrappers that make the API more Pythonic. It shouldn't mean warping Python to make it fit the ways of other languages. -- Greg From stephen at xemacs.org Mon Feb 24 01:45:36 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 24 Feb 2014 09:45:36 +0900 Subject: [Python-ideas] Infix functions In-Reply-To: References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> Message-ID: <87a9dhuymn.fsf@uwakimon.sk.tsukuba.ac.jp> Please read to the end before responding. I don't know how to write this without sounding hostile toward Windows ... I am. But I'm not hostile toward programmers who develop in Windows! Bear with me. Paul Moore writes: > 1. The Windows console is not Unicode-friendly and displaying > non-ASCII source code is still problematic. And there's no IDE (IDLE? but ISTR tkinter breaks on Windows :-( ) or similar that provides a simple Unicode-capable console you can run Python in? AFAIK the Emacs console emulations work fine on Windows (but I haven't tried Python in them). I don't know if Windows shells like Emacs very much (although Emacs does provide a shell written directly in Lisp, so you can circumvent the Windows console and shell completely). > 2. Encoding detection is *not* a solved problem, and it's certainly > not universally implemented in editors. I don't understand how this is related. Encoding detection for Python source *is* a solved problem in Python, since PEP 263. I don't really understand why the fact that it's not implemented in *your* editor should be anybody else's problem: Emacs on Windows has been capable (with a patch) since ~1991 (and without a patch since ~1998), due to the fact that PEP 263 coding cookies were modeled after the popular Emacs/vi cookies. > 3. Tools like grep may or may not work with extended characters Of course they do; all the usual stream-oriented Unix tools work fine. It's trivial due to the traditional restriction of tool languages to ASCII and the use of ASCII-compatible encodings. You do need to make sure that any tool arguments (such as regexps) use the same encoding as the stream to be processed (but I suppose that goes without saying). So if you're seeing wierdness, either arguments don't match stream (which often would be due to a console issue) or the console can't input/display the characters correctly. > But it does mean that Andrew Barnert's comment "in the far off day > that Python and source editors could handle Unicode. And we have > pretty much reached that day." may not be true unless you're > willing to ignore Windows users. Andrew's comment is *precisely* true: Python loves Unicode, and Notepad handles UTF-8 fine. I would imagine any real source editor on Windows does too. So your point comes down to "the Windows console sucks and I don't like Emacs". OK, that's a valid concern -- there are an awful lot of developers for whom the Windows console is an essential component of their environments, and Emacs is more than an acquired taste, it's a different way of seeing the world. But how many more decades is the Windows console going to be allowed to hold back the rest of the world? If it hasn't learned Unicode by now, is there any reason to suppose it ever will? If it won't learn, why not bite the bullet now? There are fewer Windows programmers to feel pain now than there ever will be again (we hope :-)! From steve at pearwood.info Mon Feb 24 02:35:40 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 24 Feb 2014 12:35:40 +1100 Subject: [Python-ideas] Infix functions In-Reply-To: <8AB414A2-5156-4DAB-AC7C-47A6A283CED4@yahoo.com> References: <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> <8AB414A2-5156-4DAB-AC7C-47A6A283CED4@yahoo.com> Message-ID: <20140224013540.GM3684@ando> On Sun, Feb 23, 2014 at 12:47:46PM -0800, Andrew Barnert wrote: > Of course this is me talking about Bruce's idea. I don't want to put > words in his mouth. And I also don't want people to forget that _my_ > idea was using infix functions for _non-mathematical_ cases, and this > is only a minor sideline to me. I can't think of many useful examples of infix operators that aren't mathematical, and even fewer that aren't just as easily written as methods or functions. I think that's why there are so few non-mathematical examples given in this thread. Apart from, say, taking the union and intersection of two dicts, I can't really think of anything where I'd want this outside of mathematics. In the absense of any compelling use-case for allowing arbitrary non-mathematical infix operators, I'm -1 on adding all this complexity and (to be frank) cryptic ugliness to the code. 99 times out of 100, the right answer is "use a method or function", and the remaining time, using a method or function is nearly always acceptable. All the low-hanging fruit is taken: between + - * / and perhaps a few other operators, nearly all the common and important use-cases are covered. -- Steven From stephen at xemacs.org Mon Feb 24 03:08:47 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 24 Feb 2014 11:08:47 +0900 Subject: [Python-ideas] Infix functions In-Reply-To: <20140224013540.GM3684@ando> References: <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> <8AB414A2-5156-4DAB-AC7C-47A6A283CED4@yahoo.com> <20140224013540.GM3684@ando> Message-ID: <8761o5uus0.fsf@uwakimon.sk.tsukuba.ac.jp> Steven D'Aprano writes: > I can't think of many useful examples of infix operators that aren't > mathematical, and even fewer that aren't just as easily written as > methods or functions. The copula, not to mention all non-mathematical transitive verbs, in English? This feature would make it a lot easier to write DSLs in Python. (I don't know if that would necessarily be a good thing.) From ncoghlan at gmail.com Mon Feb 24 04:15:38 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 24 Feb 2014 13:15:38 +1000 Subject: [Python-ideas] Infix functions In-Reply-To: References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <1393024433.83604.YahooMailNeo@web181001.mail.ne1.yahoo.com> Message-ID: On 24 February 2014 09:14, Nicholas Cole wrote: > > I'm comfortable with a language that is very slightly more verbose than it > could be. Python at version 3.4 is very readable. Please let's keep it that > way! > > I'm not getting at anyone - or any particular proposal - but there have > been a spate of similar things! Don't worry, this is just an artefact of the brainstorming nature of python-ideas. Most of the ideas that are posted here won't be a good fit for Python, and *that's OK*. This applies even to the ideas that are posted by experienced core developers - those of us that have been around for long enough to build a pretty good intuition for what counts as "Pythonic" may skip the python-ideas phase of the process when we're confident an idea is a good one and go straight to python-dev or the issue tracker. The end result is that python-ideas discussions related to more esoteric proposals will typically churn around for a while, and then dwindle away naturally. Sometimes more experienced list participants will attempt to nudge such threads in more productive directions, but other times we'll just let them take their natural course (which of those happens will depend mainly on whether there seems to be a possible seed of a good idea in the initial proposal, and how much time people have available to participate in the discussion). Occasionally, someone will hit on something that actually seems promising, and a PEP or a new third party PyPI module is born. PEP 463's except expressions are the most recent example of that, and PEP 450's statistics module is an example of one that went through the process of starting life as a third party module on PyPI (statslib). Other times, a PEP will be born that isn't actually ready for submission to python-dev, but instead acts as a historical record for a problem that is interesting but difficult to solve in a Pythonic way (my own PEPs 403 and 3150 fall into that category). The other thing to remember is that many of the more conservative members of the core development team *don't* participate in python-ideas (making that possible is one of the main reasons they're two separate lists), so even if a proposal gets to the point of being submitted as a PEP, that's still no guarantee that the PEP will be *accepted*. python-dev review may highlight issues that we missed here on python-ideas, and while those can often be fixed by slight tweaks to the PEP, sometimes they may be judged to represent such a fatal flaw that the PEP will be explicitly rejected rather than being revised. So yeah, it's part of the nature of python-ideas to be an environment for relatively free-wheeling "throw ideas at the wall to see what sticks" discussions, as well as history lessons on why things are the way they are, and providing advice on how to build a persuasive case for a particular change. By contrast, python-dev is more focused on the filtering stage of answering the question "Does this change represent an overall improvement to Python?", and the default answer to that question is always going to be No (simply on the grounds that the number of ways we could make Python worse is unbounded, so any new proposal needs to make a persuasive case for how that *particular* change actually makes Python better). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From abarnert at yahoo.com Mon Feb 24 07:49:21 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sun, 23 Feb 2014 22:49:21 -0800 Subject: [Python-ideas] Method chaining notation In-Reply-To: <530A940B.8090108@canterbury.ac.nz> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <5309E0BD.8060204@gmail.com> <62247C4E-FBA2-4F14-99BD-0B2EBE755795@yahoo.com> <530A940B.8090108@canterbury.ac.nz> Message-ID: <8C802BFB-6E14-45FC-AA22-38985BD06F13@yahoo.com> On Feb 23, 2014, at 16:36, Greg Ewing wrote: > Andrew Barnert wrote: >> Unfortunately, none of the major GUI libraries was >> designed primarily with Python in mind, so we have to adapt. > > Yes, but the adaptation can be in the form of wrappers that > make the API more Pythonic. It shouldn't mean warping > Python to make it fit the ways of other languages. But remember that there is an advantage to Gtk, Qt, etc. having their own language-agnostic idioms. They have a hard enough time documenting the whole thing as it is; if they had to write completely different documentation for C, C++, Vala, Python, .NET, etc., we just wouldn't get any documentation. Of course there's also a disadvantage. PyGtk code doesn't look very Pythonic. I think the suggestions in this thread for a language change that allows people to write code that looks like _ neither_ Python _nor_ Gtk is a bad solution to the problem. But there is a real problem, and I understand why people are trying to solve it. So what is the solution? Maybe the best thing people can put their effort into is a higher-level, more Pythonic wrapper around the most painful parts of the PyGtk wrapper (like initialization)? From rosuav at gmail.com Mon Feb 24 08:00:27 2014 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 24 Feb 2014 18:00:27 +1100 Subject: [Python-ideas] Method chaining notation In-Reply-To: <8C802BFB-6E14-45FC-AA22-38985BD06F13@yahoo.com> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <5309E0BD.8060204@gmail.com> <62247C4E-FBA2-4F14-99BD-0B2EBE755795@yahoo.com> <530A940B.8090108@canterbury.ac.nz> <8C802BFB-6E14-45FC-AA22-38985BD06F13@yahoo.com> Message-ID: On Mon, Feb 24, 2014 at 5:49 PM, Andrew Barnert wrote: > But remember that there is an advantage to Gtk, Qt, etc. having their own language-agnostic idioms. They have a hard enough time documenting the whole thing as it is; if they had to write completely different documentation for C, C++, Vala, Python, .NET, etc., we just wouldn't get any documentation. > Point to note: When I'm trying to pin down an issue that relates to GTK on Windows, I'll sometimes switch between Pike and Python, since my installations of them embed different GTK versions. (I discovered a GTK bug that way.) Being able to translate my code from one language to another is extremely useful. And when I'm looking for docs and (especially) examples, it's really common to get them for some completely different language - it's easier to translate back from (say) Perl than to hunt down an example in the language I'm actually using. A more Pythonic wrapper around object creation would pretty much look like what I was saying, except that it would take specific coding work. Actually, probably all it'd take is a module that imports all the PyGTK classes and wraps them in Steven's chain() function. But the proposal I make here would put the power directly in the hands of the programmer, rather than requiring that the module support it. Why should method chaining be in the hands of the module author? ChrisA From abarnert at yahoo.com Mon Feb 24 08:03:45 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sun, 23 Feb 2014 23:03:45 -0800 Subject: [Python-ideas] Infix functions In-Reply-To: References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <1393024433.83604.YahooMailNeo@web181001.mail.ne1.yahoo.com> Message-ID: On Feb 23, 2014, at 15:14, Nicholas Cole wrote: >> a @foo b = foo(a, b) > > Please, no! > > Everything on this list for the past few days seems to have been more and more compact syntax for things that may or may not have a use. > > > Please, no! > > Everything on this list for the past few days seems to have been more and more compact syntax for things that may or may not have a use. This thread was a spinoff from those other threads, an attempt to see whether a single change could make many of those other proposals unnecessary. I'm not sure it is, or even could be, successful at that. Unfortunately, nobody is even looking at the intended use cases, instead just assuming that this must only be useful for mathematical/numerical code (which, as I've said from the start, I think has been mostly a solved problem since numpy), and therefore either jumping off into how they'd like that to work, or knee-jerk responding that we don't need it. Obviously I've done something wrong in presenting the idea. I wish I knew what it was. -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Mon Feb 24 08:06:52 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sun, 23 Feb 2014 23:06:52 -0800 Subject: [Python-ideas] Infix functions In-Reply-To: <20140224013540.GM3684@ando> References: <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> <8AB414A2-5156-4DAB-AC7C-47A6A283CED4@yahoo.com> <20140224013540.GM3684@ando> Message-ID: <20642CB1-3D89-42B6-9739-E5CD1B94E3C9@yahoo.com> On Feb 23, 2014, at 17:35, Steven D'Aprano wrote: > On Sun, Feb 23, 2014 at 12:47:46PM -0800, Andrew Barnert wrote: > >> Of course this is me talking about Bruce's idea. I don't want to put >> words in his mouth. And I also don't want people to forget that _my_ >> idea was using infix functions for _non-mathematical_ cases, and this >> is only a minor sideline to me. > > I can't think of many useful examples of infix operators that aren't > mathematical, and even fewer that aren't just as easily written as > methods or functions. Turning with and except into readable expressions without needing to add new custom syntax for each was my initial motivation. It's possible there are only a handful of such cases, and only one of them is important enough to be worth doing, in which case custom syntax for that one is the best answer. But given that half the proposals on this list are for some form of new expression syntax, I'm not sure that's the case. From ron3200 at gmail.com Mon Feb 24 08:14:52 2014 From: ron3200 at gmail.com (Ron Adam) Date: Mon, 24 Feb 2014 01:14:52 -0600 Subject: [Python-ideas] Method chaining notation In-Reply-To: <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On 02/23/2014 05:52 PM, Stephen J. Turnbull wrote: > Ron Adam writes: > > > The operator you want is one for an in place method call. > > > seq = [] .= extend(get_data()) .= sort() > > That looks like anything but Python to me. Is it really all that different from this? >>> "Py" . __add__("th") . __add__("on") 'Python' The '.=' just says more explicitly that self will be returned after the method call. It wouldn't alter the string example here, since self isn't returned. But for mutable objects, it's an explicit reminder that it mutates rather than returns a new object. In the case that there is no __iget_method__ method, it would give an error. So it's not a make everything into a chain tool. > If I really thought of > that as a single operation, I'd do something like > > class ScarfNSort(list): > def __init__(self): > self.extend(get_data()) > self.sort() > > seq = ScarfNSort() > > If it doesn't deserve a class definition, then the repeated references > to 'seq' wouldn't bother me. > > N.B. '.=' shouldn't be called "in-place": 'sort' and 'extend' are > already in-place. The word would be "chain," "cascade," or similar. It's a chain only if you link more than one in sequence. Cheers, Ron From abarnert at yahoo.com Mon Feb 24 08:27:15 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sun, 23 Feb 2014 23:27:15 -0800 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <5309E0BD.8060204@gmail.com> <62247C4E-FBA2-4F14-99BD-0B2EBE755795@yahoo.com> <530A940B.8090108@canterbury.ac.nz> <8C802BFB-6E14-45FC-AA22-38985BD06F13@yahoo.com> Message-ID: <164EED28-079B-446F-844F-EBCA1127552B@yahoo.com> On Feb 23, 2014, at 23:00, Chris Angelico wrote: > On Mon, Feb 24, 2014 at 5:49 PM, Andrew Barnert wrote: >> But remember that there is an advantage to Gtk, Qt, etc. having their own language-agnostic idioms. They have a hard enough time documenting the whole thing as it is; if they had to write completely different documentation for C, C++, Vala, Python, .NET, etc., we just wouldn't get any documentation. > > Point to note: When I'm trying to pin down an issue that relates to > GTK on Windows, I'll sometimes switch between Pike and Python, since > my installations of them embed different GTK versions. (I discovered a > GTK bug that way.) Being able to translate my code from one language > to another is extremely useful. And when I'm looking for docs and > (especially) examples, it's really common to get them for some > completely different language - it's easier to translate back from > (say) Perl than to hunt down an example in the language I'm actually > using. > > A more Pythonic wrapper around object creation would pretty much look > like what I was saying, except that it would take specific coding > work. Actually, probably all it'd take is a module that imports all > the PyGTK classes and wraps them in Steven's chain() function. But that would be _less_ Pythonic, not more. The fact that mutating methods return None--and, more generally, the strong divide between mutation and transformation and between statements and expressions--is one of the major ways in which Python is idiomatically different from languages that are specifically meant to be fluent (like Smalltalk or C#) or that just never considered the design issue (like perl or JavaScript). You don't have to agree with Guido that method chaining is bad. But he's designed his language and stdlib to discourage it, and therefore anything you do in the opposite direction is fighting against the grain of the language and its standard idioms. From abarnert at yahoo.com Mon Feb 24 08:34:23 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sun, 23 Feb 2014 23:34:23 -0800 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Feb 23, 2014, at 23:14, Ron Adam wrote: > On 02/23/2014 05:52 PM, Stephen J. Turnbull wrote: >> Ron Adam writes: >> >> > The operator you want is one for an in place method call. >> >> > seq = [] .= extend(get_data()) .= sort() >> >> That looks like anything but Python to me. > > Is it really all that different from this? > > >>> "Py" . __add__("th") . __add__("on") > 'Python' Well, yes, it is. But, more importantly, who cares? That code is horribly unreadable and unpythonic. OK, I don't think PEP8 has a guideline saying "don't call __add__ when you can just use +", but only because it's so obvious it doesn't need to be stated. (And I'm pretty sure it _does_ have a guideline saying not to put spaces around the attribute dot.) So, is your argument is "my code looks kind of like some horribly unreadable and unpythonic, but legal, code, and therefore it should also be legal despite being unreadable and unpythonic?" From p.f.moore at gmail.com Mon Feb 24 09:03:56 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 24 Feb 2014 08:03:56 +0000 Subject: [Python-ideas] Infix functions In-Reply-To: <87a9dhuymn.fsf@uwakimon.sk.tsukuba.ac.jp> References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> <87a9dhuymn.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: (I did read to the end, and I don't really disagree with your points, even as a Windows programmer :-)) tl; dr - I agree with you but unfortunately Windows isn't dead yet and practicality means we can't ignore it. On 24 February 2014 00:45, Stephen J. Turnbull wrote: > So your point comes down to "the Windows console sucks and I don't > like Emacs". OK, that's a valid concern -- there are an awful lot of > developers for whom the Windows console is an essential component of > their environments, and Emacs is more than an acquired taste, it's a > different way of seeing the world. Essentially, yes :-) Although there's a bit more to it: * The C language (and hence all programmers') assumption that the basic abstraction for calling a progcess uses argc/atgv. That's not true on some operating systems (Windows, yeah, but "all the world is Unix" is a bad assumption even when the alternative is not Windows) and the C runtime has to jump through hoops (sometimes not perfectly) to pretend that lie is true. (That's the grep issue). I'll take your word re Emacs, as I use Vim. Vim is certainly as capable of Emacs of dealing with Unicode, but it doesn't do so out of the box (it's what happens if you *don't* think hard about Unicode that's relevant to this thread). Probably the most likely thing for people to forget is using the right encoding for Python files (for extra credit, make that UTF8 on Py3, but ASCII on Py2!), even though that differs from the system default. > But how many more decades is the Windows console going to be allowed > to hold back the rest of the world? If it hasn't learned Unicode by > now, is there any reason to suppose it ever will? If it won't learn, > why not bite the bullet now? There are fewer Windows programmers to > feel pain now than there ever will be again (we hope :-)! I wish I knew. I would kill for a proper Unicode-conforming console. But my company provides me with a Windows PC ro use at work, and the games I like to play are Windows-only. So for work and play, Windows is my environment, for better or worse. Maybe Windows 8 is so bad that the climate will shift. Maybe macs will stop costing more than my house at some point. But it's still some way off (and that was the original point - not that Unicode is desirable, but that we've reached the point where using it is not an issue). I would love to switch from Windows, but if I did so for my hobby coding, it'd lose any relevance it has to the rest of my life... Anyhow, we're way off topic here, so I'll stop. Paul From abarnert at yahoo.com Mon Feb 24 09:21:45 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Mon, 24 Feb 2014 00:21:45 -0800 (PST) Subject: [Python-ideas] The Return of Implicit Lambda (Re: Allowing breaks in generator expressions by overloading the while keyword) In-Reply-To: References: <20140221102403.GV3684@ando> <1393014891.79562.YahooMailNeo@web181003.mail.ne1.yahoo.com> <1393020176.19093.YahooMailNeo@web181002.mail.ne1.yahoo.com> Message-ID: <1393230105.34674.YahooMailNeo@web181001.mail.ne1.yahoo.com> From: Nick Coghlan Sent: Friday, February 21, 2014 3:28 PM >Yep, the implicit assumption was that the shorthand syntax could get away with handling just zero-argument functions (for simple callbacks and lazy evaluation) and one-positional-argument functions (for single object callbacks, sorting keys and filtering predicates). >More complex cases don't need to be handled, as users have the option to revert to using an explicit lambda or a named function. >This helps the proposal achieve one of useful heuristics to apply when judging a new piece of syntactic sugar: in the cases where it applies, it should be clearly superior to the existing alternatives. Once you get to 2+ arguments, it's hard to be clear without giving them names. After more thought on this, I realized that this really does solve all of the issues with the idea. Sure, it looks ugly when trying to define a function as a value in a dict (which is a common thing to do)?so don't use it there. In some contexts the ? looks wrong, so don't use it there either. And so on.?As long as there are plenty of use cases where it _would_ make code more readable?like, say, creating the callbacks in a long chain of Tkinter widget constructors?it's a win. And, although it doesn't _actually_ solve the problem of removing the need for an extra abstraction, in many cases it _practically_ does. If you're reading those 20 widget constructors, yes, you do need to read and process the fact that a function is being created out of your expression on the first one?but on the next 19, your eye will just glide over it and you can read the expressions directly. Which solves the part of the problem that's worth solving. And I'm not sure it's possible to _actually_ solve the problem anyway, without Python becoming a very different language. So I ended up writing the threatened blog post anyway:?http://stupidpythonideas.blogspot.com/2014/02/fixing-lambda.html So, I think it is worth reviving and extending PEP 312. What would help most in getting that idea considered? Gathering more concrete examples? Buiilding an implementation? Both? From rosuav at gmail.com Mon Feb 24 09:51:57 2014 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 24 Feb 2014 19:51:57 +1100 Subject: [Python-ideas] The Return of Implicit Lambda (Re: Allowing breaks in generator expressions by overloading the while keyword) In-Reply-To: <1393230105.34674.YahooMailNeo@web181001.mail.ne1.yahoo.com> References: <20140221102403.GV3684@ando> <1393014891.79562.YahooMailNeo@web181003.mail.ne1.yahoo.com> <1393020176.19093.YahooMailNeo@web181002.mail.ne1.yahoo.com> <1393230105.34674.YahooMailNeo@web181001.mail.ne1.yahoo.com> Message-ID: On Mon, Feb 24, 2014 at 7:21 PM, Andrew Barnert wrote: > So, I think it is worth reviving and extending PEP 312. What would help most in getting that idea considered? Gathering more concrete examples? Buiilding an implementation? Both? > Examples are good. Especially if you can write a script that finds potentials. Feel free to use this script of mine as a basis: https://github.com/Rosuav/ExceptExpr/blob/master/find_except_expr.py ChrisA From ncoghlan at gmail.com Mon Feb 24 10:17:22 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 24 Feb 2014 19:17:22 +1000 Subject: [Python-ideas] The Return of Implicit Lambda (Re: Allowing breaks in generator expressions by overloading the while keyword) In-Reply-To: References: <20140221102403.GV3684@ando> <1393014891.79562.YahooMailNeo@web181003.mail.ne1.yahoo.com> <1393020176.19093.YahooMailNeo@web181002.mail.ne1.yahoo.com> <1393230105.34674.YahooMailNeo@web181001.mail.ne1.yahoo.com> Message-ID: On 24 Feb 2014 18:52, "Chris Angelico" wrote: > > On Mon, Feb 24, 2014 at 7:21 PM, Andrew Barnert wrote: > > So, I think it is worth reviving and extending PEP 312. What would help most in getting that idea considered? Gathering more concrete examples? Buiilding an implementation? Both? > > > > Examples are good. Especially if you can write a script that finds > potentials. Feel free to use this script of mine as a basis: > > https://github.com/Rosuav/ExceptExpr/blob/master/find_except_expr.py Yep, real world examples have always been a challenge in these discussions. Just looking for existing lambda expressions in the stdlib might be a good starting point. Cheers, Nick. > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Mon Feb 24 14:10:58 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Mon, 24 Feb 2014 05:10:58 -0800 (PST) Subject: [Python-ideas] Method chaining notation In-Reply-To: <87d2idv22d.fsf@uwakimon.sk.tsukuba.ac.jp> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <5309DDCE.5020008@gmail.com> <87d2idv22d.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <1393247458.34759.YahooMailNeo@web181001.mail.ne1.yahoo.com> From: Stephen J. Turnbull Sent: Sunday, February 23, 2014 3:31 PM > spir writes: >> On 02/23/2014 06:06 AM, Andrew Barnert wrote: >> > All you're arguing here is that PyGtk is badly designed, or that >> > Gtk is not a good match for Python, so you have to write >> > wrappers. There's no reason the wrapper has to be fluent instead >> > of declarative. > >> That's what I was about to argue. I don't understand why the python >> wrapper does not let you construct widgets in one go, > > *Because* it's a *wrapper*, which leverages the gobject-introspection > FFI.? (gobject-introspection is an export-oriented FFI, rather than an > import-oriented FFI like ctypes.) [snip] > I don't think you need to except subwidgets.? They could be defined > recursively by including calls to their constructors in the > "description" of the parent. I think the problem here is that, to everyone who doesn't like the method chaining idea (like all three of us), it's obvious that this could be done declaratively, and it's also obvious why that would be better than writing code which breaks both Python and Gtk+ idioms, and therefore none of us have explained those obvious facts very well to the people who like the idea. So I put together examples for both the PyGtk example that started this thread and the Java example that started the whole fluent-interface fad, along with an explanation of why method chaining is neither necessary in, nor a good fit for, Python even though it's very useful in languages like Java. See?https://stupid-python-ideas.runkite.com/fluent-pythonic/ for the whole thing. That being said, I think Nick's examples using with statements should be enough to show that there are more Pythonic solutions (that already work today) than adding chaining. From rosuav at gmail.com Mon Feb 24 14:48:54 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 25 Feb 2014 00:48:54 +1100 Subject: [Python-ideas] Method chaining notation In-Reply-To: <1393247458.34759.YahooMailNeo@web181001.mail.ne1.yahoo.com> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <5309DDCE.5020008@gmail.com> <87d2idv22d.fsf@uwakimon.sk.tsukuba.ac.jp> <1393247458.34759.YahooMailNeo@web181001.mail.ne1.yahoo.com> Message-ID: On Tue, Feb 25, 2014 at 12:10 AM, Andrew Barnert wrote: > So I put together examples for both the PyGtk example that started this thread and the Java example that started the whole fluent-interface fad, along with an explanation of why method chaining is neither necessary in, nor a good fit for, Python even though it's very useful in languages like Java. See https://stupid-python-ideas.runkite.com/fluent-pythonic/ for the whole thing. > While I do think a method chaining operator would solve the problem generically, rather than requiring every module to do it individually, I do rather like your proposed window creation syntax. The thing is, Pike GTK is *almost* there: instead of creating a window and setting its title and border, you can create a window and pass it a mapping (dict) specifying the title and border. (PyGTK doesn't have anything of the sort, it seems. I haven't looked into PyGObject, which is supposed to be the new great thing; it might have that.) But it's not quite all the way, because you can't stuff children into them. Being able to do the whole job in the constructor is extremely tempting. It'd work beautifully for the objects where you basically just call add() with each thing (just provide a list of children to be added), but not so well when you want to specify parameters (eg specifying how spare space should be allocated). I'm not sure how that ought to be done. I could come up with something where you pass it a list of tuples, but I'm not sure that completely covers the issue either. There's no perfect solution, which is why the search continues. I freely admit that the suggestion I made at the beginning of this thread is unideal; it's un-Pythonic, it's a heavy language change, and it'd take a huge amount of justification to go anywhere. But there is a problem still, that it's trying to solve. ChrisA From rosuav at gmail.com Mon Feb 24 15:52:28 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 25 Feb 2014 01:52:28 +1100 Subject: [Python-ideas] Infix functions In-Reply-To: References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> <87a9dhuymn.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Mon, Feb 24, 2014 at 7:03 PM, Paul Moore wrote: > But my company provides me with a Windows PC ro use at work, and the > games I like to play are Windows-only. So for work and play, Windows > is my environment, for better or worse. Maybe Windows 8 is so bad that > the climate will shift. Maybe macs will stop costing more than my > house at some point. This is where virtualization wins big. And possibly Wine as well. Maybe both at once - on your Windows machine, create a VM running Linux, and run your Windows program under Wine in that VM... okay, maybe that's just a leeedle bit stupid. But it does have its benefits (like ease of isolating registry changes - I've never seen *any* pure-Windows solution that's as easy as "create a new WINEPREFIX", and doubt I ever will). However... On Mon, Feb 24, 2014 at 11:45 AM, Stephen J. Turnbull wrote: > Andrew's comment is *precisely* true: Python loves Unicode, and > Notepad handles UTF-8 fine. I would imagine any real source editor on > Windows does too. ... Notepad has enough flaws (like its handling of newlines and its pesky BOM header) that I wouldn't recommend it for anything, no matter how good its Unicode in other areas. Yes, it managed to save and load a non-BMP string in such a way that Python managed to retrieve it (although I couldn't find a font that would display it correctly, but that's not a Notepad problem), but unless you're really careful to make your files Notepad-friendly, I wouldn't recommend its use. There are, however, other good editors; SciTE (and presumably other Scintilla-derivatives, like Notepad++) works fine, and doesn't have Notepad's other flaws. So, I'd agree more with the second half of that: any real source editor will be fine, on Windows or any other platform. ChrisA From ron3200 at gmail.com Mon Feb 24 16:01:07 2014 From: ron3200 at gmail.com (Ron Adam) Date: Mon, 24 Feb 2014 09:01:07 -0600 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On 02/24/2014 01:34 AM, Andrew Barnert wrote: >> >On 02/23/2014 05:52 PM, Stephen J. Turnbull wrote: >>> >>Ron Adam writes: >>> >> >>> >> > The operator you want is one for an in place method call. >>> >> >>> >> > seq = [] .= extend(get_data()) .= sort() >>> >> >>> >>That looks like anything but Python to me. >> > >> >Is it really all that different from this? >> > >>>>> > >>>"Py" . __add__("th") . __add__("on") >> >'Python' > Well, yes, it is. But, more importantly, who cares? That code is horribly unreadable and unpythonic. OK, I don't think PEP8 has a guideline saying "don't call __add__ when you can just use +", but only because it's so obvious it doesn't need to be stated. (And I'm pretty sure it_does_ have a guideline saying not to put spaces around the attribute dot.) > > So, is your argument is "my code looks kind of like some horribly unreadable and unpythonic, but legal, code, and therefore it should also be legal despite being unreadable and unpythonic?" Wow, tough crowd here.. :-) Both the separation of the '.', and the use of the already special __add__ isn't important as far as the actual suggestion is concerned. Those are unrelated style issues. You would probably see it used more often like this... def names(defaults, pos_names, pos_args, kwds): return {}.=update(defaults) \ .=update(zip(pos_names, pos_args) \ .=update(kwds) Normally .update returns None. The reason for that is so that it's clear you are mutating an object instead of creating a new one. By using .=, it can return self, but still maintain the clarity between mutation and non-mutation. This particular syntax is consistent with the use of OP+equal to mean mutate in place. But you might prefer, "..", or something else. The other alternative is to use a function. But it would be difficult to get the same behaviour along with the same efficiency. Regards, Ron From p.f.moore at gmail.com Mon Feb 24 16:12:11 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 24 Feb 2014 15:12:11 +0000 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On 24 February 2014 15:01, Ron Adam wrote: > You would probably see it used more often like this... > > def names(defaults, pos_names, pos_args, kwds): > return {}.=update(defaults) \ > .=update(zip(pos_names, pos_args) \ > .=update(kwds) How is this better than def names(defaults, pos_names, pos_args, kwds): ret = {} ret.update(defaults) ret.update(zip(pos_names, pos_args) ret.update(kwds) return ret (I originally named the return value _ to cater for the tendency to insist on punctuation rather than names in this thread, but honestly, why *not* name the thing "ret"?) I get the idea of chained updates, I really do. But translating between mutation of a named value and chained updates is pretty trivial, so I don't see how this is anything but a case of "follow the preferred style for the language/API you're using". And Python uses updating named values, why is that so bad? Paul From steve at pearwood.info Mon Feb 24 16:28:27 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 25 Feb 2014 02:28:27 +1100 Subject: [Python-ideas] Infix functions In-Reply-To: <8761o5uus0.fsf@uwakimon.sk.tsukuba.ac.jp> References: <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> <8AB414A2-5156-4DAB-AC7C-47A6A283CED4@yahoo.com> <20140224013540.GM3684@ando> <8761o5uus0.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20140224152827.GP3684@ando> On Mon, Feb 24, 2014 at 11:08:47AM +0900, Stephen J. Turnbull wrote: > Steven D'Aprano writes: > > > I can't think of many useful examples of infix operators that aren't > > mathematical, and even fewer that aren't just as easily written as > > methods or functions. > > The copula, not to mention all non-mathematical transitive verbs, in > English? This feature would make it a lot easier to write DSLs in > Python. (I don't know if that would necessarily be a good thing.) Copula are "linking verbs" like "be", "is", "was" or "becomes". Under what circumstances would you want to write such operators? Using a single backtick ` as the "custom operator" syntax, I came up with: marriage = two `become one while not (self `be all_that_you_can_be): self.improve() but really, these are just jokes. I'm still no closer to actual use-cases. Any infix binary operator a `op b can be written as a function of two arguments, op(a, b). It's not that there are no use-cases for infix binary operators, but that all the obvious ones (such as string concatenation, comparisons, list repetition, etc.) already exist and the rest can nearly always be easily written as functions. -- Steven From steve at pearwood.info Mon Feb 24 16:16:23 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 25 Feb 2014 02:16:23 +1100 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20140224151623.GO3684@ando> On Mon, Feb 24, 2014 at 09:01:07AM -0600, Ron Adam wrote: > You would probably see it used more often like this... > > def names(defaults, pos_names, pos_args, kwds): > return {}.=update(defaults) \ > .=update(zip(pos_names, pos_args) \ > .=update(kwds) > > > Normally .update returns None. The reason for that is so that it's clear > you are mutating an object instead of creating a new one. > > By using .=, it can return self, but still maintain the clarity between > mutation and non-mutation. How does the update method know whether it is being called via . or via .= ? I'm trying to understand how you think this is supposed to work, and not having much success. Can you give a sketch of how this .= thingy is supposed to operate? > The other alternative is to use a function. But it would be difficult to > get the same behaviour along with the same efficiency. I don't see how you can compare the efficiency of code that can be written now with code that doesn't exist yet. How do you know how efficient your __iget_method__ suggestion will be? -- Steven From rosuav at gmail.com Mon Feb 24 16:48:17 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 25 Feb 2014 02:48:17 +1100 Subject: [Python-ideas] Method chaining notation In-Reply-To: <20140224151623.GO3684@ando> References: <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> <20140224151623.GO3684@ando> Message-ID: On Tue, Feb 25, 2014 at 2:16 AM, Steven D'Aprano wrote: > On Mon, Feb 24, 2014 at 09:01:07AM -0600, Ron Adam wrote: > >> You would probably see it used more often like this... >> >> def names(defaults, pos_names, pos_args, kwds): >> return {}.=update(defaults) \ >> .=update(zip(pos_names, pos_args) \ >> .=update(kwds) >> >> >> Normally .update returns None. The reason for that is so that it's clear >> you are mutating an object instead of creating a new one. >> >> By using .=, it can return self, but still maintain the clarity between >> mutation and non-mutation. > > How does the update method know whether it is being called via . or via > .= ? I'm trying to understand how you think this is supposed to work, > and not having much success. Can you give a sketch of how this .= thingy > is supposed to operate? I don't know how his plan is, but mine was for the function to continue to return None, or 42, or "spam", or whatever it likes, and for that to be ignored. The expression result would be the initial object, and the actual function return value is discarded. Function doesn't need any rewriting. ChrisA From ron3200 at gmail.com Mon Feb 24 17:08:18 2014 From: ron3200 at gmail.com (Ron Adam) Date: Mon, 24 Feb 2014 10:08:18 -0600 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On 02/24/2014 09:12 AM, Paul Moore wrote: > On 24 February 2014 15:01, Ron Adam wrote: >> >You would probably see it used more often like this... >> > >> > def names(defaults, pos_names, pos_args, kwds): >> > return {}.=update(defaults) \ >> > .=update(zip(pos_names, pos_args) \ >> > .=update(kwds) > How is this better than > > def names(defaults, pos_names, pos_args, kwds): > ret = {} > ret.update(defaults) > ret.update(zip(pos_names, pos_args) > ret.update(kwds) > return ret > > (I originally named the return value _ to cater for the tendency to > insist on punctuation rather than names in this thread, but honestly, > why*not* name the thing "ret"?) > > I get the idea of chained updates, I really do. But translating > between mutation of a named value and chained updates is pretty > trivial, so I don't see how this is anything but a case of "follow the > preferred style for the language/API you're using". And Python uses > updating named values, why is that so bad? It's not bad, just not as good. The chained expression is more efficient and can be used in places where you can't use more than a single expression. The point is to maintain both a visual and computational separation of mutable and immutable expressions. Compare the byte code from these. You can see how the chained version would be more efficient. Cheers, Ron def names(defaults, pos_names, pos_args, kwds): ret = {} ret.update(defaults) ret.update(zip(pos_names, pos_args)) ret.update(kwds) return ret >>> dis(names) 2 0 BUILD_MAP 0 3 STORE_FAST 4 (ret) 3 6 LOAD_FAST 4 (ret) 9 LOAD_ATTR 0 (update) 12 LOAD_FAST 0 (defaults) 15 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 18 POP_TOP 4 19 LOAD_FAST 4 (ret) 22 LOAD_ATTR 0 (update) 25 LOAD_GLOBAL 1 (zip) 28 LOAD_FAST 1 (pos_names) 31 LOAD_FAST 2 (pos_args) 34 CALL_FUNCTION 2 (2 positional, 0 keyword pair) 37 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 40 POP_TOP 5 41 LOAD_FAST 4 (ret) 44 LOAD_ATTR 0 (update) 47 LOAD_FAST 3 (kwds) 50 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 53 POP_TOP 6 54 LOAD_FAST 4 (ret) 57 RETURN_VALUE By using the '.' we can see the difference. The byte code should be very close to this, even though this function will give an error if you try to run it. (Can't update None.) The actual difference would probably be replacing LOAD_ATTR with LOAD_MUTATE_ATTR, Which would call __getmutatemethod__ instead of __getmethod__. (or something similar to that, depending on how it's implemented.) def names(defaults, pos_names, pos_args, kwds): return {}.update(defaults) \ .update(zip(pos_names, pos_args)) \ .update(kwds) >>> dis(names) 2 0 BUILD_MAP 0 3 LOAD_ATTR 0 (update) 6 LOAD_FAST 0 (defaults) 9 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 12 LOAD_ATTR 0 (update) 3 15 LOAD_GLOBAL 1 (zip) 18 LOAD_FAST 1 (pos_names) 21 LOAD_FAST 2 (pos_args) 24 CALL_FUNCTION 2 (2 positional, 0 keyword pair) 27 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 30 LOAD_ATTR 0 (update) 4 33 LOAD_FAST 3 (kwds) 36 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 39 RETURN_VALUE From p.f.moore at gmail.com Mon Feb 24 18:03:27 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 24 Feb 2014 17:03:27 +0000 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On 24 February 2014 16:08, Ron Adam wrote: > By using the '.' we can see the difference But the difference is precisely those bytecodes that are needed to replicate the argument that's required because that's not what update does. Show me the bytecode you propose for your proposed operator, and how it's faster, and (assuming it's an improvement) explain why it can't be achieved via bytecode optimisation of the existing code that the compiler could be updated to do. Otherwise your "it's faster" argument doesn't hold water. As regards "it's a single expression", I still say that's purely a style issue - Python doesn't place any emphasis on being able to write things as a single expression (quite the opposite, in fact - complex expressions are generally a sign of bad style in Python). Paul From ron3200 at gmail.com Mon Feb 24 18:03:39 2014 From: ron3200 at gmail.com (Ron Adam) Date: Mon, 24 Feb 2014 11:03:39 -0600 Subject: [Python-ideas] Method chaining notation In-Reply-To: <20140224151623.GO3684@ando> References: <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> <20140224151623.GO3684@ando> Message-ID: On 02/24/2014 09:16 AM, Steven D'Aprano wrote: > On Mon, Feb 24, 2014 at 09:01:07AM -0600, Ron Adam wrote: > >> >You would probably see it used more often like this... >> > >> > def names(defaults, pos_names, pos_args, kwds): >> > return {}.=update(defaults) \ >> > .=update(zip(pos_names, pos_args) \ >> > .=update(kwds) >> > >> > >> >Normally .update returns None. The reason for that is so that it's clear >> >you are mutating an object instead of creating a new one. >> > >> >By using .=, it can return self, but still maintain the clarity between >> >mutation and non-mutation. > How does the update method know whether it is being called via . or via > .= ? I'm trying to understand how you think this is supposed to work, > and not having much success. Can you give a sketch of how this .= thingy > is supposed to operate? > >> >The other alternative is to use a function. But it would be difficult to >> >get the same behaviour along with the same efficiency. > I don't see how you can compare the efficiency of code that can be > written now with code that doesn't exist yet. How do you know how > efficient your __iget_method__ suggestion will be? First off, I need to be more consistent with names. Apologies for that added confusion. I've been just trying to get out the gist of the idea that feels right to me, but haven't worked through the finer details, so for now on, I'll try to be more precise. The byte code for '.' and '.=' will be nearly identical. The "LOAD_ATTR" would be replaced by another byte code. That byte code would add a light wrapper in (C) to check for None, an return self. By doing that, the CALL_FUNCTION byte code deosn't need to change, and there's no need to add the check for None in the bytecode. (Although that's doable too.) Normally a method access with a "." is done with the LOAD_ATTR bytecode, which in turn calls the objects __getattribute__ method. (Is this correct?) For the .= examples, lets use LOAD_I_ATTR for the bytecode and __getiattribute__. (Or other names if you think they would be better.) The .= would differ by using (from the AST) a "LOAD_I_ATTR" in the byte code, which would call a __getiattribute__. If you use ".=" on an object without a __getiattribute__ it would give an error saying you can't mutate that object. When you use ".=" with a method on a mutable object, the call would expect None, and return self. (giving an error if it gets something other than none.) This is different, but not incompatible. This has no effect on using '.' with immutable or mutable objects. Cheers, Ron From random832 at fastmail.us Mon Feb 24 18:18:16 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Mon, 24 Feb 2014 12:18:16 -0500 Subject: [Python-ideas] Infix functions In-Reply-To: References: <1393020304.13161.YahooMailNeo@web181004.mail.ne1.yahoo.com> <5307FA7B.3050707@canterbury.ac.nz> <1393034319.15495.YahooMailNeo@web181005.mail.ne1.yahoo.com> <53080784.3040801@canterbury.ac.nz> <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> Message-ID: <1393262296.18370.87181441.0132D26C@webmail.messagingengine.com> On Sat, Feb 22, 2014, at 22:17, Bruce Leban wrote: > __?__ would complicate the tokenizer as it would have to recognize this > exact syntax and treat it as an identifier while not allowing math symbol > characters in other uses in an identifier. And I probably need to write > sample['?'] not sample.?. Without the quotes or something along those > lines, these look too similar: > > def U(a, b): pass > def ?(a, b): pass def a ? b: pass From ceronman at gmail.com Mon Feb 24 19:17:08 2014 From: ceronman at gmail.com (=?ISO-8859-1?Q?Manuel_Cer=F3n?=) Date: Mon, 24 Feb 2014 19:17:08 +0100 Subject: [Python-ideas] Unify global and nonlocal In-Reply-To: References: Message-ID: On Mon, Feb 24, 2014 at 4:22 AM, Saket Dandawate wrote: > Also why can't it be like this too >>>> global x = 3 > > rather than only >>>> global x >>>> x=3 I can't find the discussion thread, but originally PEP3104 proposed this kind of syntax for nonlocal, but it was never actually implemented because of ambiguity with the unpacking assignment case: nonlocal x, y, z = 1, 2, 3 Does this mean that x, y and z are non locals? or just x? Now I understand the differences between nonlocal and global. But still is not very clear for me why nonlocal can't access the global module namespace as well, not to be used in the same way as global, but to access names already defined in the global namespace. Manuel. From masklinn at masklinn.net Mon Feb 24 20:59:26 2014 From: masklinn at masklinn.net (Masklinn) Date: Mon, 24 Feb 2014 20:59:26 +0100 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <0FA56C9E-D7DE-406B-9586-73B7CC83ACEE@masklinn.net> On 2014-02-24, at 17:08 , Ron Adam wrote: > On 02/24/2014 09:12 AM, Paul Moore wrote: >> On 24 February 2014 15:01, Ron Adam wrote: >>> >You would probably see it used more often like this... >>> > >>> > def names(defaults, pos_names, pos_args, kwds): >>> > return {}.=update(defaults) \ >>> > .=update(zip(pos_names, pos_args) \ >>> > .=update(kwds) >> How is this better than >> >> def names(defaults, pos_names, pos_args, kwds): >> ret = {} >> ret.update(defaults) >> ret.update(zip(pos_names, pos_args) >> ret.update(kwds) >> return ret >> >> (I originally named the return value _ to cater for the tendency to >> insist on punctuation rather than names in this thread, but honestly, >> why*not* name the thing "ret"?) >> >> I get the idea of chained updates, I really do. But translating >> between mutation of a named value and chained updates is pretty >> trivial, so I don't see how this is anything but a case of "follow the >> preferred style for the language/API you're using". And Python uses >> updating named values, why is that so bad? > > It's not bad, just not as good. The chained expression is more efficient and can be used in places where you can't use more than a single expression. > > The point is to maintain both a visual and computational separation of mutable and immutable expressions. > > Compare the byte code from these. You can see how the chained version would be more efficient. The chained version is not intrinsically more efficient, the Python compiler could be smart enough to not LOAD_FAST (ret) repeatedly (if that proves beneficial to execution speed, which I'm not even certain of, and either way it's going to be extremely minor compared to the actual cost of executing methods). AFAIK the peephole optimiser does not even bother eliding out pairs of STORE_FAST $name LOAD_FAST $name e.g. as far as I know a = foo() a.bar() compiles to: 0 LOAD_* 0 (foo) 3 CALL_FUNCTION 0 (0 positional, 0 keyword pair) 6 STORE_FAST 0 (a) 9 LOAD_FAST 0 (a) 12 LOAD_ATTR 1 (bar) 15 CALL_FUNCTION 0 (0 positional, 0 keyword pair) 18 POP_TOP the pair (6, 9) is a noop and could trivially be removed (in the absence of jumps around). According to [0] a patch implementing this (although without taking care of jumps) was rejected: > because apparently the additional six lines of code didn?t buy > enough of a speed improvement for an uncommon case. (although no link to the patch so he might have been optimizing the triplet of (STORE_FAST, LOAD_FAST, RETURN_VALUE)). If removing 2 bytecode instructions once in a while does not sway the core team, I really can't see removing a single one even more rarely doing so. > By using the '.' we can see the difference. The byte code should be very close to this, even though this function will give an error if you try to run it. (Can't update None.) The actual difference would probably be replacing LOAD_ATTR with LOAD_MUTATE_ATTR, Which would call __getmutatemethod__ instead of __getmethod__. (or something similar to that, depending on how it's implemented.) Why? There's no need for LOAD_MUTATE_ATTR. And LOAD_ATTR calls __getattribute__ (and __getattr__ if necessary), a bound method is a form callable attribute, the bytecode for a method call (assuming an object on the stack) is LOAD_ATTR $attrname CALL_FUNCTION that the function mutates the original object (or not) has no relevance to attribute loading. > def names(defaults, pos_names, pos_args, kwds): > return {}.update(defaults) \ > .update(zip(pos_names, pos_args)) \ > .update(kwds) > > >>> dis(names) > 2 0 BUILD_MAP 0 > 3 LOAD_ATTR 0 (update) > 6 LOAD_FAST 0 (defaults) > 9 CALL_FUNCTION 1 (1 positional, 0 keyword pair) > 12 LOAD_ATTR 0 (update) > > 3 15 LOAD_GLOBAL 1 (zip) > 18 LOAD_FAST 1 (pos_names) > 21 LOAD_FAST 2 (pos_args) > 24 CALL_FUNCTION 2 (2 positional, 0 keyword pair) > 27 CALL_FUNCTION 1 (1 positional, 0 keyword pair) > 30 LOAD_ATTR 0 (update) > > 4 33 LOAD_FAST 3 (kwds) > 36 CALL_FUNCTION 1 (1 positional, 0 keyword pair) > 39 RETURN_VALUE That bytecode's not correct for the case: * the return value of each method call needs to be discarded with a POP_TOP * LOAD_ATTR needs an object on the stack so you need a DUP_TOP before each LOAD_ATTR (update) (you can create the correct bytecode with something like byteplay, it'll work) [0] http://www.coactivate.org/projects/topp-engineering/blog/2008/11/03/optimizing-python/ From ncoghlan at gmail.com Mon Feb 24 22:40:05 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 25 Feb 2014 07:40:05 +1000 Subject: [Python-ideas] Unify global and nonlocal In-Reply-To: References: Message-ID: On 25 Feb 2014 04:18, "Manuel Cer?n" wrote: > > On Mon, Feb 24, 2014 at 4:22 AM, Saket Dandawate wrote: > > Also why can't it be like this too > >>>> global x = 3 > > > > rather than only > >>>> global x > >>>> x=3 > > I can't find the discussion thread, but originally PEP3104 proposed > this kind of syntax for nonlocal, but it was never actually > implemented because of ambiguity with the unpacking assignment case: > > nonlocal x, y, z = 1, 2, 3 > > Does this mean that x, y and z are non locals? or just x? > > Now I understand the differences between nonlocal and global. But > still is not very clear for me why nonlocal can't access the global > module namespace as well, not to be used in the same way as global, > but to access names already defined in the global namespace. Because functions are compiled as a unit, allowing nonlocal references to be checked at compile time, but globals are resolved lazily and hence aren't checked until runtime. There's no way to flip a reference from a closure variable to a global dynamically, so the compiler treats an explicit closure reference (i.e. a nonlocal declaration) to a name that doesn't exist as an error (either the variable name is wrong or the user meant to write global rather than nonlocal). Cheers, Nick. > > Manuel. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Mon Feb 24 23:48:14 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 25 Feb 2014 09:48:14 +1100 Subject: [Python-ideas] Method chaining notation In-Reply-To: <0FA56C9E-D7DE-406B-9586-73B7CC83ACEE@masklinn.net> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> <0FA56C9E-D7DE-406B-9586-73B7CC83ACEE@masklinn.net> Message-ID: On Tue, Feb 25, 2014 at 6:59 AM, Masklinn wrote: > a = foo() > a.bar() > > compiles to: > > 0 LOAD_* 0 (foo) > 3 CALL_FUNCTION 0 (0 positional, 0 keyword pair) > 6 STORE_FAST 0 (a) > > 9 LOAD_FAST 0 (a) > 12 LOAD_ATTR 1 (bar) > 15 CALL_FUNCTION 0 (0 positional, 0 keyword pair) > 18 POP_TOP > > the pair (6, 9) is a noop and could trivially be removed (in the absence > of jumps around). According to [0] a patch implementing this (although > without taking care of jumps) was rejected: Possible reason for rejection: The optimizer would have to be sure that a wasn't used anywhere else. a = foo() a.bar() a.spam() 2 0 LOAD_GLOBAL 0 (foo) 3 CALL_FUNCTION 0 (0 positional, 0 keyword pair) 6 STORE_FAST 0 (a) 3 9 LOAD_FAST 0 (a) 12 LOAD_ATTR 1 (bar) 15 CALL_FUNCTION 0 (0 positional, 0 keyword pair) 18 POP_TOP 4 19 LOAD_FAST 0 (a) 22 LOAD_ATTR 2 (spam) 25 CALL_FUNCTION 0 (0 positional, 0 keyword pair) 28 POP_TOP The subsequent LOAD_FAST of a depends on the STORE_FAST having been done. In the specific case mentioned in your link, he was looking for a RETURN_VALUE opcode, so that would be safe. (But if there really is code like he's seeing, I'd look at tidying it up on the Python source level. Just return the value directly. No need for "single exit point" in Python code.) ChrisA From abarnert at yahoo.com Tue Feb 25 00:29:27 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Mon, 24 Feb 2014 15:29:27 -0800 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Feb 24, 2014, at 7:01, Ron Adam wrote: > On 02/24/2014 01:34 AM, Andrew Barnert wrote: >> >> So, is your argument is "my code looks kind of like some horribly unreadable and unpythonic, but legal, code, and therefore it should also be legal despite being unreadable and unpythonic?" > > Wow, tough crowd here.. :-) > > Both the separation of the '.', and the use of the already special __add__ isn't important as far as the actual suggestion is concerned. Those are unrelated style issues. > > You would probably see it used more often like this... > > def names(defaults, pos_names, pos_args, kwds): > return {}.=update(defaults) \ > .=update(zip(pos_names, pos_args) \ > .=update(kwds) > > > Normally .update returns None. The reason for that is so that it's clear you are mutating an object instead of creating a new one. And so you can't chain it, so you can only mutate one thing in a statement. People keep assuming that's an accidental unwanted side effect of the rule, but Guido explicitly saying that he doesn't like method chaining, can't you imagine it's at least possible that this is intentional, not a bug? > By using .=, it can return self, but still maintain the clarity between mutation and non-mutation. > > This particular syntax is consistent with the use of OP+equal to mean mutate in place. But you might prefer, "..", or something else. You're missing a key distinction here. OP+equal does not return self. In fact, it doesn't return _anything_, because it's not an expression at all, it's a statement. It would be very easy to make augmented assignment an expression, and even easier to make it return self (after all, it's implemented by calling dunder methods that _do_ return self!). But it wasn't designed that way. Intentionally. > The other alternative is to use a function. But it would be difficult to get the same behaviour along with the same efficiency. How do you think a regular function would have less efficiency than an operator? Operators work by doing a complex chain of lookups to find a function to call. Ordinary calls do a simpler lookup. They both call the function the same way once they find it. You're also missing the other alternative: write it Pythonically. For example: return merge_dicts( defaults, zip(pos_names, pos_args), kwds) It's shorter, it has less extraneous syntax, it doesn't need awkward backslash continuations, and it created or modifies one value in one place. It's easier to read, and easier to reason about. Why would you want to write it the other way? Of course that merge_dicts function isn't in the stdlib. Maybe it should be. But you can write it yourself trivially. For example: def merge_dicts(*args): return {k: v for arg in args for (k, v) in dict(arg).items()} That nested comprehension might be a little too complicated; if you think so, you can split it into two expressions, or even write an explicit loop around dict.update. Whatever; this is something you write once and use every time you want to merge a bunch of dicts. From abarnert at yahoo.com Tue Feb 25 01:11:19 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Mon, 24 Feb 2014 16:11:19 -0800 Subject: [Python-ideas] Infix functions In-Reply-To: <20140224152827.GP3684@ando> References: <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> <8AB414A2-5156-4DAB-AC7C-47A6A283CED4@yahoo.com> <20140224013540.GM3684@ando> <8761o5uus0.fsf@uwakimon.sk.tsukuba.ac.jp> <20140224152827.GP3684@ando> Message-ID: <06A045B6-C169-4886-B19A-EC932F6A95BD@yahoo.com> On Feb 24, 2014, at 7:28, Steven D'Aprano wrote: > but really, these are just jokes. I'm still no closer to actual > use-cases. I will repeat one that I already posted, but try to put it differently: a using function that wraps a with statement. Every kind of statement in Python would sometimes be useful as an expression. You can always wrap the statement in a function, but it looks ugly--even with Nick's concise lambda proposal. For example, lets say we wrapped with, try/except, and if/else in functions named using, catch, and cond: data = using(:?.read(), open(path)) buf = catch(:read_file(path), (FileNotFoundError, :'') b = cond(a, :1/a, :NaN) All of these are possible today, but were not sufficient to curb the desire for with, except, and if expressions. One was already added, one is under consideration, and the last gets suggested at least once/year. But if you could infix those functions: data = :?.read() `using` open(path) buf = :read_file(path) `catch` FileNotFoundError, :'' b = a `cond` :1/a, :NaN Would they then be good enough to obviate the need for new syntax? I assume not having to add a corresponding expression for every statement is a desirable goal. The only question is whether this would help achieve that goal. > Any infix binary operator a `op b can be written as a function of two > arguments, op(a, b). Of course. The whole point of the idea--in the very first paragraph of the initial post--is that a `op` b would be compile to the exact same code as op(a, b). (Others have suggested that maybe a method would be better, but that doesn't change the point.) This is pure syntactic sugar for those functions, nothing else. > It's not that there are no use-cases for infix > binary operators, but that all the obvious ones (such as string > concatenation, comparisons, list repetition, etc.) already exist and the > rest can nearly always be easily written as functions. The question is not whether they can be written as functions, but whether those functions' calls can be read more easily with infix syntax. From steve at pearwood.info Tue Feb 25 02:47:44 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 25 Feb 2014 12:47:44 +1100 Subject: [Python-ideas] Infix functions In-Reply-To: <06A045B6-C169-4886-B19A-EC932F6A95BD@yahoo.com> References: <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> <8AB414A2-5156-4DAB-AC7C-47A6A283CED4@yahoo.com> <20140224013540.GM3684@ando> <8761o5uus0.fsf@uwakimon.sk.tsukuba.ac.jp> <20140224152827.GP3684@ando> <06A045B6-C169-4886-B19A-EC932F6A95BD@yahoo.com> Message-ID: <20140225014743.GR3684@ando> On Mon, Feb 24, 2014 at 04:11:19PM -0800, Andrew Barnert wrote: > On Feb 24, 2014, at 7:28, Steven D'Aprano wrote: > > > but really, these are just jokes. I'm still no closer to actual > > use-cases. > > I will repeat one that I already posted, but try to put it > differently: a using function that wraps a with statement. > > Every kind of statement in Python would sometimes be useful as an > expression. You can always wrap the statement in a function, but it > looks ugly--even with Nick's concise lambda proposal. For example, > lets say we wrapped with, try/except, and if/else in functions named > using, catch, and cond: > > data = using(:?.read(), open(path)) > buf = catch(:read_file(path), (FileNotFoundError, :'') > b = cond(a, :1/a, :NaN) > > All of these are possible today, You must have a different understanding of "possible" than I do, because they all give me syntax errors. > but were not sufficient to curb the desire for with, except, and if > expressions. One was already added, one is under consideration, and > the last gets suggested at least once/year. > > > But if you could infix those functions: > > data = :?.read() `using` open(path) > buf = :read_file(path) `catch` FileNotFoundError, :'' > b = a `cond` :1/a, :NaN > > Would they then be good enough to obviate the need for new syntax? All of those are completely unreadable messes to me. If you're trying to sell the idea of one new controversial feature, you really shouldn't demonstrate it with another controversial new syntax. Let me try to make your argument for you, I hope fairly. You're hoping that by introducing custom infix binary operators, you can replace syntax for (say) result = 23 if condition else 42 with result = 23 `if_operator` condition, 42 or something similar. The fact that ternary if already exists isn't important here -- it is your argument that if we had custom infix operators, we wouldn't have needed to add ternary if. I don't think this suggestion comes even close to working well. Without proper support interpreter support for ternary operators, forcing three arguments around a binary operator is never going to look right, and may have precedence issues. And it lacks support for lazy evaluation, which means you're stuck with an ugly work-around of wrapping operands in functions and delaying calling the function when you want laziness: # define the two getter functions elsewhere: result = (get_true_value `if_operator` condition, get_false_value)() # in-place, using lambda: result = ((lambda: true_expression) `if_operator` condition, (lambda: false_expression))() # using proposed lambda alternative result = (:true_expression `if_operator` condition, :false_expression)() None of which are as good as the syntax we have for ternary if. This suggests that dedicated syntax beats custom syntax. [Aside: one can remove the outer pair of parentheses by embedding the final call inside the custom operator, but that would mean you *always* have to use functions as arguments, even when lazy evaluation is unnecessary. That's a bad idea.] Not to put too fine a point to it, mapping ternary statements like if...else and try...except to an infix binary operator sucks. Fortunately there are only two of those, one is already syntax and the other is subject to a PEP. What's left? There aren't that many expressions left... for-loops can be written as list comps, and besides they require a name-binding so don't fit well to this model; del doesn't need to be an expression, because it doesn't return anything; likewise for pass; assignment-as-expression goes against the grain of the language; import can be written as __import__ if you really must; it might be handy to raise an exception from inside an expression, but I don't see that this maps to your proposal; classes can be created with type(); and functions with lambda. Have I missed anything? This leaves perhaps while loops and with statement, and I don't think that they're going to map any better to this proposal than did if. result = expression `while_operator` condition doesn't work for me. It's unclear that the result is a list (or a tuple? set? lazy generator?), but even if we decided that you can map while and with statements to infix binary operators, I don't think that having an over-generalised system like this is an improvement over dedicated while-comprehensions and with-expressions. Besides, there is strong value in having (say) a single way to spell a for expression, namely [blah for name in seq], rather than a plethora of custom infix operators: # assume there is magic for name-binding somewhere result = blah `for_operator` name, seq result = blah `for_loop` seq, name result = name, blah `do` seq result = seq `repeat` name, blah etc., depending on the personal preferences of the coder writing it. I've given this idea a good shake, and I think it's a limp rag. I don't think that mapping statements to infix binary operators in this way is a workable idea. I'll be honest, it seems to be *so grossly and fundamentally unworkable* to me that I'm kind of astonished that you've raised it not once but twice, instead of discarding it as ridiculous, like the idea of a paper submarine or a hot iron balloon. Perhaps there are other requirements needed to make this work that you haven't mentioned and I haven't thought of? Sorry to be so dismissive, but that's the truth of it. [...] > > Any infix binary operator a `op b can be written as a function of two > > arguments, op(a, b). > > Of course. The whole point of the idea--in the very first paragraph of > the initial post--is that a `op` b would be compile to the exact same > code as op(a, b). (Others have suggested that maybe a method would be > better, but that doesn't change the point.) This is pure syntactic > sugar for those functions, nothing else. > > > It's not that there are no use-cases for infix > > binary operators, but that all the obvious ones (such as string > > concatenation, comparisons, list repetition, etc.) already exist and the > > rest can nearly always be easily written as functions. > > The question is not whether they can be written as functions, but > whether those functions' calls can be read more easily with infix > syntax. That's not the only question. A very important question is whether it is worth adding all this generalized infix operator machinary just so that people can write: x `becomes` y rather than becomes(x, y) If this is *pure syntactic sugar* with absolutely no extra features, as you suggest, then I think we're just wasting our time discussing it. It's not that I'm entirely against sugar, I like the fact that I can do string repetition with "spam"*23 rather than "spam".repeat(23), but (re-iterating what I said earlier) all the important and obvious flavours of sugar have already been done. Adding more will give you source code diabetes. If we've missed one or two use-cases, perhaps we can add operator support for those, e.g. "spam" - "eggs" (whatever that might mean!), without needing to over-generalise the concept to support arbitrary operators everywhere. -- Steven From rosuav at gmail.com Tue Feb 25 03:18:11 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 25 Feb 2014 13:18:11 +1100 Subject: [Python-ideas] Infix functions In-Reply-To: <20140225014743.GR3684@ando> References: <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> <8AB414A2-5156-4DAB-AC7C-47A6A283CED4@yahoo.com> <20140224013540.GM3684@ando> <8761o5uus0.fsf@uwakimon.sk.tsukuba.ac.jp> <20140224152827.GP3684@ando> <06A045B6-C169-4886-B19A-EC932F6A95BD@yahoo.com> <20140225014743.GR3684@ando> Message-ID: On Tue, Feb 25, 2014 at 12:47 PM, Steven D'Aprano wrote: > it might be handy to raise an exception from inside an > expression, but I don't see that this maps to your proposal; It's easy enough to write, anyway. def throw(x): raise x I've used that in a few syntactic demos, but never in production code. ChrisA From abarnert at yahoo.com Tue Feb 25 04:46:44 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Mon, 24 Feb 2014 19:46:44 -0800 (PST) Subject: [Python-ideas] Infix functions In-Reply-To: <20140225014743.GR3684@ando> References: <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> <8AB414A2-5156-4DAB-AC7C-47A6A283CED4@yahoo.com> <20140224013540.GM3684@ando> <8761o5uus0.fsf@uwakimon.sk.tsukuba.ac.jp> <20140224152827.GP3684@ando> <06A045B6-C169-4886-B19A-EC932F6A95BD@yahoo.com> <20140225014743.GR3684@ando> Message-ID: <1393300004.18792.YahooMailNeo@web181002.mail.ne1.yahoo.com> From: Steven D'Aprano Sent: Monday, February 24, 2014 5:47 PM > On Mon, Feb 24, 2014 at 04:11:19PM -0800, Andrew Barnert wrote: >> >> Every kind of statement in Python would sometimes be useful as an >> expression. You can always wrap the statement in a function, but it >> looks ugly--even with Nick's concise lambda proposal. For example, >> lets say we wrapped with, try/except, and if/else in functions named >> using, catch, and cond: >> >> ? ? data = using(:?.read(), open(path)) >> ? ? buf = catch(:read_file(path), (FileNotFoundError, :'') >> ? ? b = cond(a, :1/a, :NaN) >> >> All of these are possible today, > > You must have a different understanding of "possible" than I do, > because they all give me syntax errors. I explicitly said "even with Nick's concise lambda proposal". Have you worked out an implementation for that proposal and patched your compiler? If not, yeah, you will get syntax errors. Without that proposal, they're even worse, hence the "even with?": ? ? ?data = using(lambda f: f.read(), open(path)) ? ? ?buf = catch(lambda: read_file(path), (FileNotFoundError, lambda: '') ? ? ?b = cond(a, lambda: 1/a, lambda: NaN) > If you're trying to? > sell the idea of one new controversial feature, you really shouldn't > demonstrate it with another controversial new syntax. As I already explained, I don't think the infix functions are worthwhile without some fix for lambdas being too obtrusive?whether Nick's or a different one. So, should I just pretend that problem already had a solution and ignore it? > Let me try to make your argument for you, I hope fairly. You're hoping? > that by introducing custom infix binary operators, you can replace > syntax for (say) > > ? ? result = 23 if condition else 42 > > with > > ? ? result = 23 `if_operator` condition, 42 >? > or something similar. > > The fact that ternary if already exists isn't? > important here -- it is your argument that if we had custom infix > operators, we wouldn't have needed to add ternary if. > > I don't think this suggestion comes even close to working well. Without? > proper support interpreter support for ternary operators, forcing three > arguments around a binary operator is never going to look right, and > may have precedence issues.? You say you're trying to be fair here, but then why have you ignored the two cases that are binary (which I listed first), and then rearranged the third case to make it harder to parse and to read? Yes, the if expression we ended up with is also out-of-order. But I wrote the function in the order of the if statement, the ternary if expression in most other languages, and the initial proposals that everyone wanted until nobody could get the syntax right. And that makes it a lot clearer: it's a binary operator between a condition and a tuple of then/else values. (Sure, that makes it read more like a two-case indexed switch than an if, but it's still a lot better than the way you wrote it.) > And it lacks support for lazy evaluation,? I've mentioned from the start, and repeatedly, that this relies on some other solution to that problem, like Nick's concise lambdas. I don't know how else I can say it at this point that will stop someone from saying, "But that doesn't solve lazy evaluation." > None of which are as good as the syntax we have for ternary if. Which, again, I explicitly said multiple times, including in the very section of the message you're replying to. > This suggests that dedicated syntax beats custom syntax. Yes. That's almost _universally_ true. But that doesn't mean you should create dedicated syntax for every kind of expression anyone ever dreams up.?That way lies madness, or at least Lisp. The bar is very high for what deserves dedicated syntax. The except expression may just barely pass it, but the verdict is still out; the with expression doesn't seem likely to clear; some expression that you need for your own project like piping to an Erlang-style channel or binding two dataflow variables?definitely won't. > I'll be honest, it seems to be *so grossly and fundamentally unworkable*? > to me that I'm kind of astonished that you've raised it not once but > twice, instead of discarding it as ridiculous, like the idea of a paper > submarine or a hot iron balloon. Perhaps there are other requirements > needed to make this work that you haven't mentioned and I haven't > thought of? Sorry to be so dismissive, but that's the truth of it. If you want me to be honest, I think you haven't actually read most of what I've mentioned. Otherwise, you wouldn't be repeating things that I said as if they were arguments I'd never considered, or trying to imagine what something might look like directly under an example of what it looks like. Here's another example: > If we've missed one or two use-cases, perhaps we can add operator > support for those, e.g. "spam" - "eggs" (whatever that might > mean!), > without needing to over-generalise the concept to support arbitrary > operators everywhere. Here's what I said in a previous reply to you: > Turning with and except into readable expressions without needing to add new custom syntax for each was my initial motivation. It's possible there are only a handful of such cases, and only one of them is important enough to be worth doing, in which case custom syntax for that one is the best answer. So, do you really think that I haven't considered the possibility that, if there are only one or two use-cases. dedicated custom syntax for those one or two use cases are a better answer, because I only said that about "one" rather than "one or two"? Or are you just assuming that you know the arguments and don't need to read them? From kn0m0n3 at gmail.com Tue Feb 25 04:49:58 2014 From: kn0m0n3 at gmail.com (Jason Bursey) Date: Mon, 24 Feb 2014 21:49:58 -0600 Subject: [Python-ideas] hey Stephanie, this is DJ, how are you? In-Reply-To: References: Message-ID: I went to Amsterdam bout a year ago, but am looking to move from Dallas to Seattle were I could win a case. I think a 501(c) to help relocation of patients is needed. Cheers, j On Friday, January 31, 2014, Stephanie Bishop (Googlersa with search r s a primes with shorsalgorthm=1&Flintstones findprimenumber. Low kelvin +) wrote: > < https://lh6.googleusercontent.com/-aRFDqfZ-Llo/AAAAAAAAAAI/AAAAAAAAAi8/7uQsVkrgCoo/s75-c-k-a-no/photo.jpg > > I am fabulous. Is this the DJ from Europe? S > Reply to this email to comment on Google+. Or view post ? > Stephanie Bishop commented on your post. Mute Stephanie Bishop to stop receiving notifications from her. Mute updates to this post. This notification was sent to kn0m0n3 at gmail.com; Go to your notification delivery settings to update your address. Manage subscriptions to change what emails you receive from Google+. > Privacy tip: Protect your info. Remove your email signature before you reply. > Google Inc., 1600 Amphitheatre Pkwy, Mountain View, CA 94043 USA > < https://ci4.googleusercontent.com/proxy/GYehbMfqpOfmkZni3YXcVpYFnSdFa4_3HNmCzVHxFFhtCBk_QulXrkB97v_UVSU0gt8t42RnDKOqw0SvszkMjvrdKHZjm3UErjYHQI7vsurAMj3tGuzIFiqw8xIvgCy_aoN9ujcdkHDJYGLdO9h6jySufmfLtNIRr8tXVfdR=s0-d-e1-ft#https://ssl.gstatic.com/s2/oz/images/notifications/logo/google-plus-6617a72bb36cc548861652780c9e6ff1.png > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Tue Feb 25 09:21:00 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 25 Feb 2014 17:21:00 +0900 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87zjlftxg3.fsf@uwakimon.sk.tsukuba.ac.jp> Ron Adam writes: > You would probably see it used more often like this... > > def names(defaults, pos_names, pos_args, kwds): > return {}.=update(defaults) \ > .=update(zip(pos_names, pos_args) \ > .=update(kwds) I actually have a bunch of code in one of my apps that implements the same thing for a different reason (cascading configs), but my implementation is def names(defaults, pos_names, pos_args, kwds): for dct in pos_names, pos_args, kwds: defaults.update(dct) return defaults The other obvious use for this (as several have posted) is accumulating a sequence. In which case most uses will be well-handled with a genexp, or if you need a concrete sequence, a listcomp, and the body becomes a one (logical) liner (although it will very likely be formatted in multiple lines). From stephen at xemacs.org Tue Feb 25 09:43:13 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 25 Feb 2014 17:43:13 +0900 Subject: [Python-ideas] Infix functions In-Reply-To: <20140224152827.GP3684@ando> References: <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> <8AB414A2-5156-4DAB-AC7C-47A6A283CED4@yahoo.com> <20140224013540.GM3684@ando> <8761o5uus0.fsf@uwakimon.sk.tsukuba.ac.jp> <20140224152827.GP3684@ando> Message-ID: <87y50ztwf2.fsf@uwakimon.sk.tsukuba.ac.jp> Steven D'Aprano writes: > Copula are "linking verbs" like "be", "is", "was" or "becomes". > > Under what circumstances would you want to write such operators? For one, in establishing object identity. "is" *is* an operator in Python. > Any infix binary operator a `op b can be written as a function of two > arguments, op(a, b). It's not that there are no use-cases for infix > binary operators, but that all the obvious ones (such as string > concatenation, comparisons, list repetition, etc.) already exist and the > rest can nearly always be easily written as functions. All the obvious ones (such as binary math operators) also can written as functions: def polymorphically_add_things(x, y): return x + y We know where that leads to (hi, John McC!). Obviously, that's not logic you're willing to follow to the bitter end! As for use cases, how about DSLs: @make_named_operator def to(x, y): return (x, y) # There really oughtta be a Graph class with an appropriate repr. tricycle = [ 'a' `to 'b', 'b' `to 'c', 'c' `to 'a'] whine("But, Daddy!!!! I wanna PONEEEEEEEE!!") (Sorry, bad cold, can't remember any use of graph theory in Monty Python.) As I said, I dunno if easy creation of DSLs is desirable. And of course it's a toy example, but I don't think it's fair to hold that against me; a real example would take up a lot of space without being any more convincing, I suspect. From ron3200 at gmail.com Tue Feb 25 09:53:49 2014 From: ron3200 at gmail.com (Ron Adam) Date: Tue, 25 Feb 2014 02:53:49 -0600 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On 02/24/2014 05:29 PM, Andrew Barnert wrote: > On Feb 24, 2014, at 7:01, Ron > Adam wrote: > >>> On 02/24/2014 01:34 AM, Andrew Barnert wrote: >>>>> >>>>> So, is your argument is "my code looks kind of like some >>>>> horribly unreadable and unpythonic, but legal, code, and >>>>> therefore it should also be legal despite being unreadable and >>>>> unpythonic?" >>> >>> Wow, tough crowd here..:-) >>> >>> Both the separation of the '.', and the use of the already special >>> __add__ isn't important as far as the actual suggestion is >>> concerned. Those are unrelated style issues. >>> >>> You would probably see it used more often like this... >>> >>> def names(defaults, pos_names, pos_args, kwds): >>> return {}.=update(defaults) \ >>> .=update(zip(pos_names, pos_args) \ >>> .=update(kwds) >>> >>> Normally .update returns None. The reason for that is so that it's >>> clear you are mutating an object instead of creating a new one. > And so you can't chain it, so you can only mutate one thing in a > statement. People keep assuming that's an accidental unwanted side > effect of the rule, but Guido explicitly saying that he doesn't like > method chaining, can't you imagine it's at least possible that this is > intentional, not a bug? Yes, I know it's an intentional design choice. Did he ever say why he doesn't like chained methods? I tried to look it up, and all I found was a lot of other people saying he doesn't. But nothing that indicated what his reasoning for it was. >>> By using .=, it can return self, but still maintain the clarity >>> between mutation and non-mutation. >>> >>> This particular syntax is consistent with the use of OP+equal to >>> mean mutate in place. But you might prefer, "..", or something >>> else. > You're missing a key distinction here. OP+equal does not return self. In > fact, it doesn't return_anything_, because it's not an expression at > all, it's a statement. I was referring to the dunder method that gets called as you noted below. > It would be very easy to make augmented assignment an expression, and > even easier to make it return self (after all, it's implemented by > calling dunder methods that_do_ return self!). But it wasn't designed > that way. Intentionally. How else would it be designed? >>> The other alternative is to use a function. But it would be >>> difficult to get the same behaviour along with the same efficiency. > How do you think a regular function would have less efficiency than an > operator? Operators work by doing a complex chain of lookups to find a > function to call. Ordinary calls do a simpler lookup. They both call the > function the same way once they find it. Ok, you lost me with this.. what complex chain of lookups are you referring to? (Besides the normal name and method resolution.) Operators are one level above methods... And yes, so are direct method calls. But the alternative I was talking about was to write a function to get the same behaviour with chained methods as I was describing, That adds another layer, so obviously it wouldn't be quite as efficient. I wasn't saying functions aren't efficient. > You're also missing the other alternative: write it Pythonically. For > example: Never said writing functions was bad. I use functions all the time to make code nicer and cleaner. So no, I'm not missing that. > return merge_dicts( defaults, zip(pos_names, pos_args), kwds) > > It's shorter, it has less extraneous syntax, it doesn't need awkward > backslash continuations, and it created or modifies one value in one > place. It's easier to read, and easier to reason about. Why would you > want to write it the other way? > > Of course that merge_dicts function isn't in the stdlib. Maybe it should > be. But you can write it yourself trivially. For example: > > def merge_dicts(*args): return {k: v for arg in args for (k, v) in > dict(arg).items()} > > That nested comprehension might be a little too complicated; if you > think so, you can split it into two expressions, or even write an > explicit loop around dict.update. Whatever; this is something you write > once and use every time you want to merge a bunch of dicts. Much slower too. Yes, the dict.update is nicer and quicker than the comprehension. I'd probably do it this way... def merge_dicts(*args): D = {} for a in args: D.update(a) return D Chaining methods isn't a do everything everywhere kind of thing. There are times when it's handy and times when a function is better. Cheers, Ron From ram.rachum at gmail.com Tue Feb 25 11:02:15 2014 From: ram.rachum at gmail.com (Ram Rachum) Date: Tue, 25 Feb 2014 02:02:15 -0800 (PST) Subject: [Python-ideas] Allow __len__ to return infinity Message-ID: <3517a862-205d-46f0-8a0f-b93e38451945@googlegroups.com> I'd like to have some objects that return infinity from their __len__ method. Unfortunately the __len__ method may only return an int, and it's impossible to represent an infinity as an int. Do you think that Python could allow returning infinity from __len__? Thanks, Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue Feb 25 12:18:20 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 25 Feb 2014 22:18:20 +1100 Subject: [Python-ideas] Allow __len__ to return infinity In-Reply-To: <3517a862-205d-46f0-8a0f-b93e38451945@googlegroups.com> References: <3517a862-205d-46f0-8a0f-b93e38451945@googlegroups.com> Message-ID: <20140225111819.GS3684@ando> On Tue, Feb 25, 2014 at 02:02:15AM -0800, Ram Rachum wrote: > I'd like to have some objects that return infinity from their __len__ > method. Unfortunately the __len__ method may only return an int, and it's > impossible to represent an infinity as an int. Do you think that Python > could allow returning infinity from __len__? "Could"? Of course. Almost anything is possible. "Should"? No. Allowing __len__ to return float('inf') would imply one of two alternatives, both equally unpalatable. (1) __len__ can return *any* float, regardless of value, including lengths of 0.5, NAN, 1e300, etc. This is undesirable because lengths of sequences should be positive or zero whole numbers, not arbitrary floating point values like 4.5. (2) __len__ cannot return any float, but only INF. Which means that the condition that len() only returns ints will be broken in the most surprising way, with a single exception. I can see a few alternatives: - Perhaps ints should grow a pair of special values, +INF and -INF. I'm willing to be persuaded that this is a good idea. - Or perhaps __len__ could loosen the restriction that the value returned is non-negative. Perhaps -1 (or any negative length) could stand in for "infinitely long". Although I think that would be error-prone and cause more trouble than it solves. - If not, perhaps you could use sys.maxsize as a stand-in for "infinitely long". After all, if a sequence has 2147483647 items, that's effectively infinite for most purposes. There's another reason: the current implementation of len() in CPython requires that the length fit in a C long: py> class X: ... def __len__(self): ... return sys.maxsize + 1 ... py> x = X() py> len(x) Traceback (most recent call last): File "", line 1, in OverflowError: cannot fit 'int' into an index-sized integer I'm curious what your use-case for len() returning INF might be. -- Steven From p.f.moore at gmail.com Tue Feb 25 12:46:26 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 25 Feb 2014 11:46:26 +0000 Subject: [Python-ideas] Allow __len__ to return infinity In-Reply-To: <20140225111819.GS3684@ando> References: <3517a862-205d-46f0-8a0f-b93e38451945@googlegroups.com> <20140225111819.GS3684@ando> Message-ID: On 25 February 2014 11:18, Steven D'Aprano wrote: > (1) __len__ can return *any* float, regardless of value, including > lengths of 0.5, NAN, 1e300, etc. This is undesirable because lengths of > sequences should be positive or zero whole numbers, not arbitrary > floating point values like 4.5. > > (2) __len__ cannot return any float, but only INF. Which means that the > condition that len() only returns ints will be broken in the most > surprising way, with a single exception. Either of these would break range(len(x)) at a minimum, and probably most other code that uses len(). So you'd be left with code having to identify "objects that only return finite len" and "objects that could have one of the new len values". And if the OP can detect that, he can just do so in his code and special case his objects, rather than changing the len protocol. Paul From solipsis at pitrou.net Tue Feb 25 12:51:17 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 25 Feb 2014 12:51:17 +0100 Subject: [Python-ideas] Allow __len__ to return infinity References: <3517a862-205d-46f0-8a0f-b93e38451945@googlegroups.com> Message-ID: <20140225125117.1632be44@fsol> On Tue, 25 Feb 2014 02:02:15 -0800 (PST) Ram Rachum wrote: > I'd like to have some objects that return infinity from their __len__ > method. Unfortunately the __len__ method may only return an int, and it's > impossible to represent an infinity as an int. Do you think that Python > could allow returning infinity from __len__? The question is what that would achieve? Consumers of len() expect it to return a finite integer, so returning infinite would break lots of existing code. If you're willing to be practical, you can still return sys.maxsize, for example (which is not far from infinite on 64-bit systems ;-)). Regards Antoine. From ram at rachum.com Tue Feb 25 13:11:51 2014 From: ram at rachum.com (Ram Rachum) Date: Tue, 25 Feb 2014 14:11:51 +0200 Subject: [Python-ideas] Allow __len__ to return infinity In-Reply-To: References: <3517a862-205d-46f0-8a0f-b93e38451945@googlegroups.com> <20140225111819.GS3684@ando> Message-ID: This actually makes me think that `range(int('inf'))` is a more elegant construct than `itertools.count()`. Also `range(x, int('inf'))` for `itertools.count(x)`, and then you have `range(x, int('inf'), y)` or `range(0, int('-inf'), -1)` which `itertools.count` can't cover. On Tue, Feb 25, 2014 at 1:46 PM, Paul Moore wrote: > On 25 February 2014 11:18, Steven D'Aprano wrote: > > (1) __len__ can return *any* float, regardless of value, including > > lengths of 0.5, NAN, 1e300, etc. This is undesirable because lengths of > > sequences should be positive or zero whole numbers, not arbitrary > > floating point values like 4.5. > > > > (2) __len__ cannot return any float, but only INF. Which means that the > > condition that len() only returns ints will be broken in the most > > surprising way, with a single exception. > > Either of these would break range(len(x)) at a minimum, and probably > most other code that uses len(). So you'd be left with code having to > identify "objects that only return finite len" and "objects that could > have one of the new len values". And if the OP can detect that, he can > just do so in his code and special case his objects, rather than > changing the len protocol. > > Paul > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > -- > > --- > You received this message because you are subscribed to a topic in the > Google Groups "python-ideas" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/python-ideas/nFYbEpHrjlk/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > python-ideas+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From __peter__ at web.de Tue Feb 25 13:38:18 2014 From: __peter__ at web.de (Peter Otten) Date: Tue, 25 Feb 2014 13:38:18 +0100 Subject: [Python-ideas] Unbounded range(), was Re: Allow __len__ to return infinity References: <3517a862-205d-46f0-8a0f-b93e38451945@googlegroups.com> <20140225111819.GS3684@ando> Message-ID: Ram Rachum wrote: > This actually makes me think that `range(int('inf'))` is a more elegant > construct than `itertools.count()`. Also `range(x, int('inf'))` > for `itertools.count(x)`, You could achieve that with range(None) or range(start, None) which would be similar to slices like items[start:None] aka items[start:]. > and then you have `range(x, int('inf'), y)` or `range(0, int('-inf'), -1)` > which `itertools.count` can't cover. This *is* covered by count(): >>> [x for x in itertools.islice(itertools.count(step=-2), 10)] [0, -2, -4, -6, -8, -10, -12, -14, -16, -18] All you save is one import. In return you'll see your applications break in new and interesting ways ;) From ram at rachum.com Tue Feb 25 13:46:04 2014 From: ram at rachum.com (Ram Rachum) Date: Tue, 25 Feb 2014 14:46:04 +0200 Subject: [Python-ideas] Unbounded range(), was Re: Allow __len__ to return infinity In-Reply-To: References: <3517a862-205d-46f0-8a0f-b93e38451945@googlegroups.com> <20140225111819.GS3684@ando> Message-ID: I stand corrected. On Tue, Feb 25, 2014 at 2:38 PM, Peter Otten <__peter__ at web.de> wrote: > Ram Rachum wrote: > > > This actually makes me think that `range(int('inf'))` is a more elegant > > construct than `itertools.count()`. Also `range(x, int('inf'))` > > for `itertools.count(x)`, > > You could achieve that with range(None) or range(start, None) > which would be similar to slices like items[start:None] aka items[start:]. > > > and then you have `range(x, int('inf'), y)` or `range(0, int('-inf'), > -1)` > > which `itertools.count` can't cover. > > This *is* covered by count(): > > >>> [x for x in itertools.islice(itertools.count(step=-2), 10)] > [0, -2, -4, -6, -8, -10, -12, -14, -16, -18] > > All you save is one import. In return you'll see your applications break in > new and interesting ways ;) > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > -- > > --- > You received this message because you are subscribed to a topic in the > Google Groups "python-ideas" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/python-ideas/nFYbEpHrjlk/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > python-ideas+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From antony.lee at berkeley.edu Tue Feb 25 18:35:52 2014 From: antony.lee at berkeley.edu (Antony Lee) Date: Tue, 25 Feb 2014 09:35:52 -0800 Subject: [Python-ideas] Allow __len__ to return infinity In-Reply-To: References: <3517a862-205d-46f0-8a0f-b93e38451945@googlegroups.com> <20140225111819.GS3684@ando> Message-ID: This is similar to another idea I had some time ago: add keyword-argument support for range, and let range(start=x, [step=y]) (stop being equal to None) map to itertools.count(start=x, [step=y]). No need for inf the way. 2014-02-25 4:11 GMT-08:00 Ram Rachum : > This actually makes me think that `range(int('inf'))` is a more elegant > construct than `itertools.count()`. Also `range(x, int('inf'))` > for `itertools.count(x)`, and then you have `range(x, int('inf'), y)` > or `range(0, int('-inf'), -1)` which `itertools.count` can't cover. > > On Tue, Feb 25, 2014 at 1:46 PM, Paul Moore wrote: > >> On 25 February 2014 11:18, Steven D'Aprano wrote: >> > (1) __len__ can return *any* float, regardless of value, including >> > lengths of 0.5, NAN, 1e300, etc. This is undesirable because lengths of >> > sequences should be positive or zero whole numbers, not arbitrary >> > floating point values like 4.5. >> > >> > (2) __len__ cannot return any float, but only INF. Which means that the >> > condition that len() only returns ints will be broken in the most >> > surprising way, with a single exception. >> >> Either of these would break range(len(x)) at a minimum, and probably >> most other code that uses len(). So you'd be left with code having to >> identify "objects that only return finite len" and "objects that could >> have one of the new len values". And if the OP can detect that, he can >> just do so in his code and special case his objects, rather than >> changing the len protocol. >> >> Paul >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> >> -- >> >> --- >> You received this message because you are subscribed to a topic in the >> Google Groups "python-ideas" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/python-ideas/nFYbEpHrjlk/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> python-ideas+unsubscribe at googlegroups.com. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron3200 at gmail.com Tue Feb 25 20:43:58 2014 From: ron3200 at gmail.com (Ron Adam) Date: Tue, 25 Feb 2014 13:43:58 -0600 Subject: [Python-ideas] Method chaining notation In-Reply-To: <87zjlftxg3.fsf@uwakimon.sk.tsukuba.ac.jp> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> <87zjlftxg3.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On 02/25/2014 02:21 AM, Stephen J. Turnbull wrote: > Ron Adam writes: > > > You would probably see it used more often like this... > > > > def names(defaults, pos_names, pos_args, kwds): > > return {}.=update(defaults) \ > > .=update(zip(pos_names, pos_args) \ > > .=update(kwds) > > I actually have a bunch of code in one of my apps that implements the > same thing for a different reason (cascading configs), but my > implementation is > > def names(defaults, pos_names, pos_args, kwds): > for dct in pos_names, pos_args, kwds: > defaults.update(dct) > return defaults Not quite the same but close. I just tried to come up with a more realistic example without having to look up a lot code. How does pos_args in your example get paired with names? Cheers, Ron > The other obvious use for this (as several have posted) is > accumulating a sequence. In which case most uses will be well-handled > with a genexp, or if you need a concrete sequence, a listcomp, and the > body becomes a one (logical) liner (although it will very likely be > formatted in multiple lines). From ron3200 at gmail.com Tue Feb 25 21:55:58 2014 From: ron3200 at gmail.com (Ron Adam) Date: Tue, 25 Feb 2014 14:55:58 -0600 Subject: [Python-ideas] Method chaining notation In-Reply-To: <0FA56C9E-D7DE-406B-9586-73B7CC83ACEE@masklinn.net> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> <0FA56C9E-D7DE-406B-9586-73B7CC83ACEE@masklinn.net> Message-ID: On 02/24/2014 01:59 PM, Masklinn wrote: >> >By using the '.' we can see the difference. The byte code should be very close to this, even though this function will give an error if you try to run it. (Can't update None.) The actual difference would probably be replacing LOAD_ATTR with LOAD_MUTATE_ATTR, Which would call __getmutatemethod__ instead of __getmethod__. (or something similar to that, depending on how it's implemented.) > Why? There's no need for LOAD_MUTATE_ATTR. And LOAD_ATTR calls > __getattribute__ (and __getattr__ if necessary), a bound method is a > form callable attribute, the bytecode for a method call (assuming an > object on the stack) is > > LOAD_ATTR $attrname > CALL_FUNCTION > > that the function mutates the original object (or not) has no relevance > to attribute loading. Turn it around... If an object doesn't have a mutate-attribute-loader.. Then you will get an error before the CALL_FUNCION instead of during it or after it, without making any changes to existing method/function call code paths. > > def names(defaults, pos_names, pos_args, kwds): > > return {}.update(defaults) \ > > .update(zip(pos_names, pos_args)) \ > > .update(kwds) > > >>>> > >>> dis(names) > > 2 0 BUILD_MAP 0 > > 3 LOAD_ATTR 0 (update) > > 6 LOAD_FAST 0 (defaults) > > 9 CALL_FUNCTION 1 (1 positional, 0 keyword pair) > > 12 LOAD_ATTR 0 (update) > > > > 3 15 LOAD_GLOBAL 1 (zip) > > 18 LOAD_FAST 1 (pos_names) > > 21 LOAD_FAST 2 (pos_args) > > 24 CALL_FUNCTION 2 (2 positional, 0 keyword pair) > > 27 CALL_FUNCTION 1 (1 positional, 0 keyword pair) > > 30 LOAD_ATTR 0 (update) > > > > 4 33 LOAD_FAST 3 (kwds) > > 36 CALL_FUNCTION 1 (1 positional, 0 keyword pair) > > 39 RETURN_VALUE > That bytecode's not correct for the case: > * the return value of each method call needs to be discarded with a > POP_TOP > * LOAD_ATTR needs an object on the stack so you need a DUP_TOP before > each LOAD_ATTR (update) (you can create the correct bytecode with > something like byteplay, it'll work) Actually, for it to work the way I was thinking, it needs a matched pair to replace LOAD_ATTR and CALL_FUNCTION. The alternate call function bytecode would leave the funciton on the stack and give an error if the returned anything other than None. But I think that's too many new Byte codes. Even one new byte code is a hard sell. The idea is really just a more limited version of cascading with an error for non-mutatables used with that syntax, and am error if any value is returned when using with that syntax. So if you see this particular syntax you will know instantly the intent is to mutate the subject. It might be a fun patch to play with and/or try to do. But I don't think it would ever get approved. The alternative is to use the existing byte codes as you describe and that removes any (however small they might be) performance benifits. Cheers, Ron From greg.ewing at canterbury.ac.nz Wed Feb 26 00:02:01 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 26 Feb 2014 12:02:01 +1300 Subject: [Python-ideas] Infix functions In-Reply-To: <06A045B6-C169-4886-B19A-EC932F6A95BD@yahoo.com> References: <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> <8AB414A2-5156-4DAB-AC7C-47A6A283CED4@yahoo.com> <20140224013540.GM3684@ando> <8761o5uus0.fsf@uwakimon.sk.tsukuba.ac.jp> <20140224152827.GP3684@ando> <06A045B6-C169-4886-B19A-EC932F6A95BD@yahoo.com> Message-ID: <530D20E9.1070509@canterbury.ac.nz> Andrew Barnert wrote re Nick's concise lambda proposal: > data = using(:?.read(), open(path)) > buf = catch(:read_file(path), (FileNotFoundError, :'') > b = cond(a, :1/a, :NaN) This doesn't look like Python to me. It's too cryptic, especially with the '?'. If you think a colon in an except-expression looks too confusing, this would be far worse. Also, this kind of style would make it very easy to make subtle mistakes by getting the colons in the wrong places. It's not quite the same thing as a Lisp macro, because the onus is on the caller to remember which arguments need to be quoted. In Lisp, the macro takes care of that. -- Greg From rymg19 at gmail.com Wed Feb 26 00:08:44 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Tue, 25 Feb 2014 17:08:44 -0600 Subject: [Python-ideas] Caching iterators Message-ID: Note: I PROMISE this is a better idea than my last 20 bad ones(raise_if, shlex extra argument, etc.) I'll use an example to illustrate the idea first. Let's use a completely non-realistic and contrived example. Say you have a lexer that's really slow. Now, this lexer might be a function that uses generators, i.e.: def mylexer(input): while input: ... if xyz: yield SomeToken() Now, if we have a parser that uses that lexer, it always has to wait for the lexer to yield the next token *after* it already parsed the current token. That can be somewhat time consuming. Caching iterators are based on the idea: what if the iterator is running at the same time as the function getting the iterator elements? Or, better yet, it's an iterator wrapper that takes an iterator and continues to take its elements while the function that uses the iterator is running? This is easily accomplished using multiprocessing and pipes. Since that was somewhat vague, here's an example: def my_iterator(): for i in range(0,5): time.sleep(0.2) yield i for item in my_iterator(): time.sleep(0.5) print(item) Now with a normal iterator, the flow is like this: - Wait for my_iterator to return an element(0.2s) - Wait 0.5s and print the element(0.5s) In total, that takes 0.7s per element. What a waste! What if the iterator was yielding elements at the same time as the for loop was using them? Well, for every for loop iteration, the iterator could generate ~2.2 elements. That's what a caching iterator does. It runs both at the same time using multiprocessing. It's thread safe as long as the iterator doesn't depend on whatever is using it. An example: def my_iterator(): for i in range(0,5): time.sleep(0.2) yield i for item in itertools.CachingIterator(my_iterator()): # this is the only change time.sleep(0.5) print(item) Now the flow is like this: - Wait for my_iterator to return the very first element. - While that first element is looped over, continue recieving elements from my_iterator(), storing them in an intermediate space(similar to a deque). - When the loop is completed, take the next element from the intermediate space and loop over it - While that element is looped over, continue recieving elements... ...and so forth. That way, time isn't wasted waited for the loop to finish. I have a working implementation. Although there is a very slight overhead, in the above example, about 0.4s is still saved. There could also be an lmap function, which just does this: def lmap(f,it): yield from map(f,CachingIterator(it)) Thoughts? -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From moloney at ohsu.edu Wed Feb 26 00:19:06 2014 From: moloney at ohsu.edu (Brendan Moloney) Date: Tue, 25 Feb 2014 23:19:06 +0000 Subject: [Python-ideas] Caching iterators In-Reply-To: References: Message-ID: <5F6A858FD00E5F4A82E3206D2D854EF8077A4A07@EXMB12.ohsu.edu> I believe the new asyncio package in 3.4 solves this problem (and more!). Brendan ________________________________ From: Python-ideas [python-ideas-bounces+moloney=ohsu.edu at python.org] on behalf of Ryan Gonzalez [rymg19 at gmail.com] Sent: Tuesday, February 25, 2014 3:08 PM To: python-ideas Subject: [Python-ideas] Caching iterators Note: I PROMISE this is a better idea than my last 20 bad ones(raise_if, shlex extra argument, etc.) I?ll use an example to illustrate the idea first. Let?s use a completely non-realistic and contrived example. Say you have a lexer that?s really slow. Now, this lexer might be a function that uses generators, i.e.: def mylexer(input): while input: ... if xyz: yield SomeToken() Now, if we have a parser that uses that lexer, it always has to wait for the lexer to yield the next token after it already parsed the current token. That can be somewhat time consuming. Caching iterators are based on the idea: what if the iterator is running at the same time as the function getting the iterator elements? Or, better yet, it?s an iterator wrapper that takes an iterator and continues to take its elements while the function that uses the iterator is running? This is easily accomplished using multiprocessing and pipes. Since that was somewhat vague, here?s an example: def my_iterator(): for i in range(0,5): time.sleep(0.2) yield i for item in my_iterator(): time.sleep(0.5) print(item) Now with a normal iterator, the flow is like this: * Wait for my_iterator to return an element(0.2s) * Wait 0.5s and print the element(0.5s) In total, that takes 0.7s per element. What a waste! What if the iterator was yielding elements at the same time as the for loop was using them? Well, for every for loop iteration, the iterator could generate ~2.2 elements. That?s what a caching iterator does. It runs both at the same time using multiprocessing. It?s thread safe as long as the iterator doesn?t depend on whatever is using it. An example: def my_iterator(): for i in range(0,5): time.sleep(0.2) yield i for item in itertools.CachingIterator(my_iterator()): # this is the only change time.sleep(0.5) print(item) Now the flow is like this: * Wait for my_iterator to return the very first element. * While that first element is looped over, continue recieving elements from my_iterator(), storing them in an intermediate space(similar to a deque). * When the loop is completed, take the next element from the intermediate space and loop over it * While that element is looped over, continue recieving elements? ?and so forth. That way, time isn?t wasted waited for the loop to finish. I have a working implementation. Although there is a very slight overhead, in the above example, about 0.4s is still saved. There could also be an lmap function, which just does this: def lmap(f,it): yield from map(f,CachingIterator(it)) Thoughts? -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Wed Feb 26 00:40:59 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 26 Feb 2014 12:40:59 +1300 Subject: [Python-ideas] Allow __len__ to return infinity In-Reply-To: <20140225111819.GS3684@ando> References: <3517a862-205d-46f0-8a0f-b93e38451945@googlegroups.com> <20140225111819.GS3684@ando> Message-ID: <530D2A0B.30306@canterbury.ac.nz> Steven D'Aprano wrote: > (1) __len__ can return *any* float, regardless of value, including > lengths of 0.5, NAN, 1e300, etc. This is undesirable because lengths of > sequences should be positive or zero whole numbers, What about things other than sequences? For a vector type, for example, it would make sense for len(v) to return the magnitude of the vector. -- Greg From steve at pearwood.info Wed Feb 26 01:01:09 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 26 Feb 2014 11:01:09 +1100 Subject: [Python-ideas] Caching iterators In-Reply-To: References: Message-ID: <20140226000109.GV3684@ando> On Tue, Feb 25, 2014 at 05:08:44PM -0600, Ryan Gonzalez wrote: > Note: > > I PROMISE this is a better idea than my last 20 bad ones(raise_if, shlex > extra argument, etc.) Don't make promises you can't keep. (Sorry, that was a cheap shot, but you practically begged for somebody to make it :-) > I'll use an example to illustrate the idea first. Let's use a completely > non-realistic and contrived example. Say you have a lexer that's really > slow. Now, this lexer might be a function that uses generators, i.e.: [...] > Now, if we have a parser that uses that lexer, it always has to wait for > the lexer to yield the next token *after* it already parsed the current > token. That can be somewhat time consuming. This is the nature of serial processing. Normally responsibility for moving to parallel processing (whether threads or processes or something else) is handled by the user, not the standard library. Very little in the std lib is calculated in parallel. > Caching iterators are based on the idea: what if the iterator is running at > the same time as the function getting the iterator elements? Or, better > yet, it's an iterator wrapper that takes an iterator and continues to take > its elements while the function that uses the iterator is running? This is > easily accomplished using multiprocessing and pipes. > > Since that was somewhat vague, here's an example: > > def my_iterator(): > for i in range(0,5): > time.sleep(0.2) > yield i > for item in my_iterator(): > time.sleep(0.5) > print(item) So you have a delay in calculating each value in the iterator, and a longer delay using each value. If you perform those sequentially, it takes 0.7 seconds per item; if you could perform them perfectly in parallel, only 0.5 seconds. > Now with a normal iterator, the flow is like this: > > - Wait for my_iterator to return an element(0.2s) > - Wait 0.5s and print the element(0.5s) > > In total, that takes 0.7s per element. What a waste! What if the iterator > was yielding elements at the same time as the for loop was using them? > Well, for every for loop iteration, the iterator could generate ~2.2 > elements. That's what a caching iterator does. It runs both at the same > time using multiprocessing. You could use threads if the processes are IO bound rather than CPU bound. Since threads are less expensive than processing, surely that choice should have to be up to the user? In either case, one disadvantage of a cache is that values are no longer being calculated lazily (i.e. on request). Because the iterator runs faster than the consumer, rather than working in lock-step you end up generating more values than you need: each iteration sees the iterator generate 2.2 elements and the consumer use 1. By the time the consumer has used ten thousand values, the iterator has generated twenty-two thousand values, and twelve thousand remain in the cache. While tempting, the caller needs to be aware that such a caching system: (1) uses potentially unbounded amounts of memory; (2) is potentially harmful if calculating the values has side-effects; (3) it can lead to "lost" data if the caller access the underlying iterator without going through the cache; and (4) it is wasteful if the consumer stops early and never uses all the values. (CPU cycles are cheap, but they aren't free.) None of these invalidate the basic idea, but they do limit the applicability of it. [...] > Now the flow is like this: > > - Wait for my_iterator to return the very first element. > - While that first element is looped over, continue recieving elements > from my_iterator(), storing them in an intermediate space(similar to a > deque). The data structure you want is a Queue. The std lib has a thread-safe Queue implementation, in the queue module. > I have a working implementation. Although there is a very slight overhead, > in the above example, about 0.4s is still saved. I think you should publish this implementation as a recipe on the ActiveState website, and see what feedback you get there. Once it is proven to be useful in practice, rather than just theoretically useful, then it could be considered for the std lib. http://code.activestate.com/recipes/ -- Steven From greg.ewing at canterbury.ac.nz Wed Feb 26 01:17:01 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 26 Feb 2014 13:17:01 +1300 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: References: Message-ID: <530D327D.6090804@canterbury.ac.nz> I was reading a blog post here: http://stupidpythonideas.blogspot.co.nz/2014/02/fixing-lambda.html where the author points out that there are a number of different problems that the various enhanced-lambda proposals are trying to solve, which are best addressed by different solutions. Here's a suggestion for one of them. Suppose we could write things like: sorted(things, key(x) = x.date) Button("Do it!", on_click() = fire_the_ducks()) It only addresses the case of passing a function using a keyword argument, but I think it would make for very readable code in those cases. And it doesn't use any colons! -- Greg From steve at pearwood.info Wed Feb 26 01:18:14 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 26 Feb 2014 11:18:14 +1100 Subject: [Python-ideas] Allow __len__ to return infinity In-Reply-To: <530D2A0B.30306@canterbury.ac.nz> References: <3517a862-205d-46f0-8a0f-b93e38451945@googlegroups.com> <20140225111819.GS3684@ando> <530D2A0B.30306@canterbury.ac.nz> Message-ID: <20140226001814.GW3684@ando> On Wed, Feb 26, 2014 at 12:40:59PM +1300, Greg Ewing wrote: > Steven D'Aprano wrote: > >(1) __len__ can return *any* float, regardless of value, including > >lengths of 0.5, NAN, 1e300, etc. This is undesirable because lengths of > >sequences should be positive or zero whole numbers, > > What about things other than sequences? > > For a vector type, for example, it would make sense > for len(v) to return the magnitude of the vector. I don't think it does. Despite the similarities in names, I don't think Python the language should conflate *length of a sequence or mapping* with *arbitrary measures of length*. len(x) by historical precedent returns the length of a sequence/mapping/set, where the result returned is a non-negative integer value. If you want some other definition of length, such as the length of a vowel or the length of a polynomial, or some other metric such as Manhattan distance or Chebyshev distance, or even vector magnitude, you can create your own function or method and call it (say) length(). -- Steven From rosuav at gmail.com Wed Feb 26 01:25:41 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 26 Feb 2014 11:25:41 +1100 Subject: [Python-ideas] Caching iterators In-Reply-To: <20140226000109.GV3684@ando> References: <20140226000109.GV3684@ando> Message-ID: On Wed, Feb 26, 2014 at 11:01 AM, Steven D'Aprano wrote: > While tempting, the caller needs to be aware that such a caching system: > > (1) uses potentially unbounded amounts of memory; Easy fix: Limit the size of the queue. Just like with pipes between processes, the producer will block trying to push more data into the queue until the consumer's taken some out. Of course, then you have to figure out what's the right queue size. In many cases the safest and simplest might well be zero, aka current behaviour. > (2) is potentially harmful if calculating the values has side-effects; > > (3) it can lead to "lost" data if the caller access the underlying > iterator without going through the cache; and Deal with these two by making it something you have to explicitly request. In that way, it's no different from itertools.tee() - once you tee an iterator, you do not touch the underlying one at all. > (4) it is wasteful if the consumer stops early and never uses all the > values. (CPU cycles are cheap, but they aren't free.) Also partly solved by the queue size limit (don't let it run free forever). That said, though, I don't actually know of any place where I would want this facility where I wouldn't already be working with, say, a socket connection, or a queue, or something else that buffers. ChrisA From rosuav at gmail.com Wed Feb 26 01:34:10 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 26 Feb 2014 11:34:10 +1100 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: <530D327D.6090804@canterbury.ac.nz> References: <530D327D.6090804@canterbury.ac.nz> Message-ID: On Wed, Feb 26, 2014 at 11:17 AM, Greg Ewing wrote: > Here's a suggestion for one of them. Suppose we could > write things like: > > sorted(things, key(x) = x.date) > > Button("Do it!", on_click() = fire_the_ducks()) > > It only addresses the case of passing a function using > a keyword argument, but I think it would make for very > readable code in those cases. And it doesn't use any > colons! Gut feeling: Do not like something that puts magic into one specific place. Apart from the * and ** unpacking tools, there's nothing special about a function's arguments that lets you use special syntax for creating a function, or anything else. One of Python's strengths is that a given expression results in a given value, and that's true regardless of where that value's going. Having these two do very different things is confusing: Button("Do it!", on_click() = fire_the_ducks()) Button("Do it!", fire_the_ducks()) One of them is a keyword argument (spaced in violation of PEP8, incidentally; not sure if that adds to the confusion by making it look like assignment when it isn't), and the other is a positional argument. Everywhere else in Python, that would be the whole difference. Now, apparently, one of them calls a function now and passes its return value, and the other creates a lambda that'll call the function later. Oh, and also: Why create a lambda that just calls one named function? Are you expecting the name to be rebound? Otherwise, just pass fire_the_ducks as the on-click callback. ChrisA From ethan at stoneleaf.us Wed Feb 26 00:47:24 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 25 Feb 2014 15:47:24 -0800 Subject: [Python-ideas] Allow __len__ to return infinity In-Reply-To: <530D2A0B.30306@canterbury.ac.nz> References: <3517a862-205d-46f0-8a0f-b93e38451945@googlegroups.com> <20140225111819.GS3684@ando> <530D2A0B.30306@canterbury.ac.nz> Message-ID: <530D2B8C.1010607@stoneleaf.us> On 02/25/2014 03:40 PM, Greg Ewing wrote: > Steven D'Aprano wrote: >> (1) __len__ can return *any* float, regardless of value, including lengths of 0.5, NAN, 1e300, etc. This is >> undesirable because lengths of sequences should be positive or zero whole numbers, > > What about things other than sequences? > > For a vector type, for example, it would make sense > for len(v) to return the magnitude of the vector. Very good point. To which I reply: singledispatch ! ;) http://docs.python.org/dev/whatsnew/3.4.html#whatsnew-singledispatch -- ~Ethan~ From tjreedy at udel.edu Wed Feb 26 04:05:23 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 25 Feb 2014 22:05:23 -0500 Subject: [Python-ideas] Caching iterators In-Reply-To: References: <20140226000109.GV3684@ando> Message-ID: On 2/25/2014 7:25 PM, Chris Angelico wrote: > On Wed, Feb 26, 2014 at 11:01 AM, Steven D'Aprano wrote: >> While tempting, the caller needs to be aware that such a caching system: >> >> (1) uses potentially unbounded amounts of memory; > > Easy fix: Limit the size of the queue. multiprocessing.Queue is a process shared near clone of queue.Queue and has one optional arg -- maxsize. It is implemented with a pipe, locks, and semaphores. > Just like with pipes between > processes, the producer will block trying to push more data into the > queue until the consumer's taken some out. This is the default behavior of Queue.put. > Of course, then you have to > figure out what's the right queue size. 42 ;-) > In many cases the safest and > simplest might well be zero, aka current behaviour. > >> (2) is potentially harmful if calculating the values has side-effects; >> >> (3) it can lead to "lost" data if the caller access the underlying >> iterator without going through the cache; and If the iterator is in another process only connected by a pipe, it cannot be accessed otherwise than through the Queue. > Deal with these two by making it something you have to explicitly > request. In that way, it's no different from itertools.tee() - once > you tee an iterator, you do not touch the underlying one at all. > >> (4) it is wasteful if the consumer stops early and never uses all the >> values. (CPU cycles are cheap, but they aren't free.) > > Also partly solved by the queue size limit (don't let it run free forever). -- Terry Jan Reedy From stephen at xemacs.org Wed Feb 26 08:19:21 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 26 Feb 2014 16:19:21 +0900 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> <87zjlftxg3.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87ob1utk7a.fsf@uwakimon.sk.tsukuba.ac.jp> Ron Adam writes: > > > def names(defaults, pos_names, pos_args, kwds): > > > return {}.=update(defaults) \ > > > .=update(zip(pos_names, pos_args) \ > > > .=update(kwds) > > def names(defaults, pos_names, pos_args, kwds): > > for dct in pos_names, pos_args, kwds: > > defaults.update(dct) > > return defaults > > Not quite the same but close. I just tried to come up with a more > realistic example without having to look up a lot code. How does > pos_args in your example get paired with names? Sorry, I knew that before dinner but forgot after dinner. Same way as in yours: def names(defaults, pos_names, pos_args, kwds): for dct in zip(pos_names, pos_args), kwds: defaults.update(dct) return defaults If that doesn't work in my version (I've never used zip that way), how does it work in yours? BTW, I'd actually be more likely to write that now as def names(defaults, *updates): for update in updates: defaults.update(update) return defaults and call it with "names(zip(pos_names, pos_args), kwds)". From techtonik at gmail.com Wed Feb 26 06:24:10 2014 From: techtonik at gmail.com (anatoly techtonik) Date: Wed, 26 Feb 2014 08:24:10 +0300 Subject: [Python-ideas] time.Timer Message-ID: UX improvement fix for the common case: import time class Timer(object): def __init__(self, seconds): self.seconds = seconds self.restart() def restart(self): self.end = time.time() + self.seconds @property def expired(self): return (time.time() > self.end) Example: class FPS(object): def __init__(self): self.counter = 0 self.timer = Timer(1) def process(self): self.counter += 1 if self.timer.expired: print "FPS: %s" % self.counter self.counter = 0 self.timer.restart() All code above is in public domain. -- anatoly t. From rosuav at gmail.com Wed Feb 26 09:29:51 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 26 Feb 2014 19:29:51 +1100 Subject: [Python-ideas] time.Timer In-Reply-To: References: Message-ID: On Wed, Feb 26, 2014 at 4:24 PM, anatoly techtonik wrote: > UX improvement fix for the common case: > > import time > > class Timer(object): > def __init__(self, seconds): > self.seconds = seconds > self.restart() > > def restart(self): > self.end = time.time() + self.seconds > > @property > def expired(self): > return (time.time() > self.end) You just showed how easy it is to implement it manually :) Your example, though, is a really bad idea: > class FPS(object): > def __init__(self): > self.counter = 0 > self.timer = Timer(1) > def process(self): > self.counter += 1 > if self.timer.expired: > print "FPS: %s" % self.counter > self.counter = 0 > self.timer.restart() Implication is that you call process() many times a second, right? Well, querying the current system time is often quite expensive, so this could seriously damage your FPS; and more importantly, it's inaccurate. Let's suppose you're actually achieving 2.5 FPS - that is to say, you call process() every 400ms. Here's how it'll go: Time 400: not expired Time 800: not expired Time 1200: expired, reset time to zero Time 1600: timer says 400, not expired Time 2000: timer says 800, not expired Time 2400: timer says 1200, expired, reset time to zero You're going to say 3 FPS, when it's really 2.5. For measuring FPS, you should probably use an algorithm more like this: 1) Spawn an FPS counter thread, or async alarm, or something, to be executed every second. 2) Each frame, increment a counter. Never reset the counter. 3) Every time the alarm goes off, record the current time and the current frame counter. 4) Your FPS is (frames - last_frames) / (time - last_time) That way, you query the current time roughly once a second. It's safe against sleep(1) taking more than one second to return (because it actually queries the time), it's safe against the FPS being extremely low, and it's fairly efficient. Plus, it doesn't require any code in the time module :) ChrisA From breamoreboy at yahoo.co.uk Wed Feb 26 10:44:05 2014 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 26 Feb 2014 09:44:05 +0000 Subject: [Python-ideas] time.Timer In-Reply-To: References: Message-ID: On 26/02/2014 05:24, anatoly techtonik wrote: > UX improvement fix for the common case: > An unexploded improvement fix, now there's a novelty :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com From ron3200 at gmail.com Wed Feb 26 16:28:30 2014 From: ron3200 at gmail.com (Ron Adam) Date: Wed, 26 Feb 2014 09:28:30 -0600 Subject: [Python-ideas] Method chaining notation In-Reply-To: <87ob1utk7a.fsf@uwakimon.sk.tsukuba.ac.jp> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> <87zjlftxg3.fsf@uwakimon.sk.tsukuba.ac.jp> <87ob1utk7a.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On 02/26/2014 01:19 AM, Stephen J. Turnbull wrote: > Ron Adam writes: > > > > > def names(defaults, pos_names, pos_args, kwds): > > > > return {}.=update(defaults) \ > > > > .=update(zip(pos_names, pos_args) \ > > > > .=update(kwds) > > > > def names(defaults, pos_names, pos_args, kwds): > > > for dct in pos_names, pos_args, kwds: > > > defaults.update(dct) > > > return defaults > > > > Not quite the same but close. I just tried to come up with a more > > realistic example without having to look up a lot code. How does > > pos_args in your example get paired with names? > > Sorry, I knew that before dinner but forgot after dinner. Same way as > in yours: > > def names(defaults, pos_names, pos_args, kwds): > for dct in zip(pos_names, pos_args), kwds: > defaults.update(dct) > return defaults Yes, ok. > If that doesn't work in my version (I've never used zip that way), how > does it work in yours? BTW, I'd actually be more likely to write that > now as > > def names(defaults, *updates): > for update in updates: > defaults.update(update) > return defaults > > and call it with "names(zip(pos_names, pos_args), kwds)". The main difference between this and the one I posted is in this, defaults is mutated in your version. I'd prefer it not be. Dictionaries are pretty flexible on how they are initiated, so it's surprising we can't do this... D = dict(keys=names, values=args) The .fromkeys() method is almost that, but sets all the values to a single value. I think I would have written that a bit different. def fromkeys(self, keys, values=None, default=None): D = {} if D is not None: D.update(zip(keys, values)] for k in keys[len(vaues):]: D[k] = default return D And probably named it withkeys instead of fromkeys. It's what I expected fromkeys to do. cheers, Ron From masklinn at masklinn.net Wed Feb 26 17:00:24 2014 From: masklinn at masklinn.net (Masklinn) Date: Wed, 26 Feb 2014 17:00:24 +0100 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> <87zjlftxg3.fsf@uwakimon.sk.tsukuba.ac.jp> <87ob1utk7a.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <2DF8DDDA-526B-4F5F-A69F-ECF406EE0CFD@masklinn.net> On 2014-02-26, at 16:28 , Ron Adam wrote: > > > On 02/26/2014 01:19 AM, Stephen J. Turnbull wrote: >> Ron Adam writes: >> >> > > > def names(defaults, pos_names, pos_args, kwds): >> > > > return {}.=update(defaults) \ >> > > > .=update(zip(pos_names, pos_args) \ >> > > > .=update(kwds) >> >> > > def names(defaults, pos_names, pos_args, kwds): >> > > for dct in pos_names, pos_args, kwds: >> > > defaults.update(dct) >> > > return defaults >> > >> > Not quite the same but close. I just tried to come up with a more >> > realistic example without having to look up a lot code. How does >> > pos_args in your example get paired with names? >> >> Sorry, I knew that before dinner but forgot after dinner. Same way as >> in yours: >> >> def names(defaults, pos_names, pos_args, kwds): >> for dct in zip(pos_names, pos_args), kwds: >> defaults.update(dct) >> return defaults > > Yes, ok. > >> If that doesn't work in my version (I've never used zip that way), how >> does it work in yours? BTW, I'd actually be more likely to write that >> now as >> >> def names(defaults, *updates): >> for update in updates: >> defaults.update(update) >> return defaults >> >> and call it with "names(zip(pos_names, pos_args), kwds)". > > The main difference between this and the one I posted is in this, defaults is mutated in your version. I'd prefer it not be. > > Dictionaries are pretty flexible on how they are initiated, so it's surprising we can't do this... > > D = dict(keys=names, values=args) You can. It may not do what you want, but you definitely can do this: >>> dict(keys=names, values=args) {'keys': ['a', 'b', 'c', 'd'], 'values': [0, 1, 2]} Although you're probably looking for: >>> dict(zip(names, args)) {'a': 0, 'c': 2, 'b': 1} and if you want to do a fill because you don't have enough args: >>> dict(izip_longest(names, args, fillvalue=None)) {'a': 0, 'c': 2, 'b': 1, 'd': None} (itertools is like friendship, it's bloody magic) > The .fromkeys() method is almost that, but sets all the values to a single value. I think I would have written that a bit different. > > def fromkeys(self, keys, values=None, default=None): > D = {} > if D is not None: > D.update(zip(keys, values)] > for k in keys[len(vaues):]: > D[k] = default > return D > > And probably named it withkeys instead of fromkeys. It's what I expected fromkeys to do. > > cheers, > Ron > From techtonik at gmail.com Wed Feb 26 18:37:12 2014 From: techtonik at gmail.com (anatoly techtonik) Date: Wed, 26 Feb 2014 20:37:12 +0300 Subject: [Python-ideas] time.Timer In-Reply-To: References: Message-ID: On Wed, Feb 26, 2014 at 11:29 AM, Chris Angelico wrote: > On Wed, Feb 26, 2014 at 4:24 PM, anatoly techtonik wrote: >> UX improvement fix for the common case: >> >> import time >> >> class Timer(object): >> def __init__(self, seconds): >> self.seconds = seconds >> self.restart() >> >> def restart(self): >> self.end = time.time() + self.seconds >> >> @property >> def expired(self): >> return (time.time() > self.end) > > You just showed how easy it is to implement it manually :) Yes, but the frequency of this element made it a better position than other datetime functions. > Your example, though, is a really bad idea: > >> class FPS(object): >> def __init__(self): >> self.counter = 0 >> self.timer = Timer(1) >> def process(self): >> self.counter += 1 >> if self.timer.expired: >> print "FPS: %s" % self.counter >> self.counter = 0 >> self.timer.restart() > > Implication is that you call process() many times a second, right? > Well, querying the current system time is often quite expensive, so > this could seriously damage your FPS; and more importantly, it's > inaccurate. Let's suppose you're actually achieving 2.5 FPS - that is > to say, you call process() every 400ms. Here's how it'll go: > > Time 400: not expired > Time 800: not expired > Time 1200: expired, reset time to zero > Time 1600: timer says 400, not expired > Time 2000: timer says 800, not expired > Time 2400: timer says 1200, expired, reset time to zero > > You're going to say 3 FPS, when it's really 2.5. +/- 1 FPS is acceptable error as long as FPS doesn't fall below 60. For my script that in PySDL2 is about 1000 for filling a small window with lines. > For measuring FPS, > you should probably use an algorithm more like this: > > 1) Spawn an FPS counter thread, or async alarm, or something, to be > executed every second. I don't want to mess with woes of concurrent programming for this kind of accuracy. Python is good at what it is good for. https://stackoverflow.com/questions/134867/which-programming-language-makes-concurrent-programming-as-easy-as-possible > 2) Each frame, increment a counter. Never reset the counter. Although offtopic, but I don't get why resetting a counter is worse than storing two values and subtracting them. > 3) Every time the alarm goes off, record the current time and the > current frame counter. > 4) Your FPS is (frames - last_frames) / (time - last_time) > > That way, you query the current time roughly once a second. It's safe > against sleep(1) taking more than one second to return (because it > actually queries the time), it's safe against the FPS being extremely > low, and it's fairly efficient. It increases complexity, but doesn't increase accuracy, and therefore is fairly inefficient for maintenance. In formula: (frames - last_frames) / (time - last_time) (time - last_time) == 1 so where I get 3 frames, your code will get 2. The code only makes sense when timer fire is delayed (I don't know if it is the case with threads) or when you make frame-based FPS measurements (every 10 frames or so). > Plus, it doesn't require any code in the time module :) FPS is only one problem that timer solves. https://www.google.by/search?q=timer - even Google calculator has one. From techtonik at gmail.com Wed Feb 26 18:38:40 2014 From: techtonik at gmail.com (anatoly techtonik) Date: Wed, 26 Feb 2014 20:38:40 +0300 Subject: [Python-ideas] time.Timer In-Reply-To: References: Message-ID: On Wed, Feb 26, 2014 at 12:44 PM, Mark Lawrence wrote: > On 26/02/2014 05:24, anatoly techtonik wrote: >> >> UX improvement fix for the common case: > > An unexploded improvement fix, now there's a novelty :) Improvement fixes all around. -=) From rosuav at gmail.com Wed Feb 26 19:01:20 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 27 Feb 2014 05:01:20 +1100 Subject: [Python-ideas] time.Timer In-Reply-To: References: Message-ID: Yes, this is OT, but quite a bit of it is on the general subject of threading and counters and timing, all of which are important to Python. But if you want to continue this discussion further, we should probably move to python-list or something. On Thu, Feb 27, 2014 at 4:37 AM, anatoly techtonik wrote: > On Wed, Feb 26, 2014 at 11:29 AM, Chris Angelico wrote: >> For measuring FPS, >> you should probably use an algorithm more like this: >> >> 1) Spawn an FPS counter thread, or async alarm, or something, to be >> executed every second. > > I don't want to mess with woes of concurrent programming > for this kind of accuracy. Python is good at what it is good for. > https://stackoverflow.com/questions/134867/which-programming-language-makes-concurrent-programming-as-easy-as-possible Threads are pretty easy. Remember, you're working with an idle thread - it wakes up once a second to do stuff, then goes back to sleep. That's really easy to handle: a simple while loop in a separate function. >> 2) Each frame, increment a counter. Never reset the counter. > > Although offtopic, but I don't get why resetting a counter is worse > than storing two values and subtracting them. Technically, it's possible for this operation to be broken in half: frame_count += 1 If the other thread comes in between the read and the write, your next second will have twice the apparent FPS. By restricting one thread to read-only access, you guarantee atomicity. It's an obscure situation, but just get into the habit of having everything written to by one thread only, and threading is really easy. >> 3) Every time the alarm goes off, record the current time and the >> current frame counter. >> 4) Your FPS is (frames - last_frames) / (time - last_time) >> >> That way, you query the current time roughly once a second. It's safe >> against sleep(1) taking more than one second to return (because it >> actually queries the time), it's safe against the FPS being extremely >> low, and it's fairly efficient. > > It increases complexity, but doesn't increase accuracy, and therefore > is fairly inefficient for maintenance. In formula: > > (frames - last_frames) / (time - last_time) > > (time - last_time) == 1 Wrong. In the first place, time.time() returns a float, so that would be 1.0 (which means you'll get a float result, even in Py2); and in the second place, sleeping for one second is guaranteed to pause for at least one second, maybe more, and there's other code around it too. So usually, you'll have a figure higher than 1.0. This is even more pronounced if your loop waits for some other action, like the painting of the next frame. > so where I get 3 frames, your code will get 2. The code only makes > sense when timer fire is delayed (I don't know if it is the case with > threads) or when you make frame-based FPS measurements (every > 10 frames or so). Timer fire should always be expected to be delayed, unless you're running on some sort of real-time system. (And even then, I don't know how you get around the laws of physics. Maybe it could throw an exception if the sleep lasted too long.) It may not seem significant when you're expecting 60 FPS, but two points. 1) FPS displays are really important when something goes wrong and you're achieving 2.5 FPS. And I have seen times when this exact thing has happened - in fact, I've watched a game struggle through <1.0 FPS, and its display actually couldn't handle it, which means I have no idea how poorly it was really performing. 2) Running at exactly 60 FPS when your screen repaints at exactly 60 FPS is good. Running at 59 FPS when your screen repaints at 60 FPS is bad. Look up Vsync and why so many games let you turn it on or off. Misdisplaying the FPS by one could have someone toggle Vsync inappropriately, or complain that it's not working. Showing 59.99 FPS doesn't look too bad (and you could have your display round that carefully). ChrisA From abarnert at yahoo.com Wed Feb 26 19:19:02 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 26 Feb 2014 10:19:02 -0800 Subject: [Python-ideas] Infix functions In-Reply-To: <530D20E9.1070509@canterbury.ac.nz> References: <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> <8AB414A2-5156-4DAB-AC7C-47A6A283CED4@yahoo.com> <20140224013540.GM3684@ando> <8761o5uus0.fsf@uwakimon.sk.tsukuba.ac.jp> <20140224152827.GP3684@ando> <06A045B6-C169-4886-B19A-EC932F6A95BD@yahoo.com> <530D20E9.1070509@canterbury.ac.nz> Message-ID: <4CC21E88-EFA9-465D-AD1A-F449E81F877D@yahoo.com> On Feb 25, 2014, at 15:02, Greg Ewing wrote: > Andrew Barnert wrote re Nick's concise lambda proposal: > >> data = using(:?.read(), open(path)) >> buf = catch(:read_file(path), (FileNotFoundError, :'') >> b = cond(a, :1/a, :NaN) > > This doesn't look like Python to me. It's too cryptic, > especially with the '?'. Yeah, that was my initial reaction to Nick's suggestion, but it's growing on me. Anyway, there's a separate thread on that, which is stalled while I gather use cases from real code. (The stdlib uses lambdas much less than user-level code does, but the docs are full of them...) > If you think a colon in > an except-expression looks too confusing, this would > be far worse. Personally, I have no problem with the colon in the except expression. > Also, this kind of style would make it very easy to > make subtle mistakes by getting the colons in the > wrong places. I'm not sure about that. I think in most cases, anywhere you put the colons wrong will be a SyntaxError. And most of the exceptions will be places where you should be using a full-syntax lambda or an out-of-line function anyway (e.g., in a dict mapping names to functions). But until I gather more examples I don't think I can argue that effectively. > It's not quite the same thing as a Lisp macro, because > the onus is on the caller to remember which arguments > need to be quoted. In Lisp, the macro takes care of > that. Right, it's more like a regular function that takes sexprs, where it's up to the caller to quote them. But it's not even really a parallel with that, because this isn't quoting, it really is explicitly making a function. From tjreedy at udel.edu Wed Feb 26 19:21:54 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 26 Feb 2014 13:21:54 -0500 Subject: [Python-ideas] time.Timer In-Reply-To: References: Message-ID: On 2/26/2014 12:37 PM, anatoly techtonik wrote: > On Wed, Feb 26, 2014 at 11:29 AM, Chris Angelico wrote: >> For measuring FPS, >> you should probably use an algorithm more like this: >> >> 1) Spawn an FPS counter thread, or async alarm, or something, to be >> executed every second. > I don't want to mess with woes of concurrent programming > for this kind of accuracy. Python is good at what it is good for. What 'Python is good at' changes when new features and modules are added. The new asynch module is about 'non-blocking' programming. Its sleep task makes having a process periodically wake up, do something, and go back to sleep simple. We may assume that async.sleep(n) uses whatever non-blocking watchdog timer system call is available in the major OSes. There is not much reason left to program one in Python. > https://stackoverflow.com/questions/134867/which-programming-language-makes-concurrent-programming-as-easy-as-possible The first answer is to use a functional language because values are never changed. For this use, the FPS reporter should access an 'anti-functional' changing counter. Of course, the 'counter' is not actually changed (as it would be if it were a named memory slot in C). Rather than name 'counter' is rebound to another int. In any case, there worry is that is two threads or threadlets rebind the name, one or the other will get discombobulated. That is not the case here. >> 2) Each frame, increment a counter. Never reset the counter. > > Although offtopic, but I don't get why resetting a counter is worse > than storing two values and subtracting them. Since the frame function that increments the counter does not care if it is reset to 0, I might have the FPS reported reset it. Either way is a minor detail. -- Terry Jan Reedy From rosuav at gmail.com Wed Feb 26 19:32:12 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 27 Feb 2014 05:32:12 +1100 Subject: [Python-ideas] time.Timer In-Reply-To: References: Message-ID: On Thu, Feb 27, 2014 at 5:21 AM, Terry Reedy wrote: > Since the frame function that increments the counter does not care if it is > reset to 0, I might have the FPS reported reset it. Either way is a minor > detail. Only if you have an atomic increment operation, which I don't believe Python exposes. But yes, that's a minor detail. I actually hadn't thought through all that when I first said it - it was just a habit of "don't write to anything from two threads unless you really have to". ChrisA From abarnert at yahoo.com Wed Feb 26 20:44:51 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 26 Feb 2014 11:44:51 -0800 (PST) Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: <530D327D.6090804@canterbury.ac.nz> References: <530D327D.6090804@canterbury.ac.nz> Message-ID: <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> From: Greg Ewing Sent: Tuesday, February 25, 2014 4:17 PM > I was reading a blog post here: > > http://stupidpythonideas.blogspot.co.nz/2014/02/fixing-lambda.html > > where the author points out that there are a number > of different problems that the various enhanced-lambda > proposals are trying to solve, which are best addressed > by different solutions. > > Here's a suggestion for one of them. Suppose we could > write things like: > > ? sorted(things, key(x) = x.date) > > ? Button("Do it!", on_click() = fire_the_ducks()) Of course that particular case would be better written as: ? ? Button("Do it!", on_click = fire_the_ducks) But it becomes more useful if you do anything else: ? ? Button("Do it!", on_click() = fire_the_ducks(42)) At first glance, I think this is nice, but there's a nagging feeling that it may be a bit magical. Maybe if I think through what the compiler will do with it, I can resolve that feeling. (Obviously real users aren't going to care how it gets parsed and compiled, but if it's simple and clear enough, that implies that it can also be simple and clear to a human reader. Not perfectly, but? anyway, let's try it.) I'll do that at the end. > It only addresses the case of passing a function using > a keyword argument It also doesn't look quite as nice when the function comes first, but I think it's still pretty nice: ? ? itertools.takewhile(predicate(x)=x<5, iterable=spam) And functions that take a function and then *args, like map, would be tricky, but then it's fine if this doesn't work nicely in every possible case for passing a function around. Also, this _could_ be extended to work in all cases where call expressions raise a SyntaxError, although I don't know that it _should_ be. For example, people who for whatever reason prefer to write "f = lambda x: ?" instead of "def f(x): return ?" today would probably love being able to write "f(x) = ?", but I don't really want to encourage those people? > but I think it would make for very > readable code in those cases. And it doesn't use any > colons! I don't understand the problem with the colons, but never mind that; I agree that it's very readable. And the real benefit to me is that it doesn't require any new and potentially weird syntax, like Nick's magic ? parameter, for the one-argument case. On the other hand, could this add any confusion? Today, we have: ? ? Button("Do it!", on_click=fire_the_ducks) # good ? ? Button("Do it!", on_click=fire_the_ducks()) # bad, passes call result as function ? ? Button("Do it!", on_click=lambda: fire_the_ducks()) # good ? ? Button("Do it!", on_click=lambda: fire_the_ducks) # bad, passes function returning function We'd be adding: ? ? Button("Do it!", on_click()=fire_the_ducks()) # good ? ? Button("Do it!", on_click()=fire_the_ducks) # bad, passes function returning function Would that last case add to the novices' confusion? And now, the parsing: First, in the grammar (see 6.3.4 Calls), you have to expand the left side of keyword_item. The simplest idea is: ? ? keyword_item = identifier [ "(" [parameter-list] ")" ] "=" expression Then, the keyword AST node expands to take all the same args-related attributes of the Lambda node: ? ? keyword(arg="on_click", args=[], vararg=None, kwonlyargs=[], kw_defaults=[], kwargs=None, value=Call(?)) Then, at compile time, if args is None, you just compile arg and value as today and push them on the stack; if it's not (even an empty list), you instead compile args, vararg, ?, value together as a function, just as you would a Lambda node except that "body" is called "value",then push the arg and that function on the stack. That doesn't feel right. On the other hand, there's a simpler way to do it: ? ? keyword_item ::= simple_keyword_item | functional_keyword_item ? ? simple_keyword_item = identifier?"=" expression ? ??functional_keyword_item = identifier "(" [parameter_list] ")" "=" expression Now, simple_keyword_item parses to the same keyword AST node as today, and functional_keyword_item also parses into a normal keyword node, which has a normal Lambda node as a value, built from the parameter_list and expression the exact same way as in a lambda expression. That seems pretty clear and simple, but now at the AST level there's no way to distinguish between "key(x)=x.date" and "key=lambda x: x.date". Is that acceptable? Last, there's always a hybrid: create a new KeyLambda node that has the same attributes as Lambda and compiles the same but can be distinguished from it by type, and maybe even a funckeyword that's identical to keyword as well. Then, no magic, and no irreversible parse either. From haoyi.sg at gmail.com Wed Feb 26 20:52:50 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Wed, 26 Feb 2014 11:52:50 -0800 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> Message-ID: I do not feel that this is worth it for the specific case. Having a more generic "better syntax for lambdas" will do everything that this does, + add less heterogeneity to the language, + is useful in other places too. On Wed, Feb 26, 2014 at 11:44 AM, Andrew Barnert wrote: > From: Greg Ewing > > Sent: Tuesday, February 25, 2014 4:17 PM > > > > I was reading a blog post here: > > > > http://stupidpythonideas.blogspot.co.nz/2014/02/fixing-lambda.html > > > > where the author points out that there are a number > > of different problems that the various enhanced-lambda > > proposals are trying to solve, which are best addressed > > by different solutions. > > > > Here's a suggestion for one of them. Suppose we could > > write things like: > > > > sorted(things, key(x) = x.date) > > > > Button("Do it!", on_click() = fire_the_ducks()) > > > Of course that particular case would be better written as: > > Button("Do it!", on_click = fire_the_ducks) > > > But it becomes more useful if you do anything else: > > Button("Do it!", on_click() = fire_the_ducks(42)) > > > At first glance, I think this is nice, but there's a nagging feeling that > it may be a bit magical. Maybe if I think through what the compiler will do > with it, I can resolve that feeling. (Obviously real users aren't going to > care how it gets parsed and compiled, but if it's simple and clear enough, > that implies that it can also be simple and clear to a human reader. Not > perfectly, but... anyway, let's try it.) I'll do that at the end. > > > > It only addresses the case of passing a function using > > > a keyword argument > > It also doesn't look quite as nice when the function comes first, but I > think it's still pretty nice: > > itertools.takewhile(predicate(x)=x<5, iterable=spam) > > And functions that take a function and then *args, like map, would be > tricky, but then it's fine if this doesn't work nicely in every possible > case for passing a function around. > > Also, this _could_ be extended to work in all cases where call expressions > raise a SyntaxError, although I don't know that it _should_ be. For > example, people who for whatever reason prefer to write "f = lambda x: ..." > instead of "def f(x): return ..." today would probably love being able to > write "f(x) = ...", but I don't really want to encourage those people... > > > but I think it would make for very > > readable code in those cases. And it doesn't use any > > colons! > > > I don't understand the problem with the colons, but never mind that; I > agree that it's very readable. And the real benefit to me is that it > doesn't require any new and potentially weird syntax, like Nick's magic ? > parameter, for the one-argument case. > > On the other hand, could this add any confusion? Today, we have: > > Button("Do it!", on_click=fire_the_ducks) # good > > Button("Do it!", on_click=fire_the_ducks()) # bad, passes call result > as function > > Button("Do it!", on_click=lambda: fire_the_ducks()) # good > > Button("Do it!", on_click=lambda: fire_the_ducks) # bad, passes > function returning function > > > We'd be adding: > > Button("Do it!", on_click()=fire_the_ducks()) # good > > Button("Do it!", on_click()=fire_the_ducks) # bad, passes function > returning function > > > Would that last case add to the novices' confusion? > > And now, the parsing: > > First, in the grammar (see 6.3.4 Calls), you have to expand the left side > of keyword_item. > > The simplest idea is: > > keyword_item = identifier [ "(" [parameter-list] ")" ] "=" expression > > Then, the keyword AST node expands to take all the same args-related > attributes of the Lambda node: > > keyword(arg="on_click", args=[], vararg=None, kwonlyargs=[], > kw_defaults=[], kwargs=None, value=Call(...)) > > Then, at compile time, if args is None, you just compile arg and value as > today and push them on the stack; if it's not (even an empty list), you > instead compile args, vararg, ..., value together as a function, just as you > would a Lambda node except that "body" is called "value",then push the arg > and that function on the stack. > > That doesn't feel right. On the other hand, there's a simpler way to do it: > > > keyword_item ::= simple_keyword_item | functional_keyword_item > simple_keyword_item = identifier "=" expression > functional_keyword_item = identifier "(" [parameter_list] ")" "=" > expression > > Now, simple_keyword_item parses to the same keyword AST node as today, and > functional_keyword_item also parses into a normal keyword node, which has a > normal Lambda node as a value, built from the parameter_list and expression > the exact same way as in a lambda expression. > > That seems pretty clear and simple, but now at the AST level there's no > way to distinguish between "key(x)=x.date" and "key=lambda x: x.date". Is > that acceptable? > > Last, there's always a hybrid: create a new KeyLambda node that has the > same attributes as Lambda and compiles the same but can be distinguished > from it by type, and maybe even a funckeyword that's identical to keyword > as well. Then, no magic, and no irreversible parse either. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Wed Feb 26 20:55:58 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 26 Feb 2014 11:55:58 -0800 (PST) Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> <87zjlftxg3.fsf@uwakimon.sk.tsukuba.ac.jp> <87ob1utk7a.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <1393444558.52570.YahooMailNeo@web181003.mail.ne1.yahoo.com> From: Ron Adam Sent: Wednesday, February 26, 2014 7:28 AM > Dictionaries are pretty flexible on how they are initiated, so it's > surprising we can't do this... > > ? ? ? ? D = dict(keys=names, values=args) > > The .fromkeys() method is almost that, but sets all the values to a single > value.? I think I would have written that a bit different. > > ? ? ? def fromkeys(self, keys, values=None, default=None): > ? ? ? ? ? ? D = {} > ? ? ? ? ? ? if D is not None: > ? ? ? ? ? ? ? ? D.update(zip(keys, values)] > ? ? ? ? ? ? for k in keys[len(vaues):]: > ? ? ? ? ? ? ? ? D[k] = default > ? ? ? ? ? ? return D > > And probably named it withkeys instead of fromkeys.? ? It's > what I expected fromkeys to do. Sounds like you're thinking in Smalltalk/ObjC terms, both the "with" name and the expecting two "withs": ? ? [NSDictionary dictionaryWithObjects:values forKeys:keys] The reason we don't need this in Python is that the default construction method takes key-value pairs, and you can get that trivially from zip: ? ? dict(zip(keys, values)) Would it really be more readable your way? ? ? dict.withkeys(keys, values=values) Yes, to novices who haven't internalized zip yet. I guess the question is whether?requiring people to internalize zip early is a good thing about Python, or a problem to be solved. It's not like we're requiring hundreds of weird functional idioms to make everything as brief as possible, just a very small number, each an abstraction that works consistently across a broad range of uses. From abarnert at yahoo.com Wed Feb 26 21:12:36 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 26 Feb 2014 12:12:36 -0800 (PST) Subject: [Python-ideas] time.Timer In-Reply-To: References: Message-ID: <1393445556.16309.YahooMailNeo@web181004.mail.ne1.yahoo.com> From: anatoly techtonik Sent: Tuesday, February 25, 2014 9:24 PM > UX improvement fix for the common case: > > import time > > class Timer(object): > ? def __init__(self, seconds): > ? ? self.seconds = seconds > ? ? self.restart() > > ? def restart(self): > ? ? self.end = time.time() + self.seconds > > ? @property > ? def expired(self): > ? ? return (time.time() > self.end) time.time() is almost always the wrong thing to use for almost anything. It's a float, it doesn't guarantee precision better than 1 second, it can be slow (on the order of milliseconds), and it doesn't guarantee monotonicity.?You probably wanted monotonic here, although without knowing your use case, it's possible you wanted monotonic_raw with a fallback to monotonic, or perf_counter. Also, this kind of timer is only really appropriate for a frame-loop app (as in a traditional arcade game or digital audio app), where you're inherently only checking, say, once every 20ms when idle and less often when busy. But any such loop should be sharing a _single_ call to the clock for the entire loop instance, not reading the clock repeatedly. Plus, most such programs in Python are probably written in something like pygame, which already provides a nicely-integrated timer (and many of the rest probably should be?). In an event loop, or a program based on threads or coroutines, or anything other than a frame loop, this is the wrong thing to do. From ron3200 at gmail.com Wed Feb 26 21:16:41 2014 From: ron3200 at gmail.com (Ron Adam) Date: Wed, 26 Feb 2014 14:16:41 -0600 Subject: [Python-ideas] Infix functions In-Reply-To: <4CC21E88-EFA9-465D-AD1A-F449E81F877D@yahoo.com> References: <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> <8AB414A2-5156-4DAB-AC7C-47A6A283CED4@yahoo.com> <20140224013540.GM3684@ando> <8761o5uus0.fsf@uwakimon.sk.tsukuba.ac.jp> <20140224152827.GP3684@ando> <06A045B6-C169-4886-B19A-EC932F6A95BD@yahoo.com> <530D20E9.1070509@canterbury.ac.nz> <4CC21E88-EFA9-465D-AD1A-F449E81F877D@yahoo.com> Message-ID: On 02/26/2014 12:19 PM, Andrew Barnert wrote: > On Feb 25, 2014, at 15:02, Greg Ewing wrote: > >> >Andrew Barnert wrote re Nick's concise lambda proposal: >> > >>> >> data = using(:?.read(), open(path)) >>> >> buf = catch(:read_file(path), (FileNotFoundError, :'') >>> >> b = cond(a, :1/a, :NaN) >> > >> >This doesn't look like Python to me. It's too cryptic, >> >especially with the '?'. > Yeah, that was my initial reaction to Nick's suggestion, but it's growing on me. Anyway, there's a separate thread on that, which is stalled while I gather use cases from real code. (The stdlib uses lambdas much less than user-level code does, but the docs are full of them...) > >> >If you think a colon in >> >an except-expression looks too confusing, this would >> >be far worse. > Personally, I have no problem with the colon in the except expression. > >> >Also, this kind of style would make it very easy to >> >make subtle mistakes by getting the colons in the >> >wrong places. > I'm not sure about that. I think in most cases, anywhere you put the colons wrong will be a SyntaxError. And most of the exceptions will be places where you should be using a full-syntax lambda or an out-of-line function anyway (e.g., in a dict mapping names to functions). But until I gather more examples I don't think I can argue that effectively. Something I thought about when reading Gregs post is it would be nice if these special expressions, or complex expressions, used some sort of common general rules. They would be more like macro's in that sense, but not completely. But it's not that easy is it? For example if we used an introducer.. "$" before parentheses... $(...). And maybe the first word is a type or keyword, followed by a space. No colon is needed there. $(except ...) $(with ...) $(lambda ...) $(_ ...) # local lambda with no args And ... $(dict ...) dict comprehension $(list ...) list comprehension $(gen ...) generator expression containing yield/yield from It's a bit more verbose which some people may like, and maybe other won't. (?) Having the keyword or type fist is also not always wanted, but it does help self document the expression. The difference between these and functions is that these are free to not evaluate a sub part until it's needed. Functions can't do that. And they aren't really objects either although they may resolve to an object. How the rest of the expression is handles is another question. What I like about this concept is it clearly separates out complex expressions and helps to keep the core language cleaner and easier to learn for beginners. Ron >> >It's not quite the same thing as a Lisp macro, because >> >the onus is on the caller to remember which arguments >> >need to be quoted. In Lisp, the macro takes care of >> >that. > Right, it's more like a regular function that takes sexprs, where it's up to the caller to quote them. But it's not even really a parallel with that, because this isn't quoting, it really is explicitly making a function. From abarnert at yahoo.com Wed Feb 26 21:16:45 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 26 Feb 2014 12:16:45 -0800 (PST) Subject: [Python-ideas] Allow __len__ to return infinity In-Reply-To: <3517a862-205d-46f0-8a0f-b93e38451945@googlegroups.com> References: <3517a862-205d-46f0-8a0f-b93e38451945@googlegroups.com> Message-ID: <1393445805.2742.YahooMailNeo@web181004.mail.ne1.yahoo.com> I think my original reply got swallowed by Google Groups again, so apologies if this is a repeat to some or all? From: Ram Rachum Sent: Tuesday, February 25, 2014 2:02 AM >I'd like to have some objects that return infinity from their __len__ method. Unfortunately the __len__ method may only return an int, and it's impossible to represent an infinity as an int. Do you think that Python could allow returning infinity from __len__? What code do you have that needs to distinguish between actually infinite iterables and potentially infinite ones? Presumably you still want iterators to raise a TypeError rather than try to guess, so as soon as you pass an infinite iterable through a genexpr or itertools function it's going to turn into a length-less iterable. Are you looking for this as a sort of intermediate step from length-less iterables to the kind of lazy lists that some functional languages have? You actually can build a lazy list (implementing Sequence by storing a list of already-seen-prefix and an iterator of not-yet-seen values) pretty easily (and write a lazytools module full of functions like the relevant itertools functions). I did this a few years back, but never found a good use for it and abandoned it.? Anyway, if so, a lazy list that doesn't know its length already needs to say so by raising an exception from __len__, and a lazy list that knows its length is infinite can do the same thing, and I couldn't think of any uses cases where that would be a problem, even though I expected there would be some. So, if you have one, I'd love to hear it. From ron3200 at gmail.com Wed Feb 26 21:54:30 2014 From: ron3200 at gmail.com (Ron Adam) Date: Wed, 26 Feb 2014 14:54:30 -0600 Subject: [Python-ideas] Method chaining notation In-Reply-To: <1393444558.52570.YahooMailNeo@web181003.mail.ne1.yahoo.com> References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> <87zjlftxg3.fsf@uwakimon.sk.tsukuba.ac.jp> <87ob1utk7a.fsf@uwakimon.sk.tsukuba.ac.jp> <1393444558.52570.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: On 02/26/2014 01:55 PM, Andrew Barnert wrote: > From: Ron Adam > > Sent: Wednesday, February 26, 2014 7:28 AM > > >> >Dictionaries are pretty flexible on how they are initiated, so it's >> >surprising we can't do this... >> > >> > D = dict(keys=names, values=args) >> > >> >The .fromkeys() method is almost that, but sets all the values to a single >> >value. I think I would have written that a bit different. >> > >> > def fromkeys(self, keys, values=None, default=None): >> > D = {} >> > if D is not None: >> > D.update(zip(keys, values)] >> > for k in keys[len(vaues):]: >> > D[k] = default >> > return D >> > >> >And probably named it withkeys instead of fromkeys. It's >> >what I expected fromkeys to do. > > Sounds like you're thinking in Smalltalk/ObjC terms, both the "with" name and the expecting two "withs": > > [NSDictionary dictionaryWithObjects:values forKeys:keys] > > The reason we don't need this in Python is that the default construction method takes key-value pairs, and you can get that trivially from zip: > > dict(zip(keys, values)) > > Would it really be more readable your way? > > dict.withkeys(keys, values=values) > > Yes, to novices who haven't internalized zip yet. I guess the question is whether requiring people to internalize zip early is a good thing about Python, or a problem to be solved. It's not like we're requiring hundreds of weird functional idioms to make everything as brief as possible, just a very small number, each an abstraction that works consistently across a broad range of uses. The reason I expected to be able to do that is you can get just keys, or values from a dict... dict.keys() dict.values(), and pairs.. dict.items(). It just makes sense that it will take those directly too. I'm sure I'm not the only one who thought that. I don't really buy the because we can do... ... and get... as a valid reason by it's self not to do something. Not when it's clearly related to the objects type as keys, and values are. For unrelated things, yes, it is a valid reason. For example you could say, we don't need dict.items because we can do... zip(dict.keys(), dict.values()) Or we don't need dict.keys() and dict.values because we can do... [x for x, y in dict.items()] [y for x, y in dict.itmes()] But dictionaries are used so often that having these methods really helps to make the code more readable and easy to use. (or beautiful to the eye of the programmer) In any case.. It's just my opinion. Not trying to convince anyone we need it or to do it. If it was really needed, we'd have it already. (although that argument isn't very strong either. ;-) Cheers, Ron From mertz at gnosis.cx Wed Feb 26 22:17:59 2014 From: mertz at gnosis.cx (David Mertz) Date: Wed, 26 Feb 2014 13:17:59 -0800 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> <87zjlftxg3.fsf@uwakimon.sk.tsukuba.ac.jp> <87ob1utk7a.fsf@uwakimon.sk.tsukuba.ac.jp> <1393444558.52570.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: On Wed, Feb 26, 2014 at 12:54 PM, Ron Adam wrote: > For example you could say, we don't need dict.items because we can do... >> > > zip(dict.keys(), dict.values()) > Have we actually been promised that d.keys() and d.values() walk the (unordered) dictionary in the same order, for every Python implementation/version? While I think it is almost certainly true in practice, I haven't where this invariant is guaranteed: assert [d[k] for k in d] == d.values() I could trivially subclass dict to make a pretty good dictionary that violated this invariant, e.g.: class SortedDict(dict): def keys(self): return sorted(dict.keys(self)) def values(self): return sorted(dict.values(self)) I actually don't have great difficulty imagining purposes for which this would be a useful data structure even. Clearly it violates the invariant listed though. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Wed Feb 26 22:30:49 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 27 Feb 2014 10:30:49 +1300 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> <87zjlftxg3.fsf@uwakimon.sk.tsukuba.ac.jp> <87ob1utk7a.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <530E5D09.7020209@canterbury.ac.nz> Ron Adam wrote: > Dictionaries are pretty flexible on how they are initiated, so it's > surprising we can't do this... > > D = dict(keys=names, values=args) All keywords are taken by the dict(name = value, ...) constructor, so this is not so surprising. But you can write that as D = dict(zip(names, values)) -- Greg From greg.ewing at canterbury.ac.nz Wed Feb 26 22:52:47 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 27 Feb 2014 10:52:47 +1300 Subject: [Python-ideas] Infix functions In-Reply-To: <4CC21E88-EFA9-465D-AD1A-F449E81F877D@yahoo.com> References: <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> <8AB414A2-5156-4DAB-AC7C-47A6A283CED4@yahoo.com> <20140224013540.GM3684@ando> <8761o5uus0.fsf@uwakimon.sk.tsukuba.ac.jp> <20140224152827.GP3684@ando> <06A045B6-C169-4886-B19A-EC932F6A95BD@yahoo.com> <530D20E9.1070509@canterbury.ac.nz> <4CC21E88-EFA9-465D-AD1A-F449E81F877D@yahoo.com> Message-ID: <530E622F.6000407@canterbury.ac.nz> Andrew Barnert wrote: > On Feb 25, 2014, at 15:02, Greg Ewing wrote: > >>Also, this kind of style would make it very easy to >>make subtle mistakes by getting the colons in the >>wrong places. > > I'm not sure about that. I think in most cases, anywhere you put the colons wrong will be a SyntaxError. No, what I mean is forgetting to put a colon in front of an expression that should have one, or vice versa. It's related to my next point: >>It's not quite the same thing as a Lisp macro, because >>the onus is on the caller to remember which arguments >>need to be quoted. -- Greg From rosuav at gmail.com Wed Feb 26 22:56:32 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 27 Feb 2014 08:56:32 +1100 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <5307CC96.9020200@gmail.com> <4D4C984E-D84D-4278-AD49-A0554CD725E6@masklinn.net> <53087A9F.7050100@gmail.com> <9BC68697-9B93-4699-9448-F3BF42AF691B@masklinn.net> <5308C5E8.40103@gmail.com> <87bnxxv13e.fsf@uwakimon.sk.tsukuba.ac.jp> <87zjlftxg3.fsf@uwakimon.sk.tsukuba.ac.jp> <87ob1utk7a.fsf@uwakimon.sk.tsukuba.ac.jp> <1393444558.52570.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: On Thu, Feb 27, 2014 at 8:17 AM, David Mertz wrote: > On Wed, Feb 26, 2014 at 12:54 PM, Ron Adam wrote: >>> >>> For example you could say, we don't need dict.items because we can do... >> >> >> zip(dict.keys(), dict.values()) > > > Have we actually been promised that d.keys() and d.values() walk the > (unordered) dictionary in the same order, for every Python > implementation/version? While I think it is almost certainly true in > practice, I haven't where this invariant is guaranteed: Yes, it's promised. It's intrinsic to the definition of keys()/values(). http://docs.python.org/3.4/library/stdtypes.html#dict-views """ Keys and values are iterated over in an arbitrary order which is non-random, varies across Python implementations, and depends on the dictionary?s history of insertions and deletions. If keys, values and items views are iterated over with no intervening modifications to the dictionary, the order of items will directly correspond. This allows the creation of (value, key) pairs using zip(): pairs = zip(d.values(), d.keys()). Another way to create the same list is pairs = [(v, k) for (k, v) in d.items()]. """ http://docs.python.org/2/library/stdtypes.html#mapping-types-dict """ If items(), keys(), values(), iteritems(), iterkeys(), and itervalues() are called with no intervening modifications to the dictionary, the lists will directly correspond. This allows the creation of (value, key) pairs using zip(): pairs =zip(d.values(), d.keys()). The same relationship holds for the iterkeys() and itervalues() methods: pairs = zip(d.itervalues(), d.iterkeys()) provides the same value for pairs. Another way to create the same list is pairs = [(v,k) for (k, v) in d.iteritems()]. """ The latter has a "CPython implementation detail" box immediately above the piece I quoted, which in a way emphasizes the fact that the bit not in the box is not CPython-specific. Note the criteria, by the way. You have to not modify the dictionary in any way in between. I believe CPython will maintain iteration order as long as the set of keys never changes, but it would be legal for a compliant Python implementation to assert-fail here: d = {1:2, 3:4, 5:6, 7:8, 9:10} items = list(d.items()) d[3] = 4 # Breakage assert items == list(d.items()) However, retrieval is safe. A compliant Python will never assert-fail if the breakage line is changed to: spam = d[3] # No breakage So a splay tree implementation (say) would have to have some other means of iterating, or it would have to not adjust itself on reads. ChrisA From haoyi.sg at gmail.com Wed Feb 26 23:13:14 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Wed, 26 Feb 2014 14:13:14 -0800 Subject: [Python-ideas] Infix functions In-Reply-To: <530E622F.6000407@canterbury.ac.nz> References: <1393037109.35138.YahooMailNeo@web181002.mail.ne1.yahoo.com> <5308FBDB.6030709@krosing.net> <20140223012306.GH3684@ando> <20108B0E-297D-4704-B93E-943BE329016B@yahoo.com> <8AB414A2-5156-4DAB-AC7C-47A6A283CED4@yahoo.com> <20140224013540.GM3684@ando> <8761o5uus0.fsf@uwakimon.sk.tsukuba.ac.jp> <20140224152827.GP3684@ando> <06A045B6-C169-4886-B19A-EC932F6A95BD@yahoo.com> <530D20E9.1070509@canterbury.ac.nz> <4CC21E88-EFA9-465D-AD1A-F449E81F877D@yahoo.com> <530E622F.6000407@canterbury.ac.nz> Message-ID: > $(_ ...) # local lambda with no args *haoyi$ sudo pip install macropy* *haoyi$ python* *Python 2.7.5 (default, Aug 25 2013, 00:04:04)* *[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin* *Type "help", "copyright", "credits" or "license" for more information.* *>>> import macropy.console* *0=[]=====> MacroPy Enabled <=====[]=0* *>>> from macropy.quick_lambda import macros, f, _* *>>> from random import random>>> random()0.26042432926429704>>> local_lambda = f[random() + random()] >>> local_lambda()0.9829393394632971>>> local_lambda()1.4040653196619832* =) On Wed, Feb 26, 2014 at 1:52 PM, Greg Ewing wrote: > Andrew Barnert wrote: > >> On Feb 25, 2014, at 15:02, Greg Ewing >> wrote: >> >> Also, this kind of style would make it very easy to >>> make subtle mistakes by getting the colons in the >>> wrong places. >>> >> >> I'm not sure about that. I think in most cases, anywhere you put the >> colons >> > wrong will be a SyntaxError. > > No, what I mean is forgetting to put a colon in front > of an expression that should have one, or vice versa. > It's related to my next point: > > > It's not quite the same thing as a Lisp macro, because >>> the onus is on the caller to remember which arguments >>> need to be quoted. >>> >> > -- > Greg > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Wed Feb 26 23:16:45 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 27 Feb 2014 11:16:45 +1300 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> Message-ID: <530E67CD.3010105@canterbury.ac.nz> Andrew Barnert wrote: > Button("Do it!", on_click() = fire_the_ducks(42)) > > At first glance, I think this is nice, but there's a nagging feeling that it > may be a bit magical. Maybe if I think through what the compiler will do with > it, I can resolve that feeling. It's quite simple, it's just syntactic sugar for Button("Do it!", on_click = lambda: fire_the_ducks(42)) > We'd be adding: > > Button("Do it!", on_click()=fire_the_ducks()) # good > > Button("Do it!", on_click()=fire_the_ducks) # bad, passes function returning function > > > Would that last case add to the novices' confusion? Each bad case has a corresponding bad case using lambda, so I don't think we'd be adding any more badness overall. >>> And it doesn't use any colons! > > I don't understand the problem with the colons Some people don't like the idea of colons inside expressions. Personally I don't mind, as long as it's done tastefully (e.g. in conjunction with keywords). > And now, the parsing: > > First, in the grammar (see 6.3.4 Calls), you have to expand the left side of > keyword_item. The grammar in the Language Reference is not quite the same as the grammar used by CPython. The parser actually already allows a general expession on the left of the =, and then sorts it out later in the compilation process. So it should be relaively easy to implement. > but now at the AST level there's no way > to distinguish between "key(x)=x.date" and "key=lambda x: x.date". Is that > acceptable? Yes, because there isn't meant to be any semantic difference between them. -- Greg From abarnert at yahoo.com Thu Feb 27 05:13:05 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 26 Feb 2014 20:13:05 -0800 (PST) Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: <530E67CD.3010105@canterbury.ac.nz> References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <530E67CD.3010105@canterbury.ac.nz> Message-ID: <1393474385.73606.YahooMailNeo@web181005.mail.ne1.yahoo.com> From: Greg Ewing Sent: Wednesday, February 26, 2014 2:16 PM >Andrew Barnert wrote: >> Button("Do it!", on_click() = fire_the_ducks(42)) >> >> At first glance, I think this is nice, but there's a nagging feeling that it >> may be a bit magical. Maybe if I think through what the compiler will do with >> it, I can resolve that feeling. > >It's quite simple, it's just syntactic sugar for > >???Button("Do it!", on_click = lambda: fire_the_ducks(42)) > >> And now, the parsing: >> >> First, in the grammar (see 6.3.4 Calls), you have to expand the left side of >> keyword_item. > >The grammar in the Language Reference is not quite the same >as the grammar used by CPython. The parser actually already >allows a general expession on the left of the =, and then >sorts it out later in the compilation process. So it should >be relaively easy to implement. Yes, the grammar in the reference doesn't document what the first stage of the CPython parser does, but it does?document what comes out at the AST level, which is the part that should be the same across implementations, and the first thing that's visible from within the interpreter, right? But yeah, within the CPython grammar: ? ? argument: test [comp_for] | test '=' test ?# Really [keyword '='] test >> but now at the AST level there's no way >> to distinguish between "key(x)=x.date" and "key=lambda x: x.date". Is that >> acceptable? > >Yes, because there isn't meant to be any semantic difference >between them. Sure, but given that ASTs are exposed for introspection, is it expected that you should be able to distinguish two cases by their ASTs? I could see, say, MacroPy wanting to. (I could also see that not being important enough to care about, of course.) From steve at pearwood.info Thu Feb 27 05:19:58 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 27 Feb 2014 15:19:58 +1100 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> Message-ID: <20140227041958.GA1383@ando> On Wed, Feb 26, 2014 at 11:44:51AM -0800, Andrew Barnert wrote: > But it becomes more useful if you do anything else: > > ? ? Button("Do it!", on_click() = fire_the_ducks(42)) > > > At first glance, I think this is nice, At first glance, it looks like you are setting the on_click argument to the result of fire_the_ducks(42). This proposed syntax is going to be *really easy* for people to misinterpret when they see it in use. And not just novices -- I think this will be syntax that just begs to be misinterpreted when reading code, and misused when writing it. I think that having special syntax for anonymous function only inside function calls with keyword arguments is a violation of the Zen of Python (see the one about special cases) and the Principle Of Least Surprise. It's really a bad idea to have syntax for a "shorter lambda" that works here: f(arg=***whatever***) but not in these: f(***whatever***) [len, zip, map, ***whatever***, some_function] result = ***whatever***(arg) One of the best things about Python is it's internal consistency. It is remarkably free of special case syntax that works in one place but not in others. Let's keep it that way. -- Steven From steve at pearwood.info Thu Feb 27 05:27:28 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 27 Feb 2014 15:27:28 +1100 Subject: [Python-ideas] Method chaining notation In-Reply-To: References: <87zjlftxg3.fsf@uwakimon.sk.tsukuba.ac.jp> <87ob1utk7a.fsf@uwakimon.sk.tsukuba.ac.jp> <1393444558.52570.YahooMailNeo@web181003.mail.ne1.yahoo.com> Message-ID: <20140227042728.GB1383@ando> On Wed, Feb 26, 2014 at 01:17:59PM -0800, David Mertz wrote: > Have we actually been promised that d.keys() and d.values() walk the > (unordered) dictionary in the same order, for every Python > implementation/version? Yes. Any implementation which breaks the invariant that keys and values will be given in the same order (so long as there are no intervening changes to the dict) is buggy. http://docs.python.org/3/library/stdtypes.html#dictionary-view-objects > I could trivially subclass dict to make a pretty good dictionary that > violated this invariant, e.g.: The invariant only applies to dict :-) -- Steven From abarnert at yahoo.com Thu Feb 27 07:16:13 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 26 Feb 2014 22:16:13 -0800 (PST) Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: <20140227041958.GA1383@ando> References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> Message-ID: <1393481773.23494.YahooMailNeo@web181006.mail.ne1.yahoo.com> From: Steven D'Aprano Sent: Wednesday, February 26, 2014 8:19 PM >On Wed, Feb 26, 2014 at 11:44:51AM -0800, Andrew Barnert wrote: > >> But it becomes more useful if you do anything else: >> >> ? ? Button("Do it!", on_click() = fire_the_ducks(42)) >> >> >> At first glance, I think this is nice, > >At first glance, it looks like you are setting the on_click argument to >the result of fire_the_ducks(42).? It's also worth noting that this is exactly the kind of case where Nick's proposal also looks nice: ? ? Button("Do it!", on_click=:fire_the_ducks(42)) The test is in cases where Nick's syntax looks weird, either because it takes an argument, or because it's more symbols than letters: ? ? Button("Do it!", on_click=lambda btn: btn.fire_the_ducks(42)) ? ? Button("Do it!", on_click(btn)=btn.fire_the_ducks(42)) ? ? Button("Do it!", on_click=:?.fire_the_ducks(42)) You can try adding whitespace to the Nick version of last example anywhere you want, and it's still going to hurt to look at. But the Greg version looks actually clearer here than in the no-argument case. >It's really a bad idea to have syntax for a "shorter lambda"? >that works here: > >? ? f(arg=***whatever***) > >but not in these: I don't think that's _necessarily_ a problem. After all, it's not a problem that you can filter a for loop in a comprehension but not anywhere else, is it? And it's not like there would be any confusion. If someone wants to know "how do I write a lambda function in a list", the answer is just "write lambda: spam(x)", not "you can't do it". However? >? ? f(***whatever***) That one seems like the easiest to dismiss?after all,?you can, and should, use keyword arguments whenever it improves clarity?but it turns out to be the most serious problem.?First, many of the key higher-order functions, even after the recent arg clinic work, still don't take keyword arguments?like almost everything in itertools.?Second, many of these functions take a predicate first, meaning you have to keyword everything if you want to keyword that. And at that point, I think you lose all the benefits. Compare: ? ? takewhile(lambda x: x<5, a) ? ? takewhile(predicate(x)=x<5, iterable=a) In fact, even in cases where there aren't later parameters, compare: ? ? defaultdict(lamdba: []) ? ? defaultdict(default_factory()=[]) I think in both cases, the second one gains little or nothing in clarity in exchange for its verbosity. So, it's not really true that you can?always?just use a keyword.?Which means the benefit of the feature overall may be too limited.?I'm digging through examples, and so far, sorting keys and Tkinter callbacks are the only cases I've found that are improved (even assuming you accept the basic premise, which I know you personally don't). From rosuav at gmail.com Thu Feb 27 07:35:19 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 27 Feb 2014 17:35:19 +1100 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: <1393481773.23494.YahooMailNeo@web181006.mail.ne1.yahoo.com> References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> <1393481773.23494.YahooMailNeo@web181006.mail.ne1.yahoo.com> Message-ID: On Thu, Feb 27, 2014 at 5:16 PM, Andrew Barnert wrote: >>It's really a bad idea to have syntax for a "shorter lambda" > >>that works here: >> >> f(arg=***whatever***) >> >>but not in these: > > I don't think that's _necessarily_ a problem. After all, it's not a problem that you can filter a for loop in a comprehension but not anywhere else, is it? I do. spam = ***whatever1*** ***whatever2*** spam ***whatever3*** ***whatever2*** (***whatever1***) ***whatever3*** Regardless of the exact values of the whatevers, these should be equivalent (apart from the fact that the first one forces evaluation of whatever1, while the second might not). You can't, however, put a filter onto spam, so that comparison with the comprehension isn't fair. There should be absolutely no difference between: f(arg=***whatever***) and: spam=***whatever*** f(arg=spam) (Okay, okay, the bytecode will differ, and the second one evaluates f after working out the argument value, and so on. I mean in normal usage.) Breaking that expectation would confuse a lot of people. It's problematic because it's still legal - if it threw a SyntaxError, it would at least be visible, but it doesn't: spam=fire_the_ducks(42) f(onclick()=spam) ChrisA From greg.ewing at canterbury.ac.nz Thu Feb 27 08:14:08 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 27 Feb 2014 20:14:08 +1300 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: <20140227041958.GA1383@ando> References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> Message-ID: <530EE5C0.3020405@canterbury.ac.nz> Steven D'Aprano wrote: > It's really a bad idea to have syntax for a "shorter lambda" > that works here: > > f(arg=***whatever***) > > but not in these: > > f(***whatever***) > [len, zip, map, ***whatever***, some_function] > result = ***whatever***(arg) The last one could be made to work as well, then it would be less of a special case. It would be a specialised form of assignment that's optimised for binding a name to a function, similar to def except that the body is an expression rather than a statement. -- Greg From steve at pearwood.info Thu Feb 27 08:30:51 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 27 Feb 2014 18:30:51 +1100 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: <1393481773.23494.YahooMailNeo@web181006.mail.ne1.yahoo.com> References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> <1393481773.23494.YahooMailNeo@web181006.mail.ne1.yahoo.com> Message-ID: <20140227073051.GC1383@ando> On Wed, Feb 26, 2014 at 10:16:13PM -0800, Andrew Barnert wrote: > From: Steven D'Aprano > >It's really a bad idea to have syntax for a "shorter lambda"? > >that works here: > > > >? ? f(arg=***whatever***) > > > >but not in these: > > I don't think that's _necessarily_ a problem. After all, it's not a > problem that you can filter a for loop in a comprehension but not > anywhere else, is it? I'll note that the ability to write "for x in seq if condition(x)" is a frequently requested enhancement to for-loops. Special case syntax, as suggested here, means that the user has to learn a whole lot of special cases. Instead of learning: "you can create with " you have to learn: "you can create with but only under " which increases the burden of learning the language. Python is relatively easy to learn and remember because it is so consistent: if a piece of syntax works here, it almost always works there as well: Q: Can I create a function using lambda inside a list? A: Yes, you can create a function using lambda anywhere an expression is legal. Q: Can I create a function using lambda inside a dict? A: Yes, you can create a function using lambda in a dict. Q: How about inside a function call parens? A: Expressions are legal inside parens, so yes you can use lambda when calling a function. Q: How about inside a function call using a keyword parameter? A: Yes, whether the parameter is given by keyword or not makes no difference. Q: What about inside a set? Bet it doesn't work inside sets. A: Yes, it works inside sets. Q: How about tuples? A: Yes, it works inside tuples. Q. What, even on Fridays? There are very few exceptions to this consistency, and most of them can be sorted out by using parens. You nearly always can copy a valid expression from one part of a .py file, paste it into a similar but not identical context, and be very confident that it will be syntactically valid in the new context as well. -- Steven From greg.ewing at canterbury.ac.nz Thu Feb 27 08:32:48 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 27 Feb 2014 20:32:48 +1300 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> <1393481773.23494.YahooMailNeo@web181006.mail.ne1.yahoo.com> Message-ID: <530EEA20.1040805@canterbury.ac.nz> Chris Angelico wrote: > It's > problematic because it's still legal - if it threw a SyntaxError, it > would at least be visible, but it doesn't: > > spam=fire_the_ducks(42) > f(onclick()=spam) That's equivalent to spam = fire_the_ducks(42) f(onclick = lambda: spam) which is not a syntax error either, but it's just as wrong, and I'm not convinced that it's a harder mistake to make. For what it's worth, the following *could* be made to work: spam() = fire_the_ducks(42) f(onclick = spam) -- Greg From rosuav at gmail.com Thu Feb 27 08:54:55 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 27 Feb 2014 18:54:55 +1100 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: <530EEA20.1040805@canterbury.ac.nz> References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> <1393481773.23494.YahooMailNeo@web181006.mail.ne1.yahoo.com> <530EEA20.1040805@canterbury.ac.nz> Message-ID: On Thu, Feb 27, 2014 at 6:32 PM, Greg Ewing wrote: > Chris Angelico wrote: >> >> It's >> problematic because it's still legal - if it threw a SyntaxError, it >> would at least be visible, but it doesn't: >> >> spam=fire_the_ducks(42) >> f(onclick()=spam) > > > That's equivalent to > > spam = fire_the_ducks(42) > f(onclick = lambda: spam) > > which is not a syntax error either, but it's just as wrong, > and I'm not convinced that it's a harder mistake to make. But one of them is breaking an expression out and giving it a name, and the other is not. Currently, in this expression: f(arg=g(h)) you can replace f, g, and h with any expressions that result in those same values, and the function call will be exactly the same. But breaking out the expression part of a parenthesized call is suddenly very different. Yes, the same can be seen with lambda, but there it's a colon-based subexpression, and those are very obviously different (like how an if/for/except statement behaves differently from a simple block of code). ChrisA From ncoghlan at gmail.com Thu Feb 27 11:17:56 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 27 Feb 2014 20:17:56 +1000 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: <530EEA20.1040805@canterbury.ac.nz> References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> <1393481773.23494.YahooMailNeo@web181006.mail.ne1.yahoo.com> <530EEA20.1040805@canterbury.ac.nz> Message-ID: On 27 Feb 2014 17:33, "Greg Ewing" wrote: > > Chris Angelico wrote: >> >> It's >> problematic because it's still legal - if it threw a SyntaxError, it >> would at least be visible, but it doesn't: >> >> spam=fire_the_ducks(42) >> f(onclick()=spam) > > > That's equivalent to > > spam = fire_the_ducks(42) > f(onclick = lambda: spam) > > which is not a syntax error either, but it's just as wrong, > and I'm not convinced that it's a harder mistake to make. > > For what it's worth, the following *could* be made to work: > > spam() = fire_the_ducks(42) > f(onclick = spam) Let's talk about that for a moment. It would be a matter of making this: NAME(ARGSPEC) = EXPR syntactic sugar for this: def NAME(ARGSPEC): return EXPR Not what you would call a big win. Cheers, Nick. > > -- > Greg > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron3200 at gmail.com Thu Feb 27 16:05:02 2014 From: ron3200 at gmail.com (Ron Adam) Date: Thu, 27 Feb 2014 09:05:02 -0600 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> <1393481773.23494.YahooMailNeo@web181006.mail.ne1.yahoo.com> <530EEA20.1040805@canterbury.ac.nz> Message-ID: On 02/27/2014 04:17 AM, Nick Coghlan wrote: > > For what it's worth, the following *could* be made to work: > > > > spam() = fire_the_ducks(42) > > f(onclick = spam) > > Let's talk about that for a moment. It would be a matter of making this: > > NAME(ARGSPEC) = EXPR > > syntactic sugar for this: > > def NAME(ARGSPEC): return EXPR > > Not what you would call a big win. A good potential use of lambda is in constructing a dispatch table from a dictionary litteral or constructor. select = {key1:callable, key2:callable, ...} result = select[k]() select = dict(key1=callable, key2=callable, ...) result = select[k]() And there is also the case of it working in a ternary if... result = (callable if k else callable)() #k is True or False I think these are good test cases for evaluating usability and readability. It would be nice if it worked well for each of these. Cheers, Ron From antony.lee at berkeley.edu Thu Feb 27 19:27:19 2014 From: antony.lee at berkeley.edu (Antony Lee) Date: Thu, 27 Feb 2014 10:27:19 -0800 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> <1393481773.23494.YahooMailNeo@web181006.mail.ne1.yahoo.com> <530EEA20.1040805@canterbury.ac.nz> Message-ID: Now that someone mentioned dispatch tables, another possibility would be to support assignment to tables and monkey-patching directly, with a syntax like def obj.method(*args): ... # __name__ = "method" (the attribute name) def table[key](*args): ... # __name__ = ??? (perhaps "table[key]"?) (and also any other "lvalue".) Antony 2014-02-27 7:05 GMT-08:00 Ron Adam : > > > On 02/27/2014 04:17 AM, Nick Coghlan wrote: > >> > For what it's worth, the following *could* be made to work: >> > >> > spam() = fire_the_ducks(42) >> > f(onclick = spam) >> >> Let's talk about that for a moment. It would be a matter of making this: >> >> NAME(ARGSPEC) = EXPR >> >> syntactic sugar for this: >> >> def NAME(ARGSPEC): return EXPR >> >> Not what you would call a big win. >> > > A good potential use of lambda is in constructing a dispatch table from a > dictionary litteral or constructor. > > select = {key1:callable, key2:callable, ...} > result = select[k]() > > > select = dict(key1=callable, key2=callable, ...) > result = select[k]() > > And there is also the case of it working in a ternary if... > > result = (callable if k else callable)() #k is True or False > > I think these are good test cases for evaluating usability and > readability. It would be nice if it worked well for each of these. > > Cheers, > Ron > > > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joshua at landau.ws Thu Feb 27 19:48:25 2014 From: joshua at landau.ws (Joshua Landau) Date: Thu, 27 Feb 2014 18:48:25 +0000 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> <1393481773.23494.YahooMailNeo@web181006.mail.ne1.yahoo.com> <530EEA20.1040805@canterbury.ac.nz> Message-ID: On 27 February 2014 18:27, Antony Lee wrote: > Now that someone mentioned dispatch tables, another possibility would be to > support assignment to tables and monkey-patching directly, with a syntax > like > > def obj.method(*args): ... # __name__ = "method" (the attribute name) > def table[key](*args): ... # __name__ = ??? (perhaps "table[key]"?) > (and also any other "lvalue".) I would very much support this. It's actually odd that you *can't* do it, considering you *can* do class A: ... a = A() l = [0] for a.x in [1, 2, 3]: ... for l[0] in [1, 2, 3]: ... However, there are disadvantages: potentially ambiguous grammar (very) small chance of misuse could hurt introspection Further, I don't imagine I'd use it frequently. Just enough, though. From rosuav at gmail.com Thu Feb 27 19:56:51 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 28 Feb 2014 05:56:51 +1100 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> <1393481773.23494.YahooMailNeo@web181006.mail.ne1.yahoo.com> <530EEA20.1040805@canterbury.ac.nz> Message-ID: On Fri, Feb 28, 2014 at 5:48 AM, Joshua Landau wrote: > On 27 February 2014 18:27, Antony Lee wrote: >> Now that someone mentioned dispatch tables, another possibility would be to >> support assignment to tables and monkey-patching directly, with a syntax >> like >> >> def obj.method(*args): ... # __name__ = "method" (the attribute name) >> def table[key](*args): ... # __name__ = ??? (perhaps "table[key]"?) >> (and also any other "lvalue".) > > I would very much support this. It's actually odd that you *can't* do > it, considering you *can* do > > class A: ... > a = A() > l = [0] > > for a.x in [1, 2, 3]: ... > for l[0] in [1, 2, 3]: ... > > However, there are disadvantages: > > potentially ambiguous grammar > (very) small chance of misuse > could hurt introspection > > Further, I don't imagine I'd use it frequently. Just enough, though. Remember, def creates a function with a name. If there's no obvious name to attach to the function, it'd be better to either: def functionname(...): .... table[key] = functionname or: table[key] = lambda(...): ... to either set a name, or not set a name, as the case may be. ChrisA From techtonik at gmail.com Thu Feb 27 21:32:31 2014 From: techtonik at gmail.com (anatoly techtonik) Date: Thu, 27 Feb 2014 23:32:31 +0300 Subject: [Python-ideas] stdlib process GSoC 2014 ideas Message-ID: Hi, I am looking at empty ideas page at: https://wiki.python.org/moin/SummerOfCode/2014/python-core and I'd like to propose idea for GSoC students. https://bitbucket.org/techtonik/python-stdlib This needs to be extended and integrated into stdlib development process. The idea is to get a focused development gathered around modules. This includes creating a pydotorg pages (Django) that will list feeds of activity per module, including: Iteration one: - bugs - patches - discussions - votes - commits - hiscore based on all the above (shamelessly stolen from https://twistedmatrix.com/highscores/) Iteration two: - sprints info (short term activities and coordination in real-time) - past - planned - current - bug research and development (long term with visualization and reports) - subscribe to the research team - publish research materials - get scores and achievements This is just for the start. -- anatoly t. From timothy.c.delaney at gmail.com Thu Feb 27 22:00:28 2014 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Fri, 28 Feb 2014 08:00:28 +1100 Subject: [Python-ideas] stdlib process GSoC 2014 ideas In-Reply-To: References: Message-ID: On 28 February 2014 07:32, anatoly techtonik wrote: > > https://bitbucket.org/techtonik/python-stdlib > > This needs to be extended and integrated into stdlib development process. > Wow Anatoly. Are you finally going to sign the PSF Contributor Agreement? Tim Delaney -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Thu Feb 27 22:07:04 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 28 Feb 2014 08:07:04 +1100 Subject: [Python-ideas] stdlib process GSoC 2014 ideas In-Reply-To: References: Message-ID: On Fri, Feb 28, 2014 at 8:00 AM, Tim Delaney wrote: > Wow Anatoly. Are you finally going to sign the PSF Contributor Agreement? It's a lot easier to do it now than it was when I first looked into it. Can't remember what it took then, but I remember being put off by a combination of legalese verbiage and administrative fiddliness. The wads of legalese are still there (I basically just trusted that the PSF isn't trying to con me into giving up vital organs for science), but it's now a matter of proving identity and signing digitally. Fairly convenient. ChrisA From steve at pearwood.info Thu Feb 27 22:14:00 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 28 Feb 2014 08:14:00 +1100 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> <1393481773.23494.YahooMailNeo@web181006.mail.ne1.yahoo.com> <530EEA20.1040805@canterbury.ac.nz> Message-ID: <20140227211400.GC28804@ando> On Thu, Feb 27, 2014 at 06:48:25PM +0000, Joshua Landau wrote: > On 27 February 2014 18:27, Antony Lee wrote: > > Now that someone mentioned dispatch tables, another possibility would be to > > support assignment to tables and monkey-patching directly, with a syntax > > like > > > > def obj.method(*args): ... # __name__ = "method" (the attribute name) > > def table[key](*args): ... # __name__ = ??? (perhaps "table[key]"?) > > (and also any other "lvalue".) > > I would very much support this. It's actually odd that you *can't* do > it, considering you *can* do > > class A: ... > a = A() > l = [0] > > for a.x in [1, 2, 3]: ... > for l[0] in [1, 2, 3]: ... I wonder whether that is deliberate feature, or an accident of the way the syntax works. It does seem to be pretty useless though -- why are you using an attribute or list as a loop variable? As far as setting a method on an instance like that, if you're defining methods, 99.9% of the time they ought to be defined on the class, not on the instance. For the remaining one time in a thousand, it is nearly always sufficient to either embed the assignment inside __init__, or use a temporary, external, function: class Whatever: def __init__(self, value): def func(arg): return arg + value self.func = func Things like this are unusual enough that they don't need -- in fact, *shouldn't* have -- dedicated syntax to support them. That syntax simply complicates the interpreter, makes it harder for people to learn the language, adds more decisions for the programmer to make, and just for a marginal (if any) benefit. The list example is slightly better, but not enough to allow it. If you could populate the entire list as a single expression, that might be interesting: functions = [ def spam(): ..., def eggs(): ..., def cheese(): ..., ] but the syntax suggested here doesn't let you do that. You still have a separate statement for each item. All it buys you is saving *one line* per assignment: def spam(): ... functions[0] = spam or two if you absolutely must then unbind the name spam afterwards del spam (but normally I wouldn't bother). > However, there are disadvantages: > > potentially ambiguous grammar > (very) small chance of misuse > could hurt introspection > > Further, I don't imagine I'd use it frequently. Just enough, though. I'd like to see some non-contrived use-cases for where you think you would actually use this. -- Steven From rosuav at gmail.com Thu Feb 27 22:24:07 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 28 Feb 2014 08:24:07 +1100 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: <20140227211400.GC28804@ando> References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> <1393481773.23494.YahooMailNeo@web181006.mail.ne1.yahoo.com> <530EEA20.1040805@canterbury.ac.nz> <20140227211400.GC28804@ando> Message-ID: On Fri, Feb 28, 2014 at 8:14 AM, Steven D'Aprano wrote: >> class A: ... >> a = A() >> l = [0] >> >> for a.x in [1, 2, 3]: ... >> for l[0] in [1, 2, 3]: ... > > I wonder whether that is deliberate feature, or an accident of the way > the syntax works. It does seem to be pretty useless though -- why are > you using an attribute or list as a loop variable? I would say it's no accident that the for loop can accept any lvalue. That's how we can do this, which I'm sure you'll agree is *extremely* useful: for x,y in [(1,2), (3,4), (5,6)]: ... Assigning to the tuple works beautifully there (especially with enumerate or zip). Being able to assign to other complex targets is a bonus that you'll probably never come across (I can imagine someone might have a use for "for self.blah in ...", but can't concoct any right now), and something there's no point in forbidding, but I'm glad complex targets in general are accepted. It does allow stupid stuff, though. Check this out: lst=[10,20,30,40,50] # Any source list dest=[None]*len(lst) for i,dest[i] in enumerate(lst): pass assert dest == lst I suppose in theory there might be a use for that, but if you want to talk about things that accidentally work, I'd say this form of list copy would have to be one of them :) ChrisA (Why do I always come up with the stupidest ideas?) From techtonik at gmail.com Thu Feb 27 22:40:52 2014 From: techtonik at gmail.com (anatoly techtonik) Date: Fri, 28 Feb 2014 00:40:52 +0300 Subject: [Python-ideas] stdlib process GSoC 2014 ideas In-Reply-To: References: Message-ID: On Fri, Feb 28, 2014 at 12:00 AM, Tim Delaney wrote: > On 28 February 2014 07:32, anatoly techtonik wrote: >> >> >> https://bitbucket.org/techtonik/python-stdlib >> >> This needs to be extended and integrated into stdlib development process. > > Wow Anatoly. Are you finally going to sign the PSF Contributor Agreement? This idea is for GSoC students. python-stdlib is free from the license burden with UNLICENSE, so you better ask PSF why they don't accept it. Offtopic, but since it is about GSoC, and there are people studying copyright and open source, I'll explain my position just in case somebody will be able to help with that in the future. I don't sign CLA, because: 1. I don't want to release my code under restrictive PSF license https://tldrlegal.com/license/python-license-2.0 2. PSF doesn't comply with terms of Apache 2.0 license (include license) which is probably chosen by at least one contributor for CLA https://tldrlegal.com/license/apache-license-2.0-(apache-2.0) 3. I don't want to give PSF exclusive rights for all code and documentation to be released under any other "open source" license. PSF may be bought by some corporation and they will have the right to impose their own "open source license text" on it. (yes, I don't trust people at all). 4. If I sign, I will be less motivated to open the Python docs under CC-BY license with examples that can be copy-pasted without requiring PSF license in your project. Right now using logging examples without it is illegal. 5. Everything is owned by PSF is wrong. Python is a community project, and core code should be shared as open as possible (with credits where due). Public domain with optional crediting and patent grant is ideal. Trademarks are not affected. Nobody is forced and can do what they want. And current licensing uncertainty no good for collaboration. 6. I want people to have free entry for participating in open source projects, meaning that the patent grant and agreement to release their contribution under the open source license that project uses, should work by default without any CLAs. From greg.ewing at canterbury.ac.nz Thu Feb 27 23:03:16 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 28 Feb 2014 11:03:16 +1300 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> <1393481773.23494.YahooMailNeo@web181006.mail.ne1.yahoo.com> <530EEA20.1040805@canterbury.ac.nz> Message-ID: <530FB624.6070607@canterbury.ac.nz> Chris Angelico wrote: > Remember, def creates a function with a name. It doesn't *have* to do that, though. If there is no obvious name, it could use a generic fallback name, like lambda does. > def functionname(...): > .... > table[key] = functionname I don't see how that's intrinsically better. > or: > > table[key] = lambda(...): ... > > to either set a name, or not set a name, as the case may be. I don't see having a name vs. not having a name as the most important criterion for deciding whether to use a def or a lambda. -- Greg From tjreedy at udel.edu Thu Feb 27 23:04:21 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 27 Feb 2014 17:04:21 -0500 Subject: [Python-ideas] stdlib process GSoC 2014 ideas In-Reply-To: References: Message-ID: On 2/27/2014 3:32 PM, anatoly techtonik wrote: > Hi, > > I am looking at empty ideas page at: > https://wiki.python.org/moin/SummerOfCode/2014/python-core Please stop posting disinformation. It is not empty. It has listings by the people who have volunteered to be mentors (2 groups of 2). Adding projects without a mentor available is worse than useless. Besides which, your process that no one besides you wants is out of bounds for GSOC students. -- Terry Jan Reedy From ziad.sawalha at rackspace.com Thu Feb 27 23:06:28 2014 From: ziad.sawalha at rackspace.com (Ziad Sawalha) Date: Thu, 27 Feb 2014 22:06:28 +0000 Subject: [Python-ideas] PEP-257: drop recommendation for extra line at end of multi-line docstring Message-ID: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com> PEP-257 includes this recommendation: ?The BDFL [3] recommends inserting a blank line between the last paragraph in a multi-line docstring and its closing quotes, placing the closing quotes on a line by themselves. This way, Emacs' fill-paragraph command can be used on it.? I believe emacs no longer has this limitation. "If you do fill-paragraph in emacs in Python mode within a docstring, emacs already ignores the closing triple-quote. In fact, the most recent version of emacs supports several different docstring formatting styles and gives you the ability to switch between them.? - quoting Kevin L. Mitchell who is more familiar with emacs than I am. I?m considering removing that recommendation and updating some of the examples in PEP-257, but I?d like some thoughts from this group before I submit the patch. Any thoughts or references to conversations that may have already been had on this topic? Regards, Ziad From greg.ewing at canterbury.ac.nz Thu Feb 27 23:21:12 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 28 Feb 2014 11:21:12 +1300 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: <20140227211400.GC28804@ando> References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> <1393481773.23494.YahooMailNeo@web181006.mail.ne1.yahoo.com> <530EEA20.1040805@canterbury.ac.nz> <20140227211400.GC28804@ando> Message-ID: <530FBA58.4090701@canterbury.ac.nz> Steven D'Aprano wrote: > All it buys you is saving *one line* > per assignment: > > def spam(): > ... > functions[0] = spam The motivation isn't to save a line, it's to make the code clearer, like the way decorators make the use of staticmethod and classmethod clearer. -- Greg From abarnert at yahoo.com Thu Feb 27 23:26:11 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 27 Feb 2014 14:26:11 -0800 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> <1393481773.23494.YahooMailNeo@web181006.mail.ne1.yahoo.com> <530EEA20.1040805@canterbury.ac.nz> <20140227211400.GC28804@ando> Message-ID: <7A28CFA8-5360-413B-8B00-F2775863969A@yahoo.com> On Feb 27, 2014, at 13:24, Chris Angelico wrote: > On Fri, Feb 28, 2014 at 8:14 AM, Steven D'Aprano wrote: >>> class A: ... >>> a = A() >>> l = [0] >>> >>> for a.x in [1, 2, 3]: ... >>> for l[0] in [1, 2, 3]: ... >> >> I wonder whether that is deliberate feature, or an accident of the way >> the syntax works. It does seem to be pretty useless though -- why are >> you using an attribute or list as a loop variable? > > I would say it's no accident that the for loop can accept any lvalue. > That's how we can do this, which I'm sure you'll agree is *extremely* > useful: > > for x,y in [(1,2), (3,4), (5,6)]: ... > > Assigning to the tuple works beautifully there (especially with > enumerate or zip). Being able to assign to other complex targets is a > bonus that you'll probably never come across (I can imagine someone > might have a use for "for self.blah in ...", but can't concoct any > right now) Something I've seen in real code (not _good_ code, but actually deployed): for self.index in range(len(self.values)): if self.values[self.index] == spam: break else: self.index = None Apparently someone didn't want to catch the exception from list.index. (Or, more likely, they were trying to write C code in Python.) From guido at python.org Thu Feb 27 23:41:52 2014 From: guido at python.org (Guido van Rossum) Date: Thu, 27 Feb 2014 14:41:52 -0800 Subject: [Python-ideas] PEP-257: drop recommendation for extra line at end of multi-line docstring In-Reply-To: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com> References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com> Message-ID: I haven't followed this recommendation for years. :-) Good riddance! (However, the one about two spaces after a period stands. :-) On Thu, Feb 27, 2014 at 2:06 PM, Ziad Sawalha wrote: > PEP-257 includes this recommendation: > > "The BDFL [3] recommends inserting a blank line between the last paragraph > in a multi-line > docstring and its closing quotes, placing the closing quotes on a line by > themselves. This way, > Emacs' fill-paragraph command can be used on it." > > I believe emacs no longer has this limitation. "If you do fill-paragraph > in emacs in Python mode > within a docstring, emacs already ignores the closing triple-quote. In > fact, the most recent version > of emacs supports several different docstring formatting styles and gives > you the ability to switch > between them." - quoting Kevin L. Mitchell who is more familiar with emacs > than I am. > > I'm considering removing that recommendation and updating some of the > examples in PEP-257, > but I'd like some thoughts from this group before I submit the patch. Any > thoughts or references to > conversations that may have already been had on this topic? > > Regards, > Ziad > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From bruce at leapyear.org Thu Feb 27 23:41:43 2014 From: bruce at leapyear.org (Bruce Leban) Date: Thu, 27 Feb 2014 14:41:43 -0800 Subject: [Python-ideas] PEP-257: drop recommendation for extra line at end of multi-line docstring In-Reply-To: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com> References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com> Message-ID: To clarify are you asking to delete the recommendation of a blank line or the entire recommendation? That is are you suggesting the recommendation change to "It is recommended to place the closing quotes on a line by themselves." I think deleting it completely is a bad idea as otherwise it's hard to see where the docstring ends and the code begins, especially when using doctest. --- Bruce Learn how hackers think: http://j.mp/gruyere-security https://www.linkedin.com/in/bruceleban On Thu, Feb 27, 2014 at 2:06 PM, Ziad Sawalha wrote: > PEP-257 includes this recommendation: > > "The BDFL [3] recommends inserting a blank line between the last paragraph > in a multi-line > docstring and its closing quotes, placing the closing quotes on a line by > themselves. This way, > Emacs' fill-paragraph command can be used on it." > > I believe emacs no longer has this limitation. "If you do fill-paragraph > in emacs in Python mode > within a docstring, emacs already ignores the closing triple-quote. In > fact, the most recent version > of emacs supports several different docstring formatting styles and gives > you the ability to switch > between them." - quoting Kevin L. Mitchell who is more familiar with emacs > than I am. > > I'm considering removing that recommendation and updating some of the > examples in PEP-257, > but I'd like some thoughts from this group before I submit the patch. Any > thoughts or references to > conversations that may have already been had on this topic? > > Regards, > Ziad > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Thu Feb 27 22:58:33 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 27 Feb 2014 13:58:33 -0800 Subject: [Python-ideas] stdlib process GSoC 2014 ideas In-Reply-To: References: Message-ID: <530FB509.9050501@stoneleaf.us> On 02/27/2014 01:40 PM, anatoly techtonik wrote: > On Fri, Feb 28, 2014 at 12:00 AM, Tim Delaney > wrote: >> On 28 February 2014 07:32, anatoly techtonik wrote: >>> >>> >>> https://bitbucket.org/techtonik/python-stdlib >>> >>> This needs to be extended and integrated into stdlib development process. >> >> Wow Anatoly. Are you finally going to sign the PSF Contributor Agreement? > > This idea is for GSoC students. python-stdlib is free from the license burden > with UNLICENSE, so you better ask PSF why they don't accept it. Your reasons are irrelevant. If you don't sign the CLA, no code you start will find its way to Python. -- ~Ethan~ From guido at python.org Thu Feb 27 23:52:27 2014 From: guido at python.org (Guido van Rossum) Date: Thu, 27 Feb 2014 14:52:27 -0800 Subject: [Python-ideas] PEP-257: drop recommendation for extra line at end of multi-line docstring In-Reply-To: References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com> Message-ID: On Thu, Feb 27, 2014 at 2:41 PM, Bruce Leban wrote: > To clarify are you asking to delete the recommendation of a blank line or > the entire recommendation? > > That is are you suggesting the recommendation change to > > "It is recommended to place the closing quotes on a line by themselves." > > This clause must stay. > I think deleting it completely is a bad idea as otherwise it's hard to see > where the docstring ends and the code begins, especially when using doctest. > > --- Bruce > Learn how hackers think: http://j.mp/gruyere-security > https://www.linkedin.com/in/bruceleban > > > > On Thu, Feb 27, 2014 at 2:06 PM, Ziad Sawalha wrote: > >> PEP-257 includes this recommendation: >> >> "The BDFL [3] recommends inserting a blank line between the last >> paragraph in a multi-line >> docstring and its closing quotes, placing the closing quotes on a line by >> themselves. This way, >> Emacs' fill-paragraph command can be used on it." >> >> I believe emacs no longer has this limitation. "If you do fill-paragraph >> in emacs in Python mode >> within a docstring, emacs already ignores the closing triple-quote. In >> fact, the most recent version >> of emacs supports several different docstring formatting styles and gives >> you the ability to switch >> between them." - quoting Kevin L. Mitchell who is more familiar with >> emacs than I am. >> >> I'm considering removing that recommendation and updating some of the >> examples in PEP-257, >> but I'd like some thoughts from this group before I submit the patch. Any >> thoughts or references to >> conversations that may have already been had on this topic? >> >> Regards, >> Ziad >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From joshua at landau.ws Fri Feb 28 00:07:17 2014 From: joshua at landau.ws (Joshua Landau) Date: Thu, 27 Feb 2014 23:07:17 +0000 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> <1393481773.23494.YahooMailNeo@web181006.mail.ne1.yahoo.com> <530EEA20.1040805@canterbury.ac.nz> Message-ID: On 27 February 2014 18:56, Chris Angelico wrote: > On Fri, Feb 28, 2014 at 5:48 AM, Joshua Landau wrote: >> On 27 February 2014 18:27, Antony Lee wrote: >>> Now that someone mentioned dispatch tables, another possibility would be to >>> support assignment to tables and monkey-patching directly, with a syntax >>> like >>> >>> def obj.method(*args): ... # __name__ = "method" (the attribute name) >>> def table[key](*args): ... # __name__ = ??? (perhaps "table[key]"?) >>> (and also any other "lvalue".) >> >> I would very much support this. It's actually odd that you *can't* do >> it, considering you *can* do [some other things] > > Remember, def creates a function with a name. If there's no obvious > name to attach to the function, it'd be better to either: > > def functionname(...): > .... > table[key] = functionname > > or: > > table[key] = lambda(...): ... > > to either set a name, or not set a name, as the case may be. But there *is* a name. The thing is, that name might have most meaning in context. I personally don't see why from a reader's point of view, callbacks["jump"] is a worse name than jump_callback IMHO, I would just force the introspection name to be the expression in full, with the small exception of trimming *direct* dot-access; (such that the qualified name is never worse). Therefore we could have the names going from For callbacks["jump"]: jump_callback ? callbacks["jump"] For myobj.myfunc (unchanged): myfunc ? myfunc For myobj.callbacks[...].myfunc: ellipsis_myfunc_callback ? callbacks[...].myfunc (I realise this example is contrived; it's to lower the number of examples) and the qualified names: For callbacks["jump"]: jump_callback ? callbacks["jump"] For myobj.myfunc: myfunc ? myobj.myfunc For myobj.callbacks[...].myfunc: ellipsis_myfunc_callback ? myobj.callbacks[...].myfunc To me, the naming "problem" is actually a positive. From ian.team.python at gmail.com Fri Feb 28 00:26:02 2014 From: ian.team.python at gmail.com (ian o) Date: Thu, 27 Feb 2014 15:26:02 -0800 (PST) Subject: [Python-ideas] A python bridge between versions Message-ID: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> The most quoted reason for staying with python 2 is that some required library is not available to support python 3. All it takes is one legacy library module in python 2 to keep an entire project in python 2. Many significant projects (e.g web2py) continue in python 2 for this reason. Even projects that produce code that can work under either python 2 or python 3, are themselves trapped into only using python 2 features. This is really holding back python! You cannot just import python2 code into the python 3 interpreter as it is not compatible. But could this be solved by a new approach which is to treat python 2 as another language and not directly import the code but handle python 2 code with an an extension module, which acts as bridge between versions. I mean, python three program can access C/C++ code (which is not python3), why can't the access python 2 code by treating it like C/C++? This would require an extension C module called from python 3. How to do this is well known. This extension C module would itself call the required module library(s) from python2. Calling python from C is also a well known technique. Ptyhon can call C. C can call python. So python3 call C which calls python2. So in python3 imput # this usually won't work because of language changes becomes import python2 = python2.import('') The resultant imported module would appear as a c++ extension to python3 and in effect be a wrapper for the python2 module using the python2 extension to execute the code This would mean programs using python2 imports have the overhead of both interpreters running in memory and calls across the python3/python2 boundary are 'wrapped' as c++ calls that then activate the required code within python 2. In reality quite a small overhead on modern computers. python2.sys.path (within the 'python2' module) or similar would be needed for the separate import space running inside the python2 boundary. Callbacks back across the python3/python2 boundary back into python3 are a further complication but several possible solutions exist to deliver this. Not many library modules require callback functions to the library module itself so this not the central issue. There is some work for the person coding the python 3 app in terms of a different syntax for legacy module imports and potentially setting python 2 environment values (like python2.sys.path), but not a lot and the python3 is the new program being worked on as opposed to the legacy library which can remain untouched. I think this type of approach could change the recommendation on 'python 2 or python 3' to a simple 'get the latest available to you and here is how to use legacy libraries if you have the need'. Thoughts or criticisms? -------------- next part -------------- An HTML attachment was scrubbed... URL: From joshua at landau.ws Fri Feb 28 00:38:56 2014 From: joshua at landau.ws (Joshua Landau) Date: Thu, 27 Feb 2014 23:38:56 +0000 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: <20140227041958.GA1383@ando> References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> Message-ID: On 27 February 2014 04:19, Steven D'Aprano wrote: > On Wed, Feb 26, 2014 at 11:44:51AM -0800, Andrew Barnert wrote: > >> But it becomes more useful if you do anything else: >> >> Button("Do it!", on_click() = fire_the_ducks(42)) >> >> >> At first glance, I think this is nice, > > At first glance, it looks like you are setting the on_click argument to > the result of fire_the_ducks(42). This proposed syntax is going to be > *really easy* for people to misinterpret when they see it in use. And > not just novices -- I think this will be syntax that just begs to be > misinterpreted when reading code, and misused when writing it. > > I think that having special syntax for anonymous function only inside > function calls with keyword arguments is a violation of the Zen of > Python (see the one about special cases) and the Principle Of Least > Surprise. It's really a bad idea to have syntax for a "shorter lambda" > that works here: > > f(arg=***whatever***) > > but not in these: > > f(***whatever***) > [len, zip, map, ***whatever***, some_function] > result = ***whatever***(arg) I don't follow. I know it's different, but we have f(*args) but not [*args] And we have start, *mid, end = [1, 2, 3] but not (lambda start, *mid, end: ...)(1, 2, 3) and we have (lambda x: ...)(*[...]) but not foo = *[...] ... *wink?* It's not silly to think that things are context-sensitive in a context-sensitive language. Personally this proposal makes sense and I honestly don't see the confusion that's being stated. It makes things significantly prettier and more readable, for one. Further, for consistency one can define: matrix.transform((x, y)=(y, x)) as an "anonymous" version, compiling to matrix.transform(lambda x, y: (y, x)) Please admit that this is way, way prettier. PS: One problem. "f( (a) = ... )" is currently valid. So is "f( (((((a))))) = ... )". Why the hell is this so? "(a) = 2" is valid, but so is "(a, b) = (2, 3)", whereas "f( (a, b) = (2, 3) )" is not. ?http://www.python.org/dev/peps/pep-0448/ From ncoghlan at gmail.com Fri Feb 28 00:45:46 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 28 Feb 2014 09:45:46 +1000 Subject: [Python-ideas] stdlib process GSoC 2014 ideas In-Reply-To: <530FB509.9050501@stoneleaf.us> References: <530FB509.9050501@stoneleaf.us> Message-ID: On 28 Feb 2014 08:47, "Ethan Furman" wrote: > > On 02/27/2014 01:40 PM, anatoly techtonik wrote: >> >> On Fri, Feb 28, 2014 at 12:00 AM, Tim Delaney >> wrote: >>> >>> On 28 February 2014 07:32, anatoly techtonik wrote: >>>> >>>> >>>> >>>> https://bitbucket.org/techtonik/python-stdlib >>>> >>>> This needs to be extended and integrated into stdlib development process. >>> >>> >>> Wow Anatoly. Are you finally going to sign the PSF Contributor Agreement? >> >> >> This idea is for GSoC students. python-stdlib is free from the license burden >> with UNLICENSE, so you better ask PSF why they don't accept it. > > > Your reasons are irrelevant. If you don't sign the CLA, no code you start will find its way to Python. But Ethan, you don't understand. Everyone else's interests and obligations are irrelevant, the world is just supposed to conform to Anatoly's every whim. Anatoly: please stop posting ideas inspired solely by your inability to take anyone else's interests into account, even after a PSF director has taken the time to sit down with you at the PyCon US sprints and attempt to explain the legal complexities that led to the introduction of the contributor licensing agreement. Moderators: please don't let such posts out of the moderation queue, they're a complete waste of everyone's time. The core development team has already burned years on Anatoly's antics with nothing much to show for it - it is his decision to opt out of contributing, yet he obstinately refuses to accept the consequence that doing so renders his opinion largely irrelevant to many of us. Regards, Nick. > > -- > ~Ethan~ > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From haoyi.sg at gmail.com Fri Feb 28 00:47:38 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Thu, 27 Feb 2014 15:47:38 -0800 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> Message-ID: > Please admit that this is way, way prettier. If it's just a matter of prettyness, why not just alias lambda as ? Then we can have matrix.transform(? x, y: (y, x)) I think it looks way prettier than either one: it doesn't have the verbosity of the lambda version, and doesn't have the weird contextuality of the = version. On Thu, Feb 27, 2014 at 3:38 PM, Joshua Landau wrote: > On 27 February 2014 04:19, Steven D'Aprano wrote: > > On Wed, Feb 26, 2014 at 11:44:51AM -0800, Andrew Barnert wrote: > > > >> But it becomes more useful if you do anything else: > >> > >> Button("Do it!", on_click() = fire_the_ducks(42)) > >> > >> > >> At first glance, I think this is nice, > > > > At first glance, it looks like you are setting the on_click argument to > > the result of fire_the_ducks(42). This proposed syntax is going to be > > *really easy* for people to misinterpret when they see it in use. And > > not just novices -- I think this will be syntax that just begs to be > > misinterpreted when reading code, and misused when writing it. > > > > I think that having special syntax for anonymous function only inside > > function calls with keyword arguments is a violation of the Zen of > > Python (see the one about special cases) and the Principle Of Least > > Surprise. It's really a bad idea to have syntax for a "shorter lambda" > > that works here: > > > > f(arg=***whatever***) > > > > but not in these: > > > > f(***whatever***) > > [len, zip, map, ***whatever***, some_function] > > result = ***whatever***(arg) > > I don't follow. I know it's different, but we have > > f(*args) but not [*args] > > And we have > > start, *mid, end = [1, 2, 3] but not (lambda start, *mid, end: ...)(1, > 2, 3) > > and we have > > (lambda x: ...)(*[...]) but not foo = *[...] > > ... *wink?* > > It's not silly to think that things are context-sensitive in a > context-sensitive language. Personally this proposal makes sense and I > honestly don't see the confusion that's being stated. It makes things > significantly prettier and more readable, for one. > > Further, for consistency one can define: > > matrix.transform((x, y)=(y, x)) > > as an "anonymous" version, compiling to > > matrix.transform(lambda x, y: (y, x)) > > Please admit that this is way, way prettier. > > PS: One problem. "f( (a) = ... )" is currently valid. So is "f( > (((((a))))) = ... )". Why the hell is this so? "(a) = 2" is valid, but > so is "(a, b) = (2, 3)", whereas "f( (a, b) = (2, 3) )" is not. > > ?http://www.python.org/dev/peps/pep-0448/ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mal at egenix.com Fri Feb 28 00:51:26 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 28 Feb 2014 00:51:26 +0100 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> Message-ID: <530FCF7E.2090400@egenix.com> On 28.02.2014 00:26, ian o wrote: > [... Embed Python 2 in Python 3 ...] > > Thoughts or criticisms? There's a catch here: Python 2 and Python 3 use the same C APIs, so you'd have to separate the two in some way to make both live in the same process. It's not impossible, but it can potentially ruin the idea, since C extensions for both Python versions will have to link the right set of C APIs. What you can do right now is experiment with Pyro to use Python 2 and 3 in two different processes and have Pyro bridge between the two: http://pythonhosted.org/Pyro4/ -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 28 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From joshua at landau.ws Fri Feb 28 01:52:32 2014 From: joshua at landau.ws (Joshua Landau) Date: Fri, 28 Feb 2014 00:52:32 +0000 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> Message-ID: On 27 February 2014 23:47, Haoyi Li wrote: >> Please admit that this is way, way prettier. > > If it's just a matter of prettyness, why not just alias lambda as ? As much as this makes sense, and ignoring the downsides of non-ASCII characters?, I have some comments: I disagree about ? being prettier, much as one writes "myfunc (x, y) = (y, x)" in Haskell instead of "myfunc = \(x, y) -> (y, x)"?. This reminds me that we've had this argument shot down before, so chances are I'm defending a dead horse here. ? You might notice that I do so in my EMails, but only because Alt-Gr (on Linux) makes specific graphemes easy to reach. ? I don't actually know Haskell. From james at dontusethiscode.com Fri Feb 28 01:48:12 2014 From: james at dontusethiscode.com (James Powell) Date: Thu, 27 Feb 2014 19:48:12 -0500 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <530FCF7E.2090400@egenix.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <530FCF7E.2090400@egenix.com> Message-ID: <530FDCCC.6050802@dontusethiscode.com> On 02/27/2014 06:51 PM, M.-A. Lemburg wrote: > On 28.02.2014 00:26, ian o wrote: >> [... Embed Python 2 in Python 3 ...] >> >> Thoughts or criticisms? > There's a catch here: Python 2 and Python 3 use the same C APIs, > so you'd have to separate the two in some way to make both live > in the same process. I actually have a working embedding of Python 3 into Python 2, which I've presented at a few conferences, most recently at PyData London this past weekend. It's just a C-extension module that embeds a Python 3 PyRun_SimpleString. I haven't gotten around to building a shim to interact with Python 3 objects in Python 2 (and this would require a little bit of sophistication to handle GIL, GC, &c. issues.) Still, it's a working example of Python 2 and Python 3 running in the same process. > It's not impossible, but it can potentially ruin the idea, since > C extensions for both Python versions will have to link the right > set of C APIs. I did this by "source filtering" - i.e., rewriting exported symbols to allow linking against both interpreters. This is admittedly a very primitive approach and limits its use in production. I recently put some time into trying to redo this embedding using cffi and worked through a few problems in this approach with one of the cython/numba developers. I haven't gotten anything working yet. I may need to ask Xzibit if he has any suggestions. I don't know if this idea has any serious applications. It's definitely been a fun toy for presenting ideas about interpreter embedding and exploring certain facets of CPython! Cheers, James Powell follow: @dontusethiscode + @nycpython attend: nycpython.org read: seriously.dontusethiscode.com From ziad.sawalha at rackspace.com Fri Feb 28 02:48:25 2014 From: ziad.sawalha at rackspace.com (Ziad Sawalha) Date: Fri, 28 Feb 2014 01:48:25 +0000 Subject: [Python-ideas] PEP-257: drop recommendation for extra line at end of multi-line docstring In-Reply-To: References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com>, Message-ID: <1FD9DEBB-1E68-4BB0-9349-B3FA2598D627@rackspace.com> On Feb 27, 2014, at 4:42 PM, "Bruce Leban" > wrote: To clarify are you asking to delete the recommendation of a blank line or the entire recommendation? That is are you suggesting the recommendation change to ?It is recommended to place the closing quotes on a line by themselves." That's right. I just want to get rid of the recommendation for the blank line. I think deleting it completely is a bad idea as otherwise it's hard to see where the docstring ends and the code begins, especially when using doctest. --- Bruce Learn how hackers think: http://j.mp/gruyere-security https://www.linkedin.com/in/bruceleban On Thu, Feb 27, 2014 at 2:06 PM, Ziad Sawalha > wrote: PEP-257 includes this recommendation: ?The BDFL [3] recommends inserting a blank line between the last paragraph in a multi-line docstring and its closing quotes, placing the closing quotes on a line by themselves. This way, Emacs' fill-paragraph command can be used on it.? I believe emacs no longer has this limitation. "If you do fill-paragraph in emacs in Python mode within a docstring, emacs already ignores the closing triple-quote. In fact, the most recent version of emacs supports several different docstring formatting styles and gives you the ability to switch between them.? - quoting Kevin L. Mitchell who is more familiar with emacs than I am. I?m considering removing that recommendation and updating some of the examples in PEP-257, but I?d like some thoughts from this group before I submit the patch. Any thoughts or references to conversations that may have already been had on this topic? Regards, Ziad _______________________________________________ Python-ideas mailing list Python-ideas at python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ian.team.python at gmail.com Fri Feb 28 03:47:15 2014 From: ian.team.python at gmail.com (Ian) Date: Fri, 28 Feb 2014 13:47:15 +1100 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <530FDCCC.6050802@dontusethiscode.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <530FCF7E.2090400@egenix.com> <530FDCCC.6050802@dontusethiscode.com> Message-ID: <530FF8B3.9060400@gmail.com> On 28/02/2014 11:48 AM, James Powell wrote: > > There's a catch here: Python 2 and Python 3 use the same C APIs, > so you'd have to separate the two in some way to make both live > in the same process. > I actually have a working embedding of Python 3 into Python 2, which > I've presented at a few conferences, most recently at PyData London this > past weekend. > > It's just a C-extension module that embeds a Python 3 > PyRun_SimpleString. I haven't gotten around to building a shim to > interact with Python 3 objects in Python 2 (and this would require a > little bit of sophistication to handle GIL, GC, &c. issues.) > > Still, it's a working example of Python 2 and Python 3 running in the > same process. > I don't know if this idea has any serious applications. It's definitely > been a fun toy for presenting ideas about interpreter embedding and > exploring certain facets of CPython! > > Cheers, > James Powell > James, I would suggest there is a very significant application for for calling python 2 objects from python3, in that suddenly the main roadblock to migration to python 3, the dependency on legacy python2 modules, could be removed as a roadblock. Certainly well worth working through the issues. Ian From rymg19 at gmail.com Fri Feb 28 03:47:54 2014 From: rymg19 at gmail.com (Ryan) Date: Thu, 27 Feb 2014 20:47:54 -0600 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: <7A28CFA8-5360-413B-8B00-F2775863969A@yahoo.com> References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> <1393481773.23494.YahooMailNeo@web181006.mail.ne1.yahoo.com> <530EEA20.1040805@canterbury.ac.nz> <20140227211400.GC28804@ando> <7A28CFA8-5360-413B-8B00-F2775863969A@yahoo.com> Message-ID: Last option's way more likely. I made that mistake about 2000 times moving over from C++. Andrew Barnert wrote: >On Feb 27, 2014, at 13:24, Chris Angelico wrote: > >> On Fri, Feb 28, 2014 at 8:14 AM, Steven D'Aprano > wrote: >>>> class A: ... >>>> a = A() >>>> l = [0] >>>> >>>> for a.x in [1, 2, 3]: ... >>>> for l[0] in [1, 2, 3]: ... >>> >>> I wonder whether that is deliberate feature, or an accident of the >way >>> the syntax works. It does seem to be pretty useless though -- why >are >>> you using an attribute or list as a loop variable? >> >> I would say it's no accident that the for loop can accept any lvalue. >> That's how we can do this, which I'm sure you'll agree is *extremely* >> useful: >> >> for x,y in [(1,2), (3,4), (5,6)]: ... >> >> Assigning to the tuple works beautifully there (especially with >> enumerate or zip). Being able to assign to other complex targets is a >> bonus that you'll probably never come across (I can imagine someone >> might have a use for "for self.blah in ...", but can't concoct any >> right now) > >Something I've seen in real code (not _good_ code, but actually >deployed): > > for self.index in range(len(self.values)): > if self.values[self.index] == spam: > break > else: > self.index = None > >Apparently someone didn't want to catch the exception from list.index. >(Or, more likely, they were trying to write C code in Python.) >_______________________________________________ >Python-ideas mailing list >Python-ideas at python.org >https://mail.python.org/mailman/listinfo/python-ideas >Code of Conduct: http://python.org/psf/codeofconduct/ -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Fri Feb 28 04:33:15 2014 From: guido at python.org (Guido van Rossum) Date: Thu, 27 Feb 2014 19:33:15 -0800 Subject: [Python-ideas] PEP-257: drop recommendation for extra line at end of multi-line docstring In-Reply-To: <1FD9DEBB-1E68-4BB0-9349-B3FA2598D627@rackspace.com> References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com> <1FD9DEBB-1E68-4BB0-9349-B3FA2598D627@rackspace.com> Message-ID: How about this patch? http://codereview.appspot.com/69870043 On Thu, Feb 27, 2014 at 5:48 PM, Ziad Sawalha wrote: > > On Feb 27, 2014, at 4:42 PM, "Bruce Leban" wrote: > > To clarify are you asking to delete the recommendation of a blank line > or the entire recommendation? > > That is are you suggesting the recommendation change to > > "It is recommended to place the closing quotes on a line by themselves." > > > That's right. I just want to get rid of the recommendation for the blank > line. > > I think deleting it completely is a bad idea as otherwise it's hard to > see where the docstring ends and the code begins, especially when using > doctest. > > --- Bruce > Learn how hackers think: http://j.mp/gruyere-security > https://www.linkedin.com/in/bruceleban > > > > On Thu, Feb 27, 2014 at 2:06 PM, Ziad Sawalha wrote: > >> PEP-257 includes this recommendation: >> >> "The BDFL [3] recommends inserting a blank line between the last >> paragraph in a multi-line >> docstring and its closing quotes, placing the closing quotes on a line by >> themselves. This way, >> Emacs' fill-paragraph command can be used on it." >> >> I believe emacs no longer has this limitation. "If you do fill-paragraph >> in emacs in Python mode >> within a docstring, emacs already ignores the closing triple-quote. In >> fact, the most recent version >> of emacs supports several different docstring formatting styles and gives >> you the ability to switch >> between them." - quoting Kevin L. Mitchell who is more familiar with >> emacs than I am. >> >> I'm considering removing that recommendation and updating some of the >> examples in PEP-257, >> but I'd like some thoughts from this group before I submit the patch. Any >> thoughts or references to >> conversations that may have already been had on this topic? >> >> Regards, >> Ziad >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Fri Feb 28 04:35:11 2014 From: greg.ewing at canterbury.ac.nz (Greg) Date: Fri, 28 Feb 2014 16:35:11 +1300 Subject: [Python-ideas] Syntax for passing lambdas to functions In-Reply-To: References: <530D327D.6090804@canterbury.ac.nz> <1393443891.47275.YahooMailNeo@web181005.mail.ne1.yahoo.com> <20140227041958.GA1383@ando> Message-ID: <531003EF.1060100@canterbury.ac.nz> On 28/02/2014 12:38 p.m., Joshua Landau wrote: > Further, for consistency one can define: > > matrix.transform((x, y)=(y, x)) > Please admit that this is way, way prettier. Hmm, I'm not so sure. That looks more like it should mean matrix.transform(x = y, y = x) i.e. an unpacking assignment to keyword args. -- Greg From rosuav at gmail.com Fri Feb 28 05:40:11 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 28 Feb 2014 15:40:11 +1100 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> Message-ID: On Fri, Feb 28, 2014 at 10:26 AM, ian o wrote: > Ptyhon can call C. C can call python. So python3 call C which calls python2. That's all very well if all you want to do is execute code. But as soon as you want to transfer data across, you have to do a whole lot of careful rummaging. First off, you'll need to have some C-level module that imports a Py2 module, identifies all its exported symbols (technically all its symbols, but you could save some effort by just looking at __all__ and most people won't see the difference), and creates a buffer layer between that and Py3: whenever someone calls up one of those symbols, create the translation and send it along. You can't have the two interpreters sharing GC state. But second, you have to deal with the fundamental data type differences. It may seem obvious enough - a function call goes through C and eventually becomes a function call, and so on - but the two interpreters fundamentally differ. Start with the easy one: int versus long. Py2 has two integer types; Py3 dropped int and renamed long to int. You need to traverse that gap. And then the big one. The really really big one. Bytes versus Unicode. How are you going to share strings across the boundary? Will all Py2's quoted strings come out as bytes in Py3? Will you encode and decode across the boundary (and if so, what encoding?)? It's fundamentally *hard* to do this. And there's another problem, too - a philosophical one. Let's suppose you put in all this work to make spamify.py available under Python 3. Your application, which needs spamify, can now migrate; but where's the impetus for spamify itself to start supporting Py3? Now that this shim is available, they can just carry on forever, right? So as soon as you create this, you effectively doom yourself to indefinite support, which is a *lot* of work. A new version of spamify might add a new feature, which you then need to support. Or it might start using a different encoding for its strings, and suddenly breaking everything. You have to keep up with all of that, *and* you're encouraging libraries to stay on Py2. In effect, you're encouraging applications to move to Py3, but encouraging libraries to stay on Py2. Personally, I'd rather people lean on their library creators to start supporting Py3. In the long run, it's going to be less work anyway. The only person who truly knows how a module should translate Py2 strings into Py3 strings is that module's author. ChrisA From ziad.sawalha at rackspace.com Fri Feb 28 05:58:35 2014 From: ziad.sawalha at rackspace.com (Ziad Sawalha) Date: Fri, 28 Feb 2014 04:58:35 +0000 Subject: [Python-ideas] PEP-257: drop recommendation for extra line at end of multi-line docstring In-Reply-To: References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com> <1FD9DEBB-1E68-4BB0-9349-B3FA2598D627@rackspace.com>, Message-ID: Approved :-) On Feb 27, 2014, at 9:33 PM, "Guido van Rossum" > wrote: How about this patch? http://codereview.appspot.com/69870043 On Thu, Feb 27, 2014 at 5:48 PM, Ziad Sawalha > wrote: On Feb 27, 2014, at 4:42 PM, "Bruce Leban" > wrote: To clarify are you asking to delete the recommendation of a blank line or the entire recommendation? That is are you suggesting the recommendation change to ?It is recommended to place the closing quotes on a line by themselves." That's right. I just want to get rid of the recommendation for the blank line. I think deleting it completely is a bad idea as otherwise it's hard to see where the docstring ends and the code begins, especially when using doctest. --- Bruce Learn how hackers think: http://j.mp/gruyere-security https://www.linkedin.com/in/bruceleban On Thu, Feb 27, 2014 at 2:06 PM, Ziad Sawalha > wrote: PEP-257 includes this recommendation: ?The BDFL [3] recommends inserting a blank line between the last paragraph in a multi-line docstring and its closing quotes, placing the closing quotes on a line by themselves. This way, Emacs' fill-paragraph command can be used on it.? I believe emacs no longer has this limitation. "If you do fill-paragraph in emacs in Python mode within a docstring, emacs already ignores the closing triple-quote. In fact, the most recent version of emacs supports several different docstring formatting styles and gives you the ability to switch between them.? - quoting Kevin L. Mitchell who is more familiar with emacs than I am. I?m considering removing that recommendation and updating some of the examples in PEP-257, but I?d like some thoughts from this group before I submit the patch. Any thoughts or references to conversations that may have already been had on this topic? Regards, Ziad _______________________________________________ Python-ideas mailing list Python-ideas at python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ _______________________________________________ Python-ideas mailing list Python-ideas at python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ian.team.python at gmail.com Fri Feb 28 07:03:42 2014 From: ian.team.python at gmail.com (ian o) Date: Thu, 27 Feb 2014 22:03:42 -0800 (PST) Subject: [Python-ideas] A python bridge between versions In-Reply-To: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> Message-ID: <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> Chris, great reply. Yes it is not easy. Especially if you wish to make use of python2 modules painless. And I agree wholeheartedly that serious thought is needed on the 'could it work so well that it actually slow the migration of library modules?'. This is an important debate to have. I suggest if the answer to 'can a specification be created that will significantly accelerate migration?', then it is worth the effort to deliver this. But could it accelerate migration to python 3? Currently, the python.org website advice on : 'Python2 or Python3 - Which version should I use?' Lists two reasons for using python 2. This 'shim' could eliminate that second very significant reason. Then the advice would be 'if you have a choice of version, use python 3. And I suggest the topic of 'how do I use legacy python2 library modules?', would start by recommending if at all possible, find a python 3 replacement. As things stand, the popularity of Python has dropped by most measures since the release of Python 3. Improve the language and lose market share? I suggest that is all about the pain of the transition. You do not have to search hard to find comments to the effect 'nobody is using python3'. Or 'even if you use python 3, do not use the new features since you need to ensure code is compatible with both versions'. This is worth careful thought, and again I suggest part of the answer lies in the specification. If the 'do it all' approach to a 'shim' is taken, and using python2 modules is completely painless, no one will ever bother replacing them. Make it too hard to use python2 from python3 and migration is slowed again people do not bother to port libraries because not enough users have yet migrated. On python.org survey (2013-2014) , 60% of respondents report that dependencies are keeping them still working on python 2 and 80% are doing more code in python 2. For most people right now... all the changes and improvements made in python 3 are simply not available and the language is frozen. If something like this is done, without making it so painless that there is no incentive to replace or convert dependencies in a new world where the main reason for staying in python 2 has been basically eliminated, then this could be huge benefit. Just being able to change the python.org web site to say 'use python 3 unless you have no freedom of what version runs on your environment' would be huge! . -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Fri Feb 28 09:09:58 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 28 Feb 2014 17:09:58 +0900 Subject: [Python-ideas] stdlib process GSoC 2014 ideas In-Reply-To: References: <530FB509.9050501@stoneleaf.us> Message-ID: <87r46nslnt.fsf@uwakimon.sk.tsukuba.ac.jp> Nick Coghlan writes: > Moderators: please don't let such posts out of the moderation > queue, they're a complete waste of everyone's time. +1 They're 50% FUD, and 50% just plain wrong. More to the point, they're 100% off-topic once they go beyond "I won't sign the CA so my code is not useful to further the purposes of python-ideas." (I'm happy to discuss both Anatoly's statements and my own -- philosophical BS is a hobby of mine -- but *not on this list*. Reply-to set to , please observe.) From rosuav at gmail.com Fri Feb 28 09:30:12 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 28 Feb 2014 19:30:12 +1100 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> Message-ID: On Fri, Feb 28, 2014 at 5:03 PM, ian o wrote: > Yes it is not easy. Especially if you wish to make use of python2 modules > painless. It's fundamentally not going to be painless. You can either try to move the pain into the thunking module, or keep it in the application, but using a Py2 module is not painless. And I'm not sure it's even possible to move all the pain into the thunker - most Py2 code simply doesn't concern itself with the difference between text and bytes, so there's no automated way to create that difference. (And if the module already does care, then chances are it's easy to port it and officially support Py3, which will be far FAR easier than using the shim - way higher performance too.) > And I agree wholeheartedly that serious thought is needed on the 'could it > work so well that it actually slow the migration of library modules?'. > > This is an important debate to have. I suggest if the answer to 'can a > specification be created that will significantly accelerate migration?', > then it is worth the effort to deliver this. > > But could it accelerate migration to python 3? At best, it'll accelerate migration of applications, while slowing migration of libraries. > Currently, the python.org website advice on : 'Python2 or Python3 - Which > version should I use?' > Lists two reasons for using python 2. > This 'shim' could eliminate that second very significant reason. Then the > advice would be 'if you have a choice of version, use python 3. > And I suggest the topic of 'how do I use legacy python2 library modules?', > would start by recommending if at all possible, find a python 3 replacement. Yeah. And if you can't find a Py3 replacement, it might even be worth helping the library to support Py3. If running it through 2to3 does the job, maybe that's all you need to do! > As things stand, the popularity of Python has dropped by most measures since > the release of Python 3. Improve the language and lose market share? I > suggest that is all about the pain of the transition. What measures? > You do not have to search hard to find comments to the effect 'nobody is > using python3'. Or 'even if you use python 3, do not use the new features > since you need to ensure code is compatible with both versions'. > > This is worth careful thought, and again I suggest part of the answer lies > in the specification. If the 'do it all' approach to a 'shim' is taken, and > using python2 modules is completely painless, no one will ever bother > replacing them. Make it too hard to use python2 from python3 and migration > is slowed again people do not bother to port libraries because not enough > users have yet migrated. The problem is that there'll still be pain for the application, and there still won't be any for the library. The pain won't move at all. The impetus for a library to support Py3 comes from application developers saying "We need this to support Py3" (and, quite possibly, putting in the work to make it support Py3). Having a fiddly solution will encourage you to make the fiddly one work better, rather than seek the true one. > On python.org survey (2013-2014) , 60% of respondents report that > dependencies are keeping them still working on python 2 and 80% are doing > more code in python 2. It's worth noting that that survey asked how many people were using Py2, not about how many weren't using Py3. Lots of people are using both. What that really means, then, is that 20% of those people are adding absolutely no new Py2 code. Quite a lot of those 80% are probably writing 2/3 compatible code. > For most people right now... all the changes and improvements made in python > 3 are simply not available and the language is frozen. Which means the "temptation gap" is getting progressively wider. The temptation to move from 2.4 to 2.6 is "hey look, you get the with statement", the temptation to move from 3.3 to 3.4 is "oh look, enumerations", and so on. The temptation to move to the current version is the sum of all the temptations from your current version onward. Sooner or later, that's going to be a sufficiently-strong incentive that people will just make the jump. > If something like this is done, without making it so painless that there is > no incentive to replace or convert dependencies in a new world where the > main reason for staying in python 2 has been basically eliminated, then this > could be huge benefit. > > Just being able to change the python.org web site to say 'use python 3 > unless you have no freedom of what version runs on your environment' would > be huge! That's never really going to happen, partly because it's not very helpful :) Unless you're running on a locked-down server that has Py2 installed and no Py3, the problem isn't "no freedom what runs" but "the version suite I want doesn't exist". So the solution is to make what you want exist. In the open source world, there are two ways to do that: lean on the developers, or do the work yourself. (In the closed source world, you lose the second option, but on the flip side, it's that much more likely that your "lean" carries the weight of money.) A number of library/framework maintainers have said that the reason for not supporting Py3 is "lack of interest", which means they quite probably *would* support it if someone expresses enough interest to do the work of porting (or at least help with it); the more applications developers who say "I'd use Py3 except that libspamify.py needs Py2", the more likely that libspamify will sprout Py3 wings. Binaries of the latest Python 3 are available for most of the popular OSes. I don't keep track of Macs, but on Windows, it's easy to just grab a 3.3 (or, soon, a 3.4), and most Linux distros are carrying at least some form of 3.x (Debian's current stable, Wheezy, ships with 3.2, but Jessie ships with 3.3; RHEL 6 seems to have 3.3, but without having a copy, I can't check authoritatively). So it really is just a matter of library support. ChrisA From mal at egenix.com Fri Feb 28 10:16:34 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 28 Feb 2014 10:16:34 +0100 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <530FDCCC.6050802@dontusethiscode.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <530FCF7E.2090400@egenix.com> <530FDCCC.6050802@dontusethiscode.com> Message-ID: <531053F2.9070109@egenix.com> On 28.02.2014 01:48, James Powell wrote: > On 02/27/2014 06:51 PM, M.-A. Lemburg wrote: >> On 28.02.2014 00:26, ian o wrote: >>> [... Embed Python 2 in Python 3 ...] >>> >>> Thoughts or criticisms? > >> There's a catch here: Python 2 and Python 3 use the same C APIs, >> so you'd have to separate the two in some way to make both live >> in the same process. > > I actually have a working embedding of Python 3 into Python 2, which > I've presented at a few conferences, most recently at PyData London this > past weekend. > > It's just a C-extension module that embeds a Python 3 > PyRun_SimpleString. I haven't gotten around to building a shim to > interact with Python 3 objects in Python 2 (and this would require a > little bit of sophistication to handle GIL, GC, &c. issues.) > > Still, it's a working example of Python 2 and Python 3 running in the > same process. Interesting :-) Do you have some pointers to slides or videos ? I'm been thinking of doing something like this but the other way around - embed Python2 in Python3. When starting to brainstorm that idea, I quickly ended up postponing the idea again due to problems with C extension linking, dual interpreter environments, object interfacing between the two worlds, having two separate exception class trees, two sets of basic types/classes, two GILs, etc. The linking part was the most important to me, since being able to use Python 2 extensions would be my main reason to stick with Python 2 for some more time. >> It's not impossible, but it can potentially ruin the idea, since >> C extensions for both Python versions will have to link the right >> set of C APIs. > > I did this by "source filtering" - i.e., rewriting exported symbols to > allow linking against both interpreters. This is admittedly a very > primitive approach and limits its use in production. > > I recently put some time into trying to redo this embedding using cffi > and worked through a few problems in this approach with one of the > cython/numba developers. I haven't gotten anything working yet. I may > need to ask Xzibit if he has any suggestions. > > I don't know if this idea has any serious applications. It's definitely > been a fun toy for presenting ideas about interpreter embedding and > exploring certain facets of CPython! > > Cheers, > James Powell > > follow: @dontusethiscode + @nycpython > attend: nycpython.org > read: seriously.dontusethiscode.com Cheers, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 28 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From g.brandl at gmx.net Fri Feb 28 11:29:06 2014 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 28 Feb 2014 11:29:06 +0100 Subject: [Python-ideas] stdlib process GSoC 2014 ideas In-Reply-To: References: Message-ID: Am 27.02.2014 22:40, schrieb anatoly techtonik: > On Fri, Feb 28, 2014 at 12:00 AM, Tim Delaney > wrote: >> On 28 February 2014 07:32, anatoly techtonik wrote: >>> >>> >>> https://bitbucket.org/techtonik/python-stdlib >>> >>> This needs to be extended and integrated into stdlib development process. >> >> Wow Anatoly. Are you finally going to sign the PSF Contributor Agreement? > > This idea is for GSoC students. python-stdlib is free from the license burden > with UNLICENSE, so you better ask PSF why they don't accept it. > > > > Offtopic, but since it is about GSoC, and there are people studying copyright > and open source, I'll explain my position just in case somebody will be able to > help with that in the future. > > I don't sign CLA, because: > > 1. I don't want to release my code under restrictive PSF license > https://tldrlegal.com/license/python-license-2.0 You don't, the PSF does. > 2. PSF doesn't comply with terms of Apache 2.0 license (include license) > which is probably chosen by at least one contributor for CLA > https://tldrlegal.com/license/apache-license-2.0-(apache-2.0) This is not true. > 3. I don't want to give PSF exclusive rights for all code and documentation > to be released under any other "open source" license. PSF may be bought > by some corporation and they will have the right to impose their own > "open source license text" on it. (yes, I don't trust people at all). This is a valid concern, however for code with your "UNLICENSE" any corporation can do anything with it right now anyway. > 4. If I sign, I will be less motivated to open the Python docs under CC-BY > license with examples that can be copy-pasted without requiring PSF > license in your project. Right now using logging examples without it is > illegal. This *might* be a valid concern (IANAL), but it's one I've never heard from you so far. Why don't you start discussing this one explicitly? > 5. Everything is owned by PSF is wrong. Well, the PSF owns a lot of money, and I don't think money is wrong. Jokes aside, the PSF explicitly *doesn't* own your copyright due to the CLA. > Python is a community project, > and core code should be shared as open as possible (with credits where > due). Public domain with optional crediting and patent grant is ideal. > Trademarks are not affected. Nobody is forced and can do what they > want. And current licensing uncertainty no good for collaboration. > 6. I want people to have free entry for participating in open source projects, > meaning that the patent grant and agreement to release their contribution > under the open source license that project uses, should work by default > without any CLAs. These are again valid concerns from your side, but you will have to understand that the PSF does not have the freedom to abolish the CLA. Georg From rosuav at gmail.com Fri Feb 28 13:24:50 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 28 Feb 2014 23:24:50 +1100 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <531053F2.9070109@egenix.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <530FCF7E.2090400@egenix.com> <530FDCCC.6050802@dontusethiscode.com> <531053F2.9070109@egenix.com> Message-ID: On Fri, Feb 28, 2014 at 8:16 PM, M.-A. Lemburg wrote: > I'm been thinking of doing something like this but the other way > around - embed Python2 in Python3. When starting to brainstorm > that idea, I quickly ended up postponing the idea again due to > problems with C extension linking, dual interpreter environments, > object interfacing between the two worlds, having two separate > exception class trees, two sets of basic types/classes, two GILs, > etc. The another common solution is to make the two halves completely language-independent. Separate two processes with a pipe, socket, etc, and communicate with streams of bytes moving back and forth. I'd consider that to be the zero-mark for any blending solution: it's simple, it's straight-forward, it's guaranteed to work, so you have to make something that's better than that - faster, easier to use, cleaner, whatever. That's the mark to beat. ChrisA From ncoghlan at gmail.com Fri Feb 28 14:18:31 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 28 Feb 2014 23:18:31 +1000 Subject: [Python-ideas] A python bridge between versions In-Reply-To: References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> Message-ID: On 28 February 2014 18:30, Chris Angelico wrote: > On Fri, Feb 28, 2014 at 5:03 PM, ian o wrote: >> Yes it is not easy. Especially if you wish to make use of python2 modules >> painless. > > It's fundamentally not going to be painless. You can either try to > move the pain into the thunking module, or keep it in the application, > but using a Py2 module is not painless. And I'm not sure it's even > possible to move all the pain into the thunker - most Py2 code simply > doesn't concern itself with the difference between text and bytes, so > there's no automated way to create that difference. (And if the module > already does care, then chances are it's easy to port it and > officially support Py3, which will be far FAR easier than using the > shim - way higher performance too.) > >> And I agree wholeheartedly that serious thought is needed on the 'could it >> work so well that it actually slow the migration of library modules?'. >> >> This is an important debate to have. I suggest if the answer to 'can a >> specification be created that will significantly accelerate migration?', >> then it is worth the effort to deliver this. >> >> But could it accelerate migration to python 3? > > At best, it'll accelerate migration of applications, while slowing > migration of libraries. And we don't actually mind all that much if applications don't migrate in the near term - Python 2 will have commercial support available well past 2020. (The developers of *those applications* might mind, though, just as anyone maintaining Python 2.4 compatibility for the benefit of RHEL/CentOS 5 typically isn't happy about it) By far the path of least resistance if developers would like to write Python 3 code, but are relying on some legacy Python 2 modules, is to instead write "Python 3 like" code in Python 2. Most of the interesting new Python 3 standard library modules (even asyncio) have Python 2 backports available on PyPI, and python-future (http://python-future.org/index.html) provides backports of the core data types that otherwise don't have Python 2 counterparts. This isn't *as* nice as actually running under Python 3 (you miss out on exception chaining for one thing, as well as the improvements to error messages in various parts of the standard library, and the infrastructure work that has gone into improving the core interpreter), but it's still a pretty nice environment to work in (heck, Python 2.*6* is still a pretty nice language to work in, although it definitely relies on more external library support than 3.x, or even 2.7). The group we *do* really care about supporting is authors of existing Python *libraries*, and "2in3" and "3in2" approaches don't really help them at all, since library and framework authors with users on both Python 2 and Python 3 will likely face demand to support both versions natively anyway. That's where various Python 3 changes like restoring Unicode string prefixes, restoring the binary transform codecs, and (for 3.5 in 2015) likely restoring binary interpolation support come in - by making the common subset of Python 2 and Python 3 larger, we make it easier for library authors to support Python 3 without breaking things for their existing Python 2 users. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ron3200 at gmail.com Fri Feb 28 18:17:18 2014 From: ron3200 at gmail.com (Ron Adam) Date: Fri, 28 Feb 2014 11:17:18 -0600 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. Message-ID: Starting new thread because this bike has a different shape and color. Yesterday I was thinking that just making the keyword lambda assignable like True, False, and None, would be enough. But the issue with that is lambda isn't a name to an actual object or type. That was the seed for this idea. How to get lambda like functionality into some sort of object that would be easy to use and explain. This morning I thought we could have in a functions definition something, like "*", and "**", to take an expression. Similar to Nicks idea with =:, but more general. The idea is to have "***" used in def mean to take "any" call expression and not evaluate it until *** is used on it. ie... the same rules as *. When used in a def to pack a tuple, and when used outside def, to unpack it. So, "***" used in a def, stores the call expression, at call time, and when used later, expresses it. A function call that captures an expression may be tricky to do. Here's one approach that requires sugar when a function defined with "***" is called. class TriStar: def __init__(self, expr): """ expr is a callable that takes no arguments. """ self.expr = expr def __tristar__(self): """ ***obj --> result """ return self.expr() def fn(***expr):... (Any other suggestions for how to do this would be good.) And at call time.... fn(...) --> fn(TriStar(expr=lambda:...)) So presuming we can do something like the above, the first case is ... def star_fn(***expr) return ***expr ... = star_fn(...) Which is a function that just returns whatever it's input is, and is even more general than using *args, **kwds. The call signature stored in expr isn't evaluated until it's returned with ***expr. So the evaluation is delayed, or lazy, but it's still explicit and very easy to read. This returns a lambda-like function. def star_lambda(***expr): return expr And is used this way... result = star_lambda(a * b + c) # captures expression. actual_result = ***result # *** resolves "result" here! The resolution is done with ***name, rather than name(). That's actually very good because it can pass through callable tests. So you can safely pass callable objects around without them getting called at the wrong time or place. We can shorten the name because star_lambda is just a function. L = star_lambda To me this is an exceptionally clean solution. Easy to use, and not to hard to explain. Seems a lot more like a python solution to me as well. Hoping it doesn't get shot down too quickly, Ron ;-) From guido at python.org Fri Feb 28 18:53:45 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 28 Feb 2014 09:53:45 -0800 Subject: [Python-ideas] PEP-257: drop recommendation for extra line at end of multi-line docstring In-Reply-To: References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com> <1FD9DEBB-1E68-4BB0-9349-B3FA2598D627@rackspace.com> Message-ID: Updated! On Thu, Feb 27, 2014 at 8:58 PM, Ziad Sawalha wrote: > Approved :-) > > > On Feb 27, 2014, at 9:33 PM, "Guido van Rossum" wrote: > > How about this patch? http://codereview.appspot.com/69870043 > > > On Thu, Feb 27, 2014 at 5:48 PM, Ziad Sawalha wrote: > >> >> On Feb 27, 2014, at 4:42 PM, "Bruce Leban" wrote: >> >> To clarify are you asking to delete the recommendation of a blank line >> or the entire recommendation? >> >> That is are you suggesting the recommendation change to >> >> "It is recommended to place the closing quotes on a line by themselves." >> >> >> That's right. I just want to get rid of the recommendation for the >> blank line. >> >> I think deleting it completely is a bad idea as otherwise it's hard to >> see where the docstring ends and the code begins, especially when using >> doctest. >> >> --- Bruce >> Learn how hackers think: http://j.mp/gruyere-security >> https://www.linkedin.com/in/bruceleban >> >> >> >> On Thu, Feb 27, 2014 at 2:06 PM, Ziad Sawalha > > wrote: >> >>> PEP-257 includes this recommendation: >>> >>> "The BDFL [3] recommends inserting a blank line between the last >>> paragraph in a multi-line >>> docstring and its closing quotes, placing the closing quotes on a line >>> by themselves. This way, >>> Emacs' fill-paragraph command can be used on it." >>> >>> I believe emacs no longer has this limitation. "If you do fill-paragraph >>> in emacs in Python mode >>> within a docstring, emacs already ignores the closing triple-quote. In >>> fact, the most recent version >>> of emacs supports several different docstring formatting styles and >>> gives you the ability to switch >>> between them." - quoting Kevin L. Mitchell who is more familiar with >>> emacs than I am. >>> >>> I'm considering removing that recommendation and updating some of the >>> examples in PEP-257, >>> but I'd like some thoughts from this group before I submit the patch. >>> Any thoughts or references to >>> conversations that may have already been had on this topic? >>> >>> Regards, >>> Ziad >>> _______________________________________________ >>> Python-ideas mailing list >>> Python-ideas at python.org >>> https://mail.python.org/mailman/listinfo/python-ideas >>> Code of Conduct: http://python.org/psf/codeofconduct/ >>> >> >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > > -- > --Guido van Rossum (python.org/~guido) > > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Fri Feb 28 18:54:15 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 1 Mar 2014 04:54:15 +1100 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: References: Message-ID: On Sat, Mar 1, 2014 at 4:17 AM, Ron Adam wrote: > This returns a lambda-like function. > > def star_lambda(***expr): return expr > > > And is used this way... > > result = star_lambda(a * b + c) # captures expression. > > actual_result = ***result # *** resolves "result" here! > Interesting, but I don't like the way the interpretation of a function call depends on the target function. With both * and ** notations, there's absolutely no difference: the function is called with these positional and those keyword arguments, whether they came from actual args or from * or ** unpack/repacks; and there's no difference between a function that collects args with *args,**kwargs and one that collects them with individual names (or a C-level function that might do something altogether different). With this proposal, your star_lambda function's declaration changes the call site - instead of evaluating a*b+c, it has to construct an anonymous function and pass it along. ChrisA From ethan at stoneleaf.us Fri Feb 28 19:02:07 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 28 Feb 2014 10:02:07 -0800 Subject: [Python-ideas] PEP-257: drop recommendation for extra line at end of multi-line docstring In-Reply-To: References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com> <1FD9DEBB-1E68-4BB0-9349-B3FA2598D627@rackspace.com> Message-ID: <5310CF1F.1000603@stoneleaf.us> And there was great rejoicing across the land! On 02/28/2014 09:53 AM, Guido van Rossum wrote: > Updated! > > > On Thu, Feb 27, 2014 at 8:58 PM, Ziad Sawalha > wrote: > > Approved :-) > > > On Feb 27, 2014, at 9:33 PM, "Guido van Rossum" > wrote: > >> How about this patch? http://codereview.appspot.com/69870043 >> >> >> On Thu, Feb 27, 2014 at 5:48 PM, Ziad Sawalha > wrote: >> >> >> On Feb 27, 2014, at 4:42 PM, "Bruce Leban" > wrote: >> >>> To clarify are you asking to delete the recommendation of a blank line or the entire recommendation? >>> >>> That is are you suggesting the recommendation change to >>> >>> ?It is recommended to place the closing quotes on a line by themselves." >>> >> >> That's right. I just want to get rid of the recommendation for the blank line. >> >>> I think deleting it completely is a bad idea as otherwise it's hard to see where the docstring ends and the >>> code begins, especially when using doctest. >>> >>> --- Bruce >>> Learn how hackers think: http://j.mp/gruyere-security >>> https://www.linkedin.com/in/bruceleban >>> >>> >>> >>> On Thu, Feb 27, 2014 at 2:06 PM, Ziad Sawalha >> > wrote: >>> >>> PEP-257 includes this recommendation: >>> >>> ?The BDFL [3] recommends inserting a blank line between the last paragraph in a multi-line >>> docstring and its closing quotes, placing the closing quotes on a line by themselves. This way, >>> Emacs' fill-paragraph command can be used on it.? >>> >>> I believe emacs no longer has this limitation. "If you do fill-paragraph in emacs in Python mode >>> within a docstring, emacs already ignores the closing triple-quote. In fact, the most recent version >>> of emacs supports several different docstring formatting styles and gives you the ability to switch >>> between them.? - quoting Kevin L. Mitchell who is more familiar with emacs than I am. >>> >>> I?m considering removing that recommendation and updating some of the examples in PEP-257, >>> but I?d like some thoughts from this group before I submit the patch. Any thoughts or references to >>> conversations that may have already been had on this topic? >>> >>> Regards, >>> Ziad > From ron3200 at gmail.com Fri Feb 28 20:42:57 2014 From: ron3200 at gmail.com (Ron Adam) Date: Fri, 28 Feb 2014 13:42:57 -0600 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: References: Message-ID: On 02/28/2014 11:54 AM, Chris Angelico wrote: > On Sat, Mar 1, 2014 at 4:17 AM, Ron Adam wrote: >> >This returns a lambda-like function. >> > >> > def star_lambda(***expr): return expr >> > >> > >> >And is used this way... >> > >> > result = star_lambda(a * b + c) # captures expression. >> > >> > actual_result = ***result # *** resolves "result" here! >> > > Interesting, but I don't like the way the interpretation of a function > call depends on the target function. With both * and ** notations, > there's absolutely no difference: the function is called with these > positional and those keyword arguments, whether they came from actual > args or from * or ** unpack/repacks;and there's no difference between > a function that collects args with *args,**kwargs and one that > collects them with individual names (or a C-level function that might > do something altogether different). It's not clear what differences you mean here... can you show some examples? I think we just are used to not thinking about it, But it's not really that different. def fn(*args, **kwds): ... This wraps args in a list, and kwds in a dict. It's up to the *function called* to do what is intended by the syntax. def fn(*args): ... fn(a, b, c) --> fn(list(a, b, c)) #depends on function called. def fn(**kwds): ... fn(a=1, b=2, c=3) --> fn(dict(a=1, b=2, c=3)) # here too. def fn(***expr): ... fn(expr) --> fn(TriStar(lambda:(expr))) # A bit more complex, but also the same. # Parentheses need to capture tuple packing # due to ',' having a higher precidence. The mechanism behind each of these may be somewhat different, but there are also similarities. def fn(***expr): return ***expr With these, it forwards the these existing cases nicely. a, b, c = fn(a, b, c) args = fn(*args)args, kwds kwds = fn(**kwds) args, kwds = fn(*args, **kwds) And just like '**' can't be used to pack a dictionary directly, we can't use '***' to pack an expression directly. Using "**" in a funciton unpacks the dictionary. Using "***" in a function call expresses the TriStar object. (* any name for the TriStar object would work. (small detail)) NOW here is the main limitation... :-/ a, b, c = fn(a, b, c=1) Which is because (a, b, c=1) isn't a valid expression outside of a function call. Or should this be captured as (a, b, {"c":1})? Sigh... darn edge cases. A bit more than an edge case I think. Any ideas? Cheers, Ron > With this proposal, your > star_lambda function's declaration changes the call site - instead of > evaluating a*b+c, it has to construct an anonymous function and pass > it along. From ncoghlan at gmail.com Fri Feb 28 23:33:42 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 1 Mar 2014 08:33:42 +1000 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: References: Message-ID: On 1 Mar 2014 05:43, "Ron Adam" wrote: > > > > On 02/28/2014 11:54 AM, Chris Angelico wrote: >> >> On Sat, Mar 1, 2014 at 4:17 AM, Ron Adam wrote: >>> >>> >This returns a lambda-like function. >>> > >>> > def star_lambda(***expr): return expr >>> > >>> > >>> >And is used this way... >>> > >>> > result = star_lambda(a * b + c) # captures expression. >>> > >>> > actual_result = ***result # *** resolves "result" here! >>> > >> >> Interesting, but I don't like the way the interpretation of a function >> call depends on the target function. With both * and ** notations, >> there's absolutely no difference: the function is called with these >> positional and those keyword arguments, whether they came from actual >> args or from * or ** unpack/repacks;and there's no difference between > > > a function that collects args with *args,**kwargs and one that > > collects them with individual names (or a C-level function that might > > do something altogether different). > > > It's not clear what differences you mean here... can you show some examples? Remember that at compile time, Python has *no idea* what the actual signature of the target function is. Thus, all Python function calls use the following sequence (ignoring optimisations of special cases): 1. At the call site, the arguments are collected into a tuple of positional arguments and a dict of keyword arguments. 2. The interpreter hands that tuple and dict over to the target callable 3. The *target callable* then maps the supplied arguments to the defined parameters including filling in any default values. Any function related proposals need to account for the fact that from the compiler's point of view *every* function signature looks like "(*args, **kwds)" (although it may have optimised paths for the no-args case and the positional-args-only case), and that the target callable may not even be written in Python. Cheers, Nick. -------------- next part -------------- An HTML attachment was scrubbed... URL: