From python at mrabarnett.plus.com Sun Apr 1 21:49:58 2018 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 2 Apr 2018 02:49:58 +0100 Subject: [Python-Dev] IDLE colorizer Message-ID: A thread on python-ideas is talking about the prefixes of string literals, and the regex used in IDLE. Line 25 of Lib\idlelib\colorizer.py is: stringprefix = r"(?i:\br|u|f|fr|rf|b|br|rb)?" which looks slightly wrong to me. The \b will apply only to the first choice. Shouldn't it be more like: stringprefix = r"(?:\b(?i:r|u|f|fr|rf|b|br|rb))?" ? From tim.peters at gmail.com Sun Apr 1 22:20:42 2018 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 1 Apr 2018 21:20:42 -0500 Subject: [Python-Dev] IDLE colorizer In-Reply-To: References: Message-ID: [MRAB [ > A thread on python-ideas is talking about the prefixes of string literals, > and the regex used in IDLE. > > Line 25 of Lib\idlelib\colorizer.py is: > > stringprefix = r"(?i:\br|u|f|fr|rf|b|br|rb)?" > > which looks slightly wrong to me. > > The \b will apply only to the first choice. > > Shouldn't it be more like: > > stringprefix = r"(?:\b(?i:r|u|f|fr|rf|b|br|rb))?" > > ? I believe the change would capture its real intent. It doesn't seem to matter a whole lot, though - IDLE isn't a syntax checker, and applies heuristics to color on the fly based on best guesses. As is, if you type this fragment into an IDLE shell: kr"sdf" only the last 5 characters get "string colored", presumably because of the leading \br in the original regexp. But if you type in ku"sdf" the last 6 characters get "string colored", because - as you pointed out - the \b part of the original regexp has no effect on anything other than the r following \b. But in neither case is the fragment legit Python. If you do type in legit Python, it makes no difference (legit string literals always start at a word boundary, regardless of whether the regexp checks for that). From tjreedy at udel.edu Sun Apr 1 23:52:30 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 1 Apr 2018 23:52:30 -0400 Subject: [Python-Dev] IDLE colorizer In-Reply-To: References: Message-ID: On 4/1/2018 10:20 PM, Tim Peters wrote: > [MRAB [ >> A thread on python-ideas is talking about the prefixes of string literals, >> and the regex used in IDLE. >> >> Line 25 of Lib\idlelib\colorizer.py is: >> >> stringprefix = r"(?i:\br|u|f|fr|rf|b|br|rb)?" >> >> which looks slightly wrong to me. This must be a holdover from years ago, before I was involved. I have wondered about it but left it as is. Thanks for confirming that it is not right. >> The \b will apply only to the first choice. >> >> Shouldn't it be more like: >> >> stringprefix = r"(?:\b(?i:r|u|f|fr|rf|b|br|rb))?" >> >> ? See below. > I believe the change would capture its real intent. It doesn't seem > to matter a whole lot, though - IDLE isn't a syntax checker, and > applies heuristics to color on the fly based on best guesses. As is, > if you type this fragment into an IDLE shell: > > kr"sdf" > > only the last 5 characters get "string colored", presumably because of > the leading \br in the original regexp. But if you type in > > ku"sdf" > > the last 6 characters get "string colored", because - as you pointed > out - the \b part of the original regexp has no effect on anything > other than the r following \b. I tested with uf versus ur, which are both plausibly legal but are not. > But in neither case is the fragment legit Python. If you do type in > legit Python, it makes no difference (legit string literals always > start at a word boundary, regardless of whether the regexp checks for > that). I want uniform behavior. I decided to drop the \b because I prefer coloring the maximal legal string rather than the minimum. I think the contrast between two chars legal by themselves, but differently colored when put together, makes the bug more obvious. https://bugs.python.org/issue33204 -- Terry Jan Reedy From guido at python.org Mon Apr 2 00:43:23 2018 From: guido at python.org (Guido van Rossum) Date: Sun, 1 Apr 2018 21:43:23 -0700 Subject: [Python-Dev] IDLE colorizer In-Reply-To: References: Message-ID: My question for you: how on earth did you find this?! Speaking of a needle in a haystack. Did you run some kind of analysis program that looks for regexprs? (We've received some good reports from someone who did that looking for possible DoS attacks.) On Sun, Apr 1, 2018 at 6:49 PM, MRAB wrote: > A thread on python-ideas is talking about the prefixes of string literals, > and the regex used in IDLE. > > Line 25 of Lib\idlelib\colorizer.py is: > > stringprefix = r"(?i:\br|u|f|fr|rf|b|br|rb)?" > > which looks slightly wrong to me. > > The \b will apply only to the first choice. > > Shouldn't it be more like: > > stringprefix = r"(?:\b(?i:r|u|f|fr|rf|b|br|rb))?" > > ? > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido% > 40python.org > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From songofacandy at gmail.com Mon Apr 2 02:25:51 2018 From: songofacandy at gmail.com (INADA Naoki) Date: Mon, 2 Apr 2018 15:25:51 +0900 Subject: [Python-Dev] Nuking wstr [Re: How can we use 48bit pointer safely?] In-Reply-To: <20180331113510.48624002@fsol> References: <20180330155442.6bf2cd9f@fsol> <20180331113510.48624002@fsol> Message-ID: Some of APIs are stated as "Deprecated since version 3.3, will be removed in version 4.0:". e.g. https://docs.python.org/3/c-api/unicode.html#c.PyUnicode_AS_UNICODE So we will remove them (and wstr) at Python 4.0. From songofacandy at gmail.com Mon Apr 2 02:34:00 2018 From: songofacandy at gmail.com (INADA Naoki) Date: Mon, 2 Apr 2018 15:34:00 +0900 Subject: [Python-Dev] Nuking wstr [Re: How can we use 48bit pointer safely?] In-Reply-To: <20180330155442.6bf2cd9f@fsol> References: <20180330155442.6bf2cd9f@fsol> Message-ID: > > Of course, the question is whether all this matters. Is it important > to save 8 bytes on each unicode object? Only testing would tell. > Last year, I tried to profile memory usage of web application in my company. https://gist.github.com/methane/ce723adb9a4d32d32dc7525b738d3c31#investigating-overall-memory-usage Without -OO option, str is the most memory eater and average size is about 109bytes. (Note: SQLAlchemy uses docstring very heavily). With -OO option, str is the third memory eater, and average size was about 73bytes. So I think 8bytes for each string object is not negligible. But, of course, it's vary according to applications and libraries. -- INADA Naoki From larry at hastings.org Mon Apr 2 07:41:52 2018 From: larry at hastings.org (Larry Hastings) Date: Mon, 2 Apr 2018 04:41:52 -0700 Subject: [Python-Dev] [Crosspost from python-committers] Announcing: signups are open for the 2018 Python Language Summit Message-ID: <7ad20027-bbb4-8d83-befb-1787a513f0da@hastings.org> It?s that time again: time to start thinking about the Python Language Summit!? The 2018 summit will be held on Wednesday, May 9, from 10am to 4pm, at the Huntington Convention Center in Cleveland, Ohio, USA.? Your befezzled and befuddled hosts Barry and Larry will once more be behind the big desk in front. The summit?s purpose is to disseminate information and spark conversation among core Python developers.? It?s our yearly opportunity to get together for an in-person discussion, to review interesting developments of the previous year and hash out where we?re going next.? And we have lots to talk about!? 3.7 is in beta, and we've all collectively started work on 3.8 too. As before, we?re using Google Forms to collect signups.? Signups are open now; the deadline to sign up is Wednesday April 18th, 2018 (AoE).? Please do us a favor and sign up sooner rather than later.? The signup form is simpler this year--I bet most people can be done in less than two minutes! One difference from last year: there are now *two* forms.? The first form is for signing up to attend (the "Request For Invitation" form), and the second form is for proposing a talk. Please note: if you want to present, you still need to fill out the Request For Invitation form too.? (Yes, it's more complicated this way, sorry.? But having both on the same form kind of enforced a one-to-one mapping, and it's really a many-to-many mapping: one person might propose multiple talks, and one talk might have multiple presenters.? Overall this should be less complicated.) You can find links to *both* forms on the official Python Language Summit 2018 page: https://us.pycon.org/2018/events/language-summit/ A few notes: * There will be lightning talks!? Signups will only be available during the Language Summit itself. * You don?t need to be registered for PyCon in order to attend the summit! * We?ll have badge ribbons for Language Summit participants, which we?ll hand out at the summit room in the morning. * We're inviting Jake Edge from Linux Weekly News to attend the summit and provide press coverage again.? Jake?s done a phenomenal job of covering the last few summits, providing valuable information not just for summit attendees, but also for the Python community at large.? Jake?s coverage goes a long way toward demystifying the summit, while remaining respectful of confidential information that?s deemed ?off the record? ahead of time by participants. One big final note (please read this!): When using Google Forms, you /can/ edit your responses later!? When you fill out the form and hit Submit, the submission complete page (the one that says "Thanks for playing!") will have a link on it labeled "Edit your response". BOOKMARK THIS LINK!? You can use this link at /any time/ to edit your response, up to the point that signups close on April 18th.? Keep in mind, you'll need to bookmark each response independently: once for signing up to attend ("Request For Invitation"), and once for each talk proposal you submit. Again, /please/ be sure to save this bookmark yourself--we don't know how to find the link for you later if you don't save it. We hope to see you at the summit! [BL]arry -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Mon Apr 2 14:32:17 2018 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 2 Apr 2018 19:32:17 +0100 Subject: [Python-Dev] IDLE colorizer In-Reply-To: References: Message-ID: <005e65d6-505f-a765-1d56-2bb37a998833@mrabarnett.plus.com> On 2018-04-02 05:43, Guido van Rossum wrote: > My question for you: how on earth did you find this?! Speaking of a > needle in a haystack. Did you run some kind of analysis program that > looks for regexprs? (We've received some good reports from someone who > did that looking for possible DoS attacks.) > The thread was about string prefixes. Terry Reedy wrote "IDLE's colorizer does its parsing with a giant regex." I wondered: "How bad could it be?" (It's smaller now that the IGNORECASE flag can have a local scope.) It wasn't hard to find because it was in a file called "colorizer.py" in a folder called "idlelib". > On Sun, Apr 1, 2018 at 6:49 PM, MRAB > wrote: > > A thread on python-ideas is talking about the prefixes of string > literals, and the regex used in IDLE. > > Line 25 of Lib\idlelib\colorizer.py is: > > ? ? stringprefix = r"(?i:\br|u|f|fr|rf|b|br|rb)?" > > which looks slightly wrong to me. > > The \b will apply only to the first choice. > > Shouldn't it be more like: > > ? ? stringprefix = r"(?:\b(?i:r|u|f|fr|rf|b|br|rb))?" > > ? > From guido at python.org Mon Apr 2 15:01:49 2018 From: guido at python.org (Guido van Rossum) Date: Mon, 2 Apr 2018 12:01:49 -0700 Subject: [Python-Dev] IDLE colorizer In-Reply-To: <005e65d6-505f-a765-1d56-2bb37a998833@mrabarnett.plus.com> References: <005e65d6-505f-a765-1d56-2bb37a998833@mrabarnett.plus.com> Message-ID: Heh. The good old manual approach. :-) How bad indeed? >>> from idlelib import colorizer; colorizer.make_pat() from idlelib import colorizer; colorizer.make_pat() '\\b(?PFalse|None|True|and|as|assert|break|class|continue|def|del|elif|else|except|finally|for|from|global|if|import|in|is|lambda|nonlocal|not|or|pass|raise|return|try|while|with|yield)\\b|([^.\'\\"\\\\#]\\b|^)(?PArithmeticError|AssertionError|AttributeError|BaseException|BlockingIOError|BrokenPipeError|BufferError|BytesWarning|ChildProcessError|ConnectionAbortedError|ConnectionError|ConnectionRefusedError|ConnectionResetError|DeprecationWarning|EOFError|Ellipsis|EnvironmentError|Exception|FileExistsError|FileNotFoundError|FloatingPointError|FutureWarning|GeneratorExit|IOError|ImportError|ImportWarning|IndentationError|IndexError|InterruptedError|IsADirectoryError|KeyError|KeyboardInterrupt|LookupError|MemoryError|ModuleNotFoundError|NameError|NotADirectoryError|NotImplemented|NotImplementedError|OSError|OverflowError|PendingDeprecationWarning|PermissionError|ProcessLookupError|RecursionError|ReferenceError|ResourceWarning|RuntimeError|RuntimeWarning|StopAsyncIteration|StopIteration|SyntaxError|SyntaxWarning|SystemError|SystemExit|TabError|TimeoutError|TypeError|UnboundLocalError|UnicodeDecodeError|UnicodeEncodeError|UnicodeError|UnicodeTranslateError|UnicodeWarning|UserWarning|ValueError|Warning|ZeroDivisionError|abs|all|any|ascii|bin|bool|bytearray|bytes|callable|chr|classmethod|compile|complex|copyright|credits|delattr|dict|dir|divmod|enumerate|eval|exec|exit|filter|float|format|frozenset|getattr|globals|hasattr|hash|help|hex|id|input|int|isinstance|issubclass|iter|len|license|list|locals|map|max|memoryview|min|next|object|oct|open|ord|pow|print|property|quit|range|repr|reversed|round|set|setattr|slice|sorted|staticmethod|str|sum|super|tuple|type|vars|zip)\\b|(?P#[^\\n]*)|(?P(?i:\\br|u|f|fr|rf|b|br|rb)?\'\'\'[^\'\\\\]*((\\\\.|\'(?!\'\'))[^\'\\\\]*)*(\'\'\')?|(?i:\\br|u|f|fr|rf|b|br|rb)?"""[^"\\\\]*((\\\\.|"(?!""))[^"\\\\]*)*(""")?|(?i:\\br|u|f|fr|rf|b|br|rb)?\'[^\'\\\\\\n]*(\\\\.[^\'\\\\\\n]*)*\'?|(?i:\\br|u|f|fr|rf|b|br|rb)?"[^"\\\\\\n]*(\\\\.[^"\\\\\\n]*)*"?)|(?P\\n)' >>> On Mon, Apr 2, 2018 at 11:32 AM, MRAB wrote: > On 2018-04-02 05:43, Guido van Rossum wrote: > >> My question for you: how on earth did you find this?! Speaking of a >> needle in a haystack. Did you run some kind of analysis program that looks >> for regexprs? (We've received some good reports from someone who did that >> looking for possible DoS attacks.) >> >> The thread was about string prefixes. > > Terry Reedy wrote "IDLE's colorizer does its parsing with a giant regex." > > I wondered: "How bad could it be?" (It's smaller now that the IGNORECASE > flag can have a local scope.) > > It wasn't hard to find because it was in a file called "colorizer.py" in a > folder called "idlelib". > > > On Sun, Apr 1, 2018 at 6:49 PM, MRAB > python at mrabarnett.plus.com>> wrote: >> >> A thread on python-ideas is talking about the prefixes of string >> literals, and the regex used in IDLE. >> >> Line 25 of Lib\idlelib\colorizer.py is: >> >> stringprefix = r"(?i:\br|u|f|fr|rf|b|br|rb)?" >> >> which looks slightly wrong to me. >> >> The \b will apply only to the first choice. >> >> Shouldn't it be more like: >> >> stringprefix = r"(?:\b(?i:r|u|f|fr|rf|b|br|rb))?" >> >> ? >> >> > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido% > 40python.org > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukasz at langa.pl Mon Apr 2 16:48:46 2018 From: lukasz at langa.pl (Lukasz Langa) Date: Mon, 2 Apr 2018 13:48:46 -0700 Subject: [Python-Dev] Why is pickle.DEFAULT_PROTOCOL still 3? Message-ID: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> Pickle protocol version 4.0 was originally defined back in PEP 3154 and shipped as part of Python 3.4 back in 2011. Yet it's still not the default. There's a number of things that would run faster with it like multiprocessing. This is too late for 3.7 which is a shame but can we at least bump it for 3.8? - ? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 874 bytes Desc: Message signed with OpenPGP URL: From solipsis at pitrou.net Mon Apr 2 17:13:29 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 2 Apr 2018 23:13:29 +0200 Subject: [Python-Dev] Why is pickle.DEFAULT_PROTOCOL still 3? References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> Message-ID: <20180402231329.48e78d77@fsol> On Mon, 2 Apr 2018 13:48:46 -0700 Lukasz Langa wrote: > Pickle protocol version 4.0 was originally defined back in PEP 3154 and shipped as part of Python 3.4 back in 2011. Yet it's still not the default. Because we want pickles produced with the default to be readable by earlier Python 3 versions. (the same reason protocol 0 stayed the default throughout the Python 2 lifetime) Regards Antoine. From christian at python.org Mon Apr 2 18:11:11 2018 From: christian at python.org (Christian Heimes) Date: Tue, 3 Apr 2018 00:11:11 +0200 Subject: [Python-Dev] Why is pickle.DEFAULT_PROTOCOL still 3? In-Reply-To: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> Message-ID: On 2018-04-02 22:48, Lukasz Langa wrote: > Pickle protocol version 4.0 was originally defined back in PEP 3154 and shipped as part of Python 3.4 back in 2011. Yet it's still not the default. There's a number of things that would run faster with it like multiprocessing. > > This is too late for 3.7 which is a shame but can we at least bump it for 3.8? It sounds like a reasonable request. Python 3.4 is out of commission by then. I'm sure the release manager for 3.8 is going to agree with you, too. :) Christian From greg at krypto.org Mon Apr 2 18:49:37 2018 From: greg at krypto.org (Gregory P. Smith) Date: Mon, 02 Apr 2018 22:49:37 +0000 Subject: [Python-Dev] Why is pickle.DEFAULT_PROTOCOL still 3? In-Reply-To: References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> Message-ID: Given that, go ahead and change in master (3.8). On Mon, Apr 2, 2018 at 3:13 PM Christian Heimes wrote: > On 2018-04-02 22:48, Lukasz Langa wrote: > > Pickle protocol version 4.0 was originally defined back in PEP 3154 and > shipped as part of Python 3.4 back in 2011. Yet it's still not the default. > There's a number of things that would run faster with it like > multiprocessing. > > > > This is too late for 3.7 which is a shame but can we at least bump it > for 3.8? > > It sounds like a reasonable request. Python 3.4 is out of commission by > then. I'm sure the release manager for 3.8 is going to agree with you, > too. :) > > Christian > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/greg%40krypto.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukasz at langa.pl Mon Apr 2 18:57:11 2018 From: lukasz at langa.pl (Lukasz Langa) Date: Mon, 2 Apr 2018 15:57:11 -0700 Subject: [Python-Dev] Why is pickle.DEFAULT_PROTOCOL still 3? In-Reply-To: <20180402231329.48e78d77@fsol> References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> Message-ID: > On Apr 2, 2018, at 2:13 PM, Antoine Pitrou wrote: > > On Mon, 2 Apr 2018 13:48:46 -0700 > Lukasz Langa wrote: >> Pickle protocol version 4.0 was originally defined back in PEP 3154 and shipped as part of Python 3.4 back in 2011. Yet it's still not the default. > > Because we want pickles produced with the default to be readable by > earlier Python 3 versions. > (the same reason protocol 0 stayed the default throughout the Python 2 > lifetime) Alright, so that means we can easily do this for Python 3.8, right? I mean, following Christian's logic, Python 3.3 is already dead, with its final release done in February 2016 and support dropped in September 2017 per PEP 398. I think we need to get past thinking about "Python 2" vs. "Python 3". This frame of mind creates space for another mythical release of Python that will break all the compatibilities, something we promised not to do. A moving backward compatibility window that includes the last release still under security fixes seems like a good new framework for this. What do you think? - ? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 874 bytes Desc: Message signed with OpenPGP URL: From greg at krypto.org Mon Apr 2 19:25:37 2018 From: greg at krypto.org (Gregory P. Smith) Date: Mon, 02 Apr 2018 23:25:37 +0000 Subject: [Python-Dev] Why is pickle.DEFAULT_PROTOCOL still 3? In-Reply-To: References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> Message-ID: On Mon, Apr 2, 2018 at 3:57 PM Lukasz Langa wrote: > > > On Apr 2, 2018, at 2:13 PM, Antoine Pitrou wrote: > > > > On Mon, 2 Apr 2018 13:48:46 -0700 > > Lukasz Langa wrote: > >> Pickle protocol version 4.0 was originally defined back in PEP 3154 and > shipped as part of Python 3.4 back in 2011. Yet it's still not the default. > > > > Because we want pickles produced with the default to be readable by > > earlier Python 3 versions. > > (the same reason protocol 0 stayed the default throughout the Python 2 > > lifetime) > > Alright, so that means we can easily do this for Python 3.8, right? I > mean, following Christian's logic, Python 3.3 is already dead, with its > final release done in February 2016 and support dropped in September 2017 > per PEP 398. > > I think we need to get past thinking about "Python 2" vs. "Python 3". This > frame of mind creates space for another mythical release of Python that > will break all the compatibilities, something we promised not to do. A > moving backward compatibility window that includes the last release still > under security fixes seems like a good new framework for this. > > What do you think? > +1 - That puts the words to the reason I suggest just running with the change for 3.8. If we had noticed in time I'd've suggested doing it in 3.7, too late and not a huge deal so just wait for 3.8. Never changing it in the past during 1 and 2.x led a lot of code to always specify HIGHEST as the protocol because the default was unreasonable. -gps -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablogsal at gmail.com Mon Apr 2 20:24:50 2018 From: pablogsal at gmail.com (Pablo Galindo Salgado) Date: Tue, 03 Apr 2018 00:24:50 +0000 Subject: [Python-Dev] building extensions as builtins is broken in 3.7 Message-ID: Hi, I am working on the release blocker https://bugs.python.org/issue32232. I tried to apply the patch proposed by Matthias Klose and I found that it works on Unix but it fails to build on Windows (probably because circular imports). I tried to add some tests but after some work on the problem and some comments by Ned Deily I am starting to think that the test for this is basically compiling the whole interpreter with static linking of the built-ins and run the whole test suite. This has to be done as a new build target on Travis. I did some experiments with this following: https://wiki.python.org/moin/BuildStatically But I run into multiple linking errors. The Travis build hangs forever and then fails in the best case. Here is an example of what I am talking: https://github.com/pablogsal/cpython/pull/1 In this PR you can see the patch and the static linking configured as per above and the Travis output. I am happy to work on this as long as someone can tell me what is the appropriate course of action. Thank you very much for your time! -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Tue Apr 3 04:01:09 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 3 Apr 2018 10:01:09 +0200 Subject: [Python-Dev] Why is pickle.DEFAULT_PROTOCOL still 3? In-Reply-To: References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> Message-ID: <20180403100109.12898f64@fsol> On Mon, 2 Apr 2018 15:57:11 -0700 Lukasz Langa wrote: > > On Apr 2, 2018, at 2:13 PM, Antoine Pitrou wrote: > > > > On Mon, 2 Apr 2018 13:48:46 -0700 > > Lukasz Langa wrote: > >> Pickle protocol version 4.0 was originally defined back in PEP 3154 and shipped as part of Python 3.4 back in 2011. Yet it's still not the default. > > > > Because we want pickles produced with the default to be readable by > > earlier Python 3 versions. > > (the same reason protocol 0 stayed the default throughout the Python 2 > > lifetime) > > Alright, so that means we can easily do this for Python 3.8, right? I mean, following Christian's logic, Python 3.3 is already dead, with its final release done in February 2016 and support dropped in September 2017 per PEP 398. > > I think we need to get past thinking about "Python 2" vs. "Python 3". This frame of mind creates space for another mythical release of Python that will break all the compatibilities, something we promised not to do. A moving backward compatibility window that includes the last release still under security fixes seems like a good new framework for this. > > What do you think? That sounds reasonable to me. Let's see whether other people disagree. Regards Antoine. From storchaka at gmail.com Tue Apr 3 04:18:21 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Tue, 3 Apr 2018 11:18:21 +0300 Subject: [Python-Dev] Why is pickle.DEFAULT_PROTOCOL still 3? In-Reply-To: References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> Message-ID: 03.04.18 01:57, Lukasz Langa ????: >> On Apr 2, 2018, at 2:13 PM, Antoine Pitrou wrote: >> On Mon, 2 Apr 2018 13:48:46 -0700 >> Lukasz Langa wrote: >>> Pickle protocol version 4.0 was originally defined back in PEP 3154 and shipped as part of Python 3.4 back in 2011. Yet it's still not the default. >> Because we want pickles produced with the default to be readable by >> earlier Python 3 versions. >> (the same reason protocol 0 stayed the default throughout the Python 2 >> lifetime) > > Alright, so that means we can easily do this for Python 3.8, right? I mean, following Christian's logic, Python 3.3 is already dead, with its final release done in February 2016 and support dropped in September 2017 per PEP 398. > > I think we need to get past thinking about "Python 2" vs. "Python 3". This frame of mind creates space for another mythical release of Python that will break all the compatibilities, something we promised not to do. A moving backward compatibility window that includes the last release still under security fixes seems like a good new framework for this. > > What do you think? The only possible drawback of protocol 4 is that very short pickles can be longer than with protocol 3 due to additional 9 bytes for the FRAME header and less compact pickling of globals. This may be partially compensated by implementing additional optimizations and/or passing short pickles through pickletools.optimize(). From paul at ganssle.io Tue Apr 3 08:51:43 2018 From: paul at ganssle.io (Paul G) Date: Tue, 3 Apr 2018 08:51:43 -0400 Subject: [Python-Dev] Python version numbers In-Reply-To: References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> Message-ID: Breaking this off from the pickle thread because it seems unrelated: On 04/02/2018 06:57 PM, Lukasz Langa wrote: > I think we need to get past thinking about "Python 2" vs. "Python 3". This frame of mind creates space for another mythical release of Python that will break all the compatibilities, something we promised not to do. A moving backward compatibility window that includes the last release still under security fixes seems like a good new framework for this. Maybe this has already been discussed ad nauseum, but is the idea here that Python will stay on Python 3.x, but also start breaking backwards compatibility with old versions? That would seem to be a violation of semantic versioning. I think if this is going to happen, it should either be that the major version number gets bumped with every compatibility-breaking release, or Python should switch to some form of calendrical versioning (which may amount to more or less the same thing, but be different enough that people won't freak out when you say Python 4 is coming): https://calver.org Switching to CalVer is a pretty clear sign that there is now a "rolling backwards compatibility window", and it allows Python to skip right over the mythical "Python 4" and directly to "Python 21". Additionally, since the version number will be trivially predictable, deprecation warnings can actually include the version after which they will be dropped - so if a feature is slated to be removed 5 years after it is initially deprecated, just take the deprecation release version and add 5. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From p.f.moore at gmail.com Tue Apr 3 09:07:57 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 3 Apr 2018 14:07:57 +0100 Subject: [Python-Dev] Python version numbers In-Reply-To: References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> Message-ID: On 3 April 2018 at 13:51, Paul G wrote: > Maybe this has already been discussed ad nauseum, but is the idea here that Python will stay on Python 3.x, but also start breaking backwards compatibility with old versions? That would seem to be a violation of semantic versioning. Python's versions don't follow strict semantic versioning. See https://docs.python.org/3/faq/general.html#how-does-the-python-version-numbering-scheme-work Paul From njs at pobox.com Tue Apr 3 09:24:27 2018 From: njs at pobox.com (Nathaniel Smith) Date: Tue, 3 Apr 2018 06:24:27 -0700 Subject: [Python-Dev] Python version numbers In-Reply-To: References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> Message-ID: On Tue, Apr 3, 2018 at 5:51 AM, Paul G wrote: > Maybe this has already been discussed ad nauseum, but is the idea here that Python will stay on Python 3.x, but also start breaking backwards compatibility with old versions? That would seem to be a violation of semantic versioning. Python's versioning has worked this way since well before "semantic versioning" was invented... so I think it's more fair to say semantic versioning is a violation of Python ;-). CalVer does have the nice properties that it makes a public commitment that there will never be another "x.0"-style release, and that it encodes some useful information into the version (maybe it would be slightly easier to convince people to stop using 2.7 if it was "Python 2010"). But the current scheme is so ingrained in the culture at this point that I doubt there's much appetite for fiddling with it... -n -- Nathaniel J. Smith -- https://vorpus.org From paul at ganssle.io Tue Apr 3 09:24:22 2018 From: paul at ganssle.io (Paul G) Date: Tue, 3 Apr 2018 09:24:22 -0400 Subject: [Python-Dev] Python version numbers In-Reply-To: References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> Message-ID: <43456b0e-a2ec-01c2-72d2-a6633909fcdd@ganssle.io> That documentation seems like a "layman's explanation" of how semantic versioning works. I suspect anyone familiar with semantic versioning will read that and think, "Ah, yes, this is a semantic versioning scheme." Regardless of the semantics (har har) of whether Python "follows strict semantic versioning", a change to the versioning scheme (CalVer should be backwards compatible with SemVer, mind you, since (21, 0, 0) > (3, 8, 0)) should make it absolutely clear that Python is not following SemVer. Counter-intuitively, I think the *fact* of pinning the major version number to 3 is a worse signal of "we're not going to break everything" than switching to CalVer could. By switching to CalVer, you are removing the *ability* to signal a discontinuous major breaking change just by the version number. It is very much a "burn your boats so you can't retreat" philosophy to versioning. Of course, if we want to reserve the ability to have sudden and major breaking changes, then yes, sticking with the current semi-SemVer system is fine, but I suspect that the fact that minor releases can break backwards compatibility will be confusing and annoying for most people (not me, because I know about it and I test against nightly), and as long as there's a "3" in the "major" slot, people will speculate about the possibility of a "4". On 04/03/2018 09:07 AM, Paul Moore wrote: > On 3 April 2018 at 13:51, Paul G wrote: >> Maybe this has already been discussed ad nauseum, but is the idea here that Python will stay on Python 3.x, but also start breaking backwards compatibility with old versions? That would seem to be a violation of semantic versioning. > > Python's versions don't follow strict semantic versioning. See > https://docs.python.org/3/faq/general.html#how-does-the-python-version-numbering-scheme-work > > Paul > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From ncoghlan at gmail.com Tue Apr 3 10:10:38 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 4 Apr 2018 00:10:38 +1000 Subject: [Python-Dev] Python version numbers In-Reply-To: <43456b0e-a2ec-01c2-72d2-a6633909fcdd@ganssle.io> References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> <43456b0e-a2ec-01c2-72d2-a6633909fcdd@ganssle.io> Message-ID: On 3 April 2018 at 23:24, Paul G wrote: > That documentation seems like a "layman's explanation" of how semantic versioning works. I suspect anyone familiar with semantic versioning will read that and think, "Ah, yes, this is a semantic versioning scheme." Anyone that reads the porting section in one of our What's New documents will quickly learn that we *don't* use semantic versioning - we use rolling deprecation windows, and have done so for decades :) > Regardless of the semantics (har har) of whether Python "follows strict semantic versioning", a change to the versioning scheme (CalVer should be backwards compatible with SemVer, mind you, since (21, 0, 0) > (3, 8, 0)) should make it absolutely clear that Python is not following SemVer. > > Counter-intuitively, I think the *fact* of pinning the major version number to 3 is a worse signal of "we're not going to break everything" than switching to CalVer could. By switching to CalVer, you are removing the *ability* to signal a discontinuous major breaking change just by the version number. It is very much a "burn your boats so you can't retreat" philosophy to versioning. The reason for sticking with 3.x for a while is because of the corner \*nix systems have gotten stuck into regarding the "python" symlink, and the fact it currently still points to "python2" (if it exists at all). Once we've updated PEP 394 to recommend pointing it at Python 3 (which is currently looking like it might happen around the 3.8 or 3.9 time frame), then we can start talking about instead pointing it at python 4.x, and making "python3" just be a confusingly named symlink to python4.x. That consideration applies regardless of whether the version change following the last 3.x release is a simple major version increment to python 4.x or to something CalVer based like python 22.x > Of course, if we want to reserve the ability to have sudden and major breaking changes, then yes, sticking with the current semi-SemVer system is fine, but I suspect that the fact that minor releases can break backwards compatibility will be confusing and annoying for most people (not me, because I know about it and I test against nightly), and as long as there's a "3" in the "major" slot, people will speculate about the possibility of a "4". Indeed, but the details of Python's version numbering scheme have a *lot* of backwards compatibility implications, as even we haven't always been diligent about including separators between the segments of the version number in various use cases (e.g. wheel compatibility tags allow the use of underscores for disambiguation, but I'd bet at least some code doesn't currently handle that correctly). That means going to a scheme like "22.x" would risk emitting version number that sort lexically lower than "27" in some contexts. Since it's a "not to be resolved until after 3.9" problem regardless, we have time to consider how we want to handle it :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From rosuav at gmail.com Tue Apr 3 10:15:01 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 4 Apr 2018 00:15:01 +1000 Subject: [Python-Dev] Python version numbers In-Reply-To: References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> Message-ID: On Tue, Apr 3, 2018 at 10:51 PM, Paul G wrote: > Breaking this off from the pickle thread because it seems unrelated: > > On 04/02/2018 06:57 PM, Lukasz Langa wrote: >> I think we need to get past thinking about "Python 2" vs. "Python 3". This frame of mind creates space for another mythical release of Python that will break all the compatibilities, something we promised not to do. A moving backward compatibility window that includes the last release still under security fixes seems like a good new framework for this. > > Maybe this has already been discussed ad nauseum, but is the idea here that Python will stay on Python 3.x, but also start breaking backwards compatibility with old versions? That would seem to be a violation of semantic versioning. > Compatibility can be broken in small ways by minor versions. For instance, if you have a function "def await(x):", that will work perfectly on older Pythons, but will fail now that 'await' is a keyword. Does that require a major version bump? The pickle situation was one of changing a default, and if you need compatibility with ancient versions, you can specify protocol 0. By maintaining a sane default, we encourage people to use that rather than specifying protocol=-1 and breaking compatibility *immediately* when a new protocol is created. The expectation is that a minor version bump generally won't break things, but you test your code anyway, in case it does. (A revision release should never break anything that wasn't already broken, eg depending on a bug.) A change to the default pickle protocol seems like a fairly safe change to me, since it's so easy to override if you need to. When programs use calendar-based versioning, I'm left with no information as to whether it's breaking changes or not. In fact, it might as well have no version numbers whatsoever. If I care about backward compatibility, I just have to stick with the exact same unpatched version that I had before. Not a good thing for a language that releases periodic bugfix updates to older versions. ChrisA From paul at ganssle.io Tue Apr 3 10:38:34 2018 From: paul at ganssle.io (Paul G) Date: Tue, 3 Apr 2018 10:38:34 -0400 Subject: [Python-Dev] Python version numbers In-Reply-To: References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> Message-ID: <2b189eca-a624-8142-a3e5-7962e7e1da0d@ganssle.io> > When programs use calendar-based versioning, I'm left with no > information as to whether it's breaking changes or not. In fact, it > might as well have no version numbers whatsoever. If I care about > backward compatibility, I just have to stick with the exact same > unpatched version that I had before. Not a good thing for a language > that releases periodic bugfix updates to older versions. Well, this is not true. As a general rule, you don't know if anything *you care about* breaks at a given version change anyway, and I've seen plenty of handwringing about what should and should not be considered a breaking change. A major change is basically the "nuclear option", and everything I've seen from core developers is "there will never be another major change on the scale of 2->3". Given that from an optics perspective there's no interest in creating a Python 4 and it's been repeatedly emphasized in this thread that Python doesn't use semantic versioning, I don't see how you can really say that the version number gives you much information at all about what will and will not break. This thread started in response to a proposal to just have a rolling backwards compatibility window (e.g. no discontinuous major version changes), which is basically ideal for calendar versioning - any time you make a backwards incompatible change, you give X notice period. You can encode X directly into the deprecation warnings (e.g. "This feature will be removed in Python >= 2025"), that way users will know on a *per feature basis* where to pin, even before specific version numbers are announced. Presumably if major breaking changes ever *do* occur, the mechanism for upgrading Python would be either to create a new program called something else (e.g. the core Python simply *does not break backwards compat*), or a slow rollout with individual feature flags that start out defaulting on (opt in to the new behavior), eventually default to off (opt out of the new behavior), and then are removed after a very long deprecation period. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From paul at ganssle.io Tue Apr 3 10:55:56 2018 From: paul at ganssle.io (Paul G) Date: Tue, 3 Apr 2018 10:55:56 -0400 Subject: [Python-Dev] Python version numbers In-Reply-To: References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> <43456b0e-a2ec-01c2-72d2-a6633909fcdd@ganssle.io> Message-ID: <65bd6cb5-9b9e-74b9-b61b-65e6ec4edb78@ganssle.io> On 04/03/2018 10:10 AM, Nick Coghlan wrote: > The reason for sticking with 3.x for a while is because of the corner > \*nix systems have gotten stuck into regarding the "python" symlink, > and the fact it currently still points to "python2" (if it exists at > all). Once we've updated PEP 394 to recommend pointing it at Python 3 > (which is currently looking like it might happen around the 3.8 or 3.9 > time frame), then we can start talking about instead pointing it at > python 4.x, and making "python3" just be a confusingly named symlink > to python4.x. I was definitely not suggesting this before the end of the Python 2 era, but I don't think Lukasz was suggesting this either. I'm suggesting it specifically for the period of time during which Python is using the "rolling backwards compatibility guarantee" standard laid out in the original posting. I would think in terms of evaluating this proposal, it is (in order) worth considering: 1. Is it a desirable goal 2. Is it an achievable goal? 3. How specifically can it be achieved? 4. What is the implementation timeline? If #1 and #2 are true (which is my contention), *then* the specific details can be ironed out about how and when. Ideally, if it is desirable and achievable, it might be prudent to wait to start the "rolling backwards compatibility guarantee" era as part of the new versioning scheme rollout, rather than updating the versioning scheme post-hoc to reflect the new status quo. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From radamar at protonmail.com Tue Apr 3 05:01:24 2018 From: radamar at protonmail.com (amar) Date: Tue, 03 Apr 2018 05:01:24 -0400 Subject: [Python-Dev] Hi, I am amar :) Message-ID: Hi, all. Quick joke, Do you know why all functional programmers are anarchists? Because they want to get rid of the state! :D I know there are a lot more important issues than this, but i feel this is important too. I wish some people could take a look at Python/Cpython pull request #6343[1] and the reopened PR #6362[2]. Little background here, I am a science student working on GIS(Geospatial Information Systems) Web Processing Service as a hobby project. I am a human too, i like jokes. But not at the expense of truth. I hope i can make more meaningful contributions in the future! [1]https://github.com/python/cpython/pull/6343 [2] https://github.com/python/cpython/pull/6362#issuecomment-378174084 -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Tue Apr 3 12:36:56 2018 From: brett at python.org (Brett Cannon) Date: Tue, 03 Apr 2018 16:36:56 +0000 Subject: [Python-Dev] Python version numbers In-Reply-To: <2b189eca-a624-8142-a3e5-7962e7e1da0d@ganssle.io> References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> <2b189eca-a624-8142-a3e5-7962e7e1da0d@ganssle.io> Message-ID: On Tue, 3 Apr 2018 at 07:39 Paul G wrote: > > When programs use calendar-based versioning, I'm left with no > > information as to whether it's breaking changes or not. In fact, it > > might as well have no version numbers whatsoever. If I care about > > backward compatibility, I just have to stick with the exact same > > unpatched version that I had before. Not a good thing for a language > > that releases periodic bugfix updates to older versions. > > Well, this is not true. As a general rule, you don't know if anything *you > care about* breaks at a given version change anyway, and I've seen plenty > of handwringing about what should and should not be considered a breaking > change. A major change is basically the "nuclear option", and everything > I've seen from core developers is "there will never be another major change > on the scale of 2->3". Given that from an optics perspective there's no > interest in creating a Python 4 and it's been repeatedly emphasized in this > thread that Python doesn't use semantic versioning, I don't see how you can > really say that the version number gives you much information at all about > what will and will not break. > Paul's point is that he knows e.g. code working in 3.6.0 will work when he upgrades to 3.6.5, and if his code is warning-free and works with all __future__ statements in 3.6 that it will work fine in 3.7. With CalVer you could make a similar promise if you keep the major version the year of release and then keep our feature/bugfix number promise like we already have, but at that point who cares about the year? > > This thread started in response to a proposal to just have a rolling > backwards compatibility window (e.g. no discontinuous major version > changes), which is basically ideal for calendar versioning - any time you > make a backwards incompatible change, you give X notice period. You can > encode X directly into the deprecation warnings (e.g. "This feature will be > removed in Python >= 2025"), that way users will know on a *per feature > basis* where to pin, even before specific version numbers are announced. > While we have an 18 month schedule for releases, we also can't guarantee we will hit our release window in a specific year, e.g. 3.6 was first released in December but we could have slipped into January. That makes promises for a specific version tied to a year number problematic. > > Presumably if major breaking changes ever *do* occur, the mechanism for > upgrading Python would be either to create a new program called something > else (e.g. the core Python simply *does not break backwards compat*), or a > slow rollout with individual feature flags that start out defaulting on > (opt in to the new behavior), eventually default to off (opt out of the new > behavior), and then are removed after a very long deprecation period. > We already have this with __future__ statements. The transition from 2 to 3 was such a big deal because we changed so much without using __future__ statements. We've already said that going forward we are not skipping that step for any other releases. If we chose to switch to semantic versioning, we would probably make it so that any version that makes a __future__ statement the new default is a major version bump. But then the issue becomes the stdlib. Do we drop everything that was previously deprecated in the stdlib for every major version bump? That might be a bit drastic when e.g. all you did was make old code that used `await` as a variable name raise a SyntaxError. And saying "deprecated for two versions" then becomes messy because you have no idea what that second version will be. And what if you **really** want to get rid of something in the next release? Can a single module dropping a function force a version bump for Python itself? I have not read the other thread, but knowing it's from Lukasz makes me guess he wants to move up the minor number to become the major one in terms of our current semantics so we have a rolling window of support. That would mean that as long as you are warnings-free and run fine with all __future__ statements turned on for version N then N+1 should work for you without modification (sans relying on buggy semantics). That approach would deal with the above issues cleanly while dropping what our current major number semantics which some people view as a fallacy and are getting upset over (I'm personally +0 on this last idea as it's easy to explain to folks and I have been doing Python version explanations a lot over the last several months). -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Tue Apr 3 12:42:06 2018 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Apr 2018 09:42:06 -0700 Subject: [Python-Dev] Python version numbers In-Reply-To: References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> <2b189eca-a624-8142-a3e5-7962e7e1da0d@ganssle.io> Message-ID: I personally see no reason to change anything. On Tue, Apr 3, 2018 at 9:36 AM, Brett Cannon wrote: > > > On Tue, 3 Apr 2018 at 07:39 Paul G wrote: > >> > When programs use calendar-based versioning, I'm left with no >> > information as to whether it's breaking changes or not. In fact, it >> > might as well have no version numbers whatsoever. If I care about >> > backward compatibility, I just have to stick with the exact same >> > unpatched version that I had before. Not a good thing for a language >> > that releases periodic bugfix updates to older versions. >> >> Well, this is not true. As a general rule, you don't know if anything >> *you care about* breaks at a given version change anyway, and I've seen >> plenty of handwringing about what should and should not be considered a >> breaking change. A major change is basically the "nuclear option", and >> everything I've seen from core developers is "there will never be another >> major change on the scale of 2->3". Given that from an optics perspective >> there's no interest in creating a Python 4 and it's been repeatedly >> emphasized in this thread that Python doesn't use semantic versioning, I >> don't see how you can really say that the version number gives you much >> information at all about what will and will not break. >> > > Paul's point is that he knows e.g. code working in 3.6.0 will work when he > upgrades to 3.6.5, and if his code is warning-free and works with all > __future__ statements in 3.6 that it will work fine in 3.7. With CalVer you > could make a similar promise if you keep the major version the year of > release and then keep our feature/bugfix number promise like we already > have, but at that point who cares about the year? > > >> >> This thread started in response to a proposal to just have a rolling >> backwards compatibility window (e.g. no discontinuous major version >> changes), which is basically ideal for calendar versioning - any time you >> make a backwards incompatible change, you give X notice period. You can >> encode X directly into the deprecation warnings (e.g. "This feature will be >> removed in Python >= 2025"), that way users will know on a *per feature >> basis* where to pin, even before specific version numbers are announced. >> > > While we have an 18 month schedule for releases, we also can't guarantee > we will hit our release window in a specific year, e.g. 3.6 was first > released in December but we could have slipped into January. That makes > promises for a specific version tied to a year number problematic. > > >> >> Presumably if major breaking changes ever *do* occur, the mechanism for >> upgrading Python would be either to create a new program called something >> else (e.g. the core Python simply *does not break backwards compat*), or a >> slow rollout with individual feature flags that start out defaulting on >> (opt in to the new behavior), eventually default to off (opt out of the new >> behavior), and then are removed after a very long deprecation period. >> > > We already have this with __future__ statements. The transition from 2 to > 3 was such a big deal because we changed so much without using __future__ > statements. We've already said that going forward we are not skipping that > step for any other releases. > > If we chose to switch to semantic versioning, we would probably make it so > that any version that makes a __future__ statement the new default is a > major version bump. But then the issue becomes the stdlib. Do we drop > everything that was previously deprecated in the stdlib for every major > version bump? That might be a bit drastic when e.g. all you did was make > old code that used `await` as a variable name raise a SyntaxError. And > saying "deprecated for two versions" then becomes messy because you have no > idea what that second version will be. And what if you **really** want to > get rid of something in the next release? Can a single module dropping a > function force a version bump for Python itself? > > I have not read the other thread, but knowing it's from Lukasz makes me > guess he wants to move up the minor number to become the major one in terms > of our current semantics so we have a rolling window of support. That would > mean that as long as you are warnings-free and run fine with all __future__ > statements turned on for version N then N+1 should work for you without > modification (sans relying on buggy semantics). That approach would deal > with the above issues cleanly while dropping what our current major number > semantics which some people view as a fallacy and are getting upset over > (I'm personally +0 on this last idea as it's easy to explain to folks and I > have been doing Python version explanations a lot over the last several > months). > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Tue Apr 3 12:42:13 2018 From: brett at python.org (Brett Cannon) Date: Tue, 03 Apr 2018 16:42:13 +0000 Subject: [Python-Dev] Why is pickle.DEFAULT_PROTOCOL still 3? In-Reply-To: References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> Message-ID: On Tue, 3 Apr 2018 at 01:19 Serhiy Storchaka wrote: > 03.04.18 01:57, Lukasz Langa ????: > >> On Apr 2, 2018, at 2:13 PM, Antoine Pitrou wrote: > >> On Mon, 2 Apr 2018 13:48:46 -0700 > >> Lukasz Langa wrote: > >>> Pickle protocol version 4.0 was originally defined back in PEP 3154 > and shipped as part of Python 3.4 back in 2011. Yet it's still not the > default. > >> Because we want pickles produced with the default to be readable by > >> earlier Python 3 versions. > >> (the same reason protocol 0 stayed the default throughout the Python 2 > >> lifetime) > > > > Alright, so that means we can easily do this for Python 3.8, right? I > mean, following Christian's logic, Python 3.3 is already dead, with its > final release done in February 2016 and support dropped in September 2017 > per PEP 398. > > > > I think we need to get past thinking about "Python 2" vs. "Python 3". > This frame of mind creates space for another mythical release of Python > that will break all the compatibilities, something we promised not to do. A > moving backward compatibility window that includes the last release still > under security fixes seems like a good new framework for this. > > > > What do you think? > > The only possible drawback of protocol 4 is that very short pickles can > be longer than with protocol 3 due to additional 9 bytes for the FRAME > header and less compact pickling of globals. > > This may be partially compensated by implementing additional > optimizations and/or passing short pickles through pickletools.optimize(). > I think if you're that worried about specifics to that detail then you should be specifying the protocol level manually anyway. I view this like something in the peepholer: a freebie perf bump for those that aren't too worried about such things. :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul at ganssle.io Tue Apr 3 13:09:13 2018 From: paul at ganssle.io (Paul G) Date: Tue, 3 Apr 2018 13:09:13 -0400 Subject: [Python-Dev] Python version numbers In-Reply-To: References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> <2b189eca-a624-8142-a3e5-7962e7e1da0d@ganssle.io> Message-ID: On 04/03/2018 12:36 PM, Brett Cannon wrote: > On Tue, 3 Apr 2018 at 07:39 Paul G wrote: > Paul's point is that he knows e.g. code working in 3.6.0 will work when he > upgrades to 3.6.5, and if his code is warning-free and works with all > __future__ statements in 3.6 that it will work fine in 3.7. With CalVer you > could make a similar promise if you keep the major version the year of > release and then keep our feature/bugfix number promise like we already > have, but at that point who cares about the year? I think it is reasonable to use a scheme like this: YY.MM.patch The canonical release is numbered by Year-Month, and patches intended to add new features and break nothing get a patch release, no matter when they are released. That allows for simple pinning. > While we have an 18 month schedule for releases, we also can't guarantee we > will hit our release window in a specific year, e.g. 3.6 was first released > in December but we could have slipped into January. That makes promises for > a specific version tied to a year number problematic. I think no promises for a specific version number need to be made until *after* a release is made. Using calendar-based versioning facilitates using a calendar-based deprecation schedule, irrespective of how many releases have occurred between the deprecation and the removal. If the promise is "we will support this feature for 5 years", then you know ahead of time the minimum version number in which that feature will be removed. So say you have a function foo() called in your library that is deprecated in the 20.05.0 feature release. You know at that point that any release *after* 25.05.0 will drop support for foo, so on your next release that uses foo(), you set the upper bound on your `requires_python` to `<25.05.0`. It doesn't matter if that release happens in 2025-08 or 2027-01. Similarly if you are developing some cutting edge library that is using features from the "feature release" branch, you can set `requires_python` to some value with a month greater than the current calendar version. > If we chose to switch to semantic versioning, we would probably make it so > that any version that makes a __future__ statement the new default is a > major version bump. But then the issue becomes the stdlib. Do we drop > everything that was previously deprecated in the stdlib for every major > version bump? That might be a bit drastic when e.g. all you did was make > old code that used `await` as a variable name raise a SyntaxError. And > saying "deprecated for two versions" then becomes messy because you have no > idea what that second version will be. And what if you **really** want to > get rid of something in the next release? Can a single module dropping a > function force a version bump for Python itself? This is why I suggested a calendar-version based approach, along with a pre-commitment that the "rolling backwards compatibility window" would be a specific amount of *time*, not a specific number of versions. It makes it very easy to make predicitions about when specific features are falling in and out of that window, because each deprecation warning can come *pre-populated* with the length of the deprecation period and, as a result, the maximum version that will support that feature. If that were done, you could even concievably write a "compatibility checker" tool that can be run against a repo to calculate the `requires_python` upper bound and to order deprecation warnings by urgency. > I have not read the other thread, but knowing it's from Lukasz makes me > guess he wants to move up the minor number to become the major one in terms > of our current semantics so we have a rolling window of support. That would > mean that as long as you are warnings-free and run fine with all __future__ > statements turned on for version N then N+1 should work for you without > modification (sans relying on buggy semantics). That approach would deal > with the above issues cleanly while dropping what our current major number > semantics which some people view as a fallacy and are getting upset over > (I'm personally +0 on this last idea as it's easy to explain to folks and I > have been doing Python version explanations a lot over the last several > months). > Lukasz did not propose any changes to the version number semantics, just the deprecation schedule. I was suggesting that if this happens, it's probably a good idea to "drop" the major version number - either by incrementing it with each release that breaks backwards compat (which, for a sufficiently continuous influx of rolling deprecations should be basically every feature release), or by switching to a calendar-based system. From above, I think a calendar-based system has the best features for end users. From python at mrabarnett.plus.com Tue Apr 3 13:45:13 2018 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 3 Apr 2018 18:45:13 +0100 Subject: [Python-Dev] Python version numbers In-Reply-To: References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> <2b189eca-a624-8142-a3e5-7962e7e1da0d@ganssle.io> Message-ID: <1dd740dc-cd4a-e162-ddc3-af817c31dec5@mrabarnett.plus.com> On 2018-04-03 18:09, Paul G wrote: > > > On 04/03/2018 12:36 PM, Brett Cannon wrote: >> On Tue, 3 Apr 2018 at 07:39 Paul G wrote: > >> Paul's point is that he knows e.g. code working in 3.6.0 will work when he >> upgrades to 3.6.5, and if his code is warning-free and works with all >> __future__ statements in 3.6 that it will work fine in 3.7. With CalVer you >> could make a similar promise if you keep the major version the year of >> release and then keep our feature/bugfix number promise like we already >> have, but at that point who cares about the year? > > I think it is reasonable to use a scheme like this: > > YY.MM.patch > Surely that should be: YYYY.MM.patch [snip] From barry at python.org Tue Apr 3 14:17:08 2018 From: barry at python.org (Barry Warsaw) Date: Tue, 3 Apr 2018 11:17:08 -0700 Subject: [Python-Dev] Python version numbers In-Reply-To: References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> Message-ID: On Apr 3, 2018, at 05:51, Paul G wrote: > Switching to CalVer is a pretty clear sign that there is now a "rolling backwards compatibility window", and it allows Python to skip right over the mythical "Python 4" and directly to "Python 21". Additionally, since the version number will be trivially predictable, deprecation warnings can actually include the version after which they will be dropped - so if a feature is slated to be removed 5 years after it is initially deprecated, just take the deprecation release version and add 5. Changing the versioning scheme is a topic that comes up every now and then, and I think it?s worth exploring, but I also don?t think we should do anything about it until the EOL of Python 2.7 at the earliest (if ever). That said, and assuming we keep the current scheme, I think there?s a natural place to label ?Python 4? - we have to break the C API to get rid of the GIL and/or adopt GC over refcounting, or something of that nature. I think there is no interest or appetite for a Python source level breaking change like 2->3, so I wouldn?t expect anything more than the usual changes at the Python source level for 3.x->4. Of course, you?d like to mitigate the breakage of extension modules as much as possible, but should that be unavoidable, then it would warrant a first digit version bump. OTOH, some calver-like scheme would be interesting too if we shorten the release cycle. Could we release a new Python version every 12 months instead of 18? Can we adopt a time-based release policy, so that whatever gets in, gets in, and the rest has to wait until the next release? That?s certainly more painful when that wait is 18 months rather than something shorter. But if releases come more quickly, that has implications for the deprecation policy too. And it puts pressure on the second digit because something like Python 3.53 is distasteful (especially because it would be easily confused with 3.5.3). Python 21.12 anyone? :) Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From brett at python.org Tue Apr 3 16:08:28 2018 From: brett at python.org (Brett Cannon) Date: Tue, 03 Apr 2018 20:08:28 +0000 Subject: [Python-Dev] Python version numbers In-Reply-To: References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> Message-ID: On Tue, 3 Apr 2018 at 11:18 Barry Warsaw wrote: > On Apr 3, 2018, at 05:51, Paul G wrote: > > > Switching to CalVer is a pretty clear sign that there is now a "rolling > backwards compatibility window", and it allows Python to skip right over > the mythical "Python 4" and directly to "Python 21". Additionally, since > the version number will be trivially predictable, deprecation warnings can > actually include the version after which they will be dropped - so if a > feature is slated to be removed 5 years after it is initially deprecated, > just take the deprecation release version and add 5. > > Changing the versioning scheme is a topic that comes up every now and > then, and I think it?s worth exploring, but I also don?t think we should do > anything about it until the EOL of Python 2.7 at the earliest (if ever). > > That said, and assuming we keep the current scheme, I think there?s a > natural place to label ?Python 4? - we have to break the C API to get rid > of the GIL and/or adopt GC over refcounting, or something of that nature. > I think there is no interest or appetite for a Python source level breaking > change like 2->3, so I wouldn?t expect anything more than the usual changes > at the Python source level for 3.x->4. Of course, you?d like to mitigate > the breakage of extension modules as much as possible, but should that be > unavoidable, then it would warrant a first digit version bump. > > OTOH, some calver-like scheme would be interesting too if we shorten the > release cycle. Could we release a new Python version every 12 months > instead of 18? Can we adopt a time-based release policy, so that whatever > gets in, gets in, and the rest has to wait until the next release? That?s > certainly more painful when that wait is 18 months rather than something > shorter. But if releases come more quickly, that has implications for the > deprecation policy too. And it puts pressure on the second digit because > something like Python 3.53 is distasteful (especially because it would be > easily confused with 3.5.3). Python 21.12 anyone? :) > Are we at the PEP/language summit topic point yet in this discussion since Guido has said he's not interested in changing the status quo? ;) Versioning is like naming variables, so this thread could go on forever. -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Tue Apr 3 16:16:14 2018 From: barry at python.org (Barry Warsaw) Date: Tue, 3 Apr 2018 13:16:14 -0700 Subject: [Python-Dev] Python version numbers In-Reply-To: References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> Message-ID: <6C655688-78E4-42EA-92FF-B77B84C85996@python.org> On Apr 3, 2018, at 13:08, Brett Cannon wrote: > Are we at the PEP/language summit topic point yet in this discussion since Guido has said he's not interested in changing the status quo? ;) Versioning is like naming variables, so this thread could go on forever. Yeah probably so. And if you count the multiyear hiatus between reboots on this topic, yes it will likely go on forever. :) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From ethan at stoneleaf.us Tue Apr 3 17:50:16 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 03 Apr 2018 14:50:16 -0700 Subject: [Python-Dev] Python version numbers In-Reply-To: <6C655688-78E4-42EA-92FF-B77B84C85996@python.org> References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> <6C655688-78E4-42EA-92FF-B77B84C85996@python.org> Message-ID: <5AC3F718.9070508@stoneleaf.us> On 04/03/2018 01:16 PM, Barry Warsaw wrote: > On Apr 3, 2018, at 13:08, Brett Cannon wrote: >> Are we at the PEP/language summit topic point yet in this discussion >> since Guido has said he's not interested in changing the status quo? >> ;) Versioning is like naming variables, so this thread could go on >> forever. > > Yeah probably so. And if you count the multiyear hiatus between reboots > on this topic, yes it will likely go on forever. :) Ah, so someone (Chris? ;) needs to write a PEP so it can be rejected, then? -- ~Ethan~ From robb at datalogics.com Tue Apr 3 13:29:36 2018 From: robb at datalogics.com (Rob Boehne) Date: Tue, 3 Apr 2018 17:29:36 +0000 Subject: [Python-Dev] Hi, I am amar :) In-Reply-To: References: Message-ID: <5548467A-25E4-41F4-A3F9-9AB12FD9CF24@datalogics.com> Amar, Loved that joke! I had a quick look at these PR?s and commented. I do not see any worthwhile improvement to Python by merging these, and there are significant problems with the details in both patches. Hopefully your intent was to learn about the process of contributing to Python, and and the language itself. In that vein, I treated them seriously and hopefully gave you useful feedback. Thanks, Rob Boehne From: Python-Dev on behalf of amar via Python-Dev Reply-To: amar Date: Tuesday, April 3, 2018 at 10:42 AM To: "python-dev at python.org" Subject: [Python-Dev] Hi, I am amar :) Hi, all. Quick joke, Do you know why all functional programmers are anarchists? Because they want to get rid of the state! :D I know there are a lot more important issues than this, but i feel this is important too. I wish some people could take a look at Python/Cpython pull request #6343[1] and the reopened PR #6362[2]. Little background here, I am a science student working on GIS(Geospatial Information Systems) Web Processing Service as a hobby project. I am a human too, i like jokes. But not at the expense of truth. I hope i can make more meaningful contributions in the future! [1] https://github.com/python/cpython/pull/6343 [2] https://github.com/python/cpython/pull/6362#issuecomment-378174084 -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Tue Apr 3 19:34:58 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 03 Apr 2018 16:34:58 -0700 Subject: [Python-Dev] in Message-ID: <5AC40FA2.4080503@stoneleaf.us> This behavior was recently brought to my attention [1]: --> 1 in 'hello' Traceback (most recent call last): File "", line 1, in TypeError: 'in ' requires string as left operand, not int However, in any other collection (set, dict, list, tuple, etc), the answer would be False. Does anyone remember the reason why an exception is raised in the string instance instead of returning False? -- ~Ethan~ [1] https://bugs.python.org/msg314900 From jcgoble3 at gmail.com Tue Apr 3 19:43:29 2018 From: jcgoble3 at gmail.com (Jonathan Goble) Date: Tue, 03 Apr 2018 23:43:29 +0000 Subject: [Python-Dev] in In-Reply-To: <5AC40FA2.4080503@stoneleaf.us> References: <5AC40FA2.4080503@stoneleaf.us> Message-ID: On Tue, Apr 3, 2018 at 7:34 PM Ethan Furman wrote: > This behavior was recently brought to my attention [1]: > > --> 1 in 'hello' > Traceback (most recent call last): > File "", line 1, in > TypeError: 'in ' requires string as left operand, not int > > However, in any other collection (set, dict, list, tuple, etc), the answer > would be False. > > Does anyone remember the reason why an exception is raised in the string > instance instead of returning False? > If I had to hazard a guess, I'd say it's because strings by definition, in the sense that they are a container, can only contain strings of length 1, whereas the other containers you listed can contain a wide variety of Python objects (in some cases, any Python object). Thus it's easy to detect and raise the TypeError. Note that `[] in set()` also raises TypeError, so it's consistent: values of types that are impossible for the container to contain generally raise TypeError (in this case, an unhashable type and a container that only accepts hashable types). (I may be oversimplifying this a bit, but I think the basic idea is right.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Tue Apr 3 19:47:45 2018 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Apr 2018 16:47:45 -0700 Subject: [Python-Dev] in In-Reply-To: <5AC40FA2.4080503@stoneleaf.us> References: <5AC40FA2.4080503@stoneleaf.us> Message-ID: It's because when the rhs is a string, 'in' tests for a substring rather than simple containment. E.g. "ab" in "abc" gives True. So here 'in' is not a collection membership, it only operates on two strings. On Tue, Apr 3, 2018 at 4:34 PM, Ethan Furman wrote: > This behavior was recently brought to my attention [1]: > > --> 1 in 'hello' > Traceback (most recent call last): > File "", line 1, in > TypeError: 'in ' requires string as left operand, not int > > However, in any other collection (set, dict, list, tuple, etc), the answer > would be False. > > Does anyone remember the reason why an exception is raised in the string > instance instead of returning False? > > -- > ~Ethan~ > > > > [1] https://bugs.python.org/msg314900 > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido% > 40python.org > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From schesis at gmail.com Tue Apr 3 19:53:48 2018 From: schesis at gmail.com (Zero Piraeus) Date: Tue, 3 Apr 2018 20:53:48 -0300 Subject: [Python-Dev] in In-Reply-To: <5AC40FA2.4080503@stoneleaf.us> References: <5AC40FA2.4080503@stoneleaf.us> Message-ID: : On 3 April 2018 at 20:34, Ethan Furman wrote: > This behavior was recently brought to my attention [1]: > > --> 1 in 'hello' > Traceback (most recent call last): > File "", line 1, in > TypeError: 'in ' requires string as left operand, not int > > However, in any other collection (set, dict, list, tuple, etc), the answer > would be False. > > Does anyone remember the reason why an exception is raised in the string > instance instead of returning False? The comparison doesn't seem to me to be valid: the 'in' operator for all of those other collections tests whether an element is a member of a collection, whereas for a string it tests whether a string is a substring of another string. In the first case, arbitrary objects can be members, but e.g. [2, 3, 4] in [1, 2, 3, 4, 5] is False. In the second case, no non-string can ever be 'in' a string, but 'bcd' in 'abcde' is True. It's kinda-sorta like addition for numerics versus sequences; they do different things. -[]z. From python at mrabarnett.plus.com Tue Apr 3 20:12:14 2018 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 4 Apr 2018 01:12:14 +0100 Subject: [Python-Dev] in In-Reply-To: <5AC40FA2.4080503@stoneleaf.us> References: <5AC40FA2.4080503@stoneleaf.us> Message-ID: <7ecc256b-e325-7aec-c3fa-db7f0480d625@mrabarnett.plus.com> On 2018-04-04 00:34, Ethan Furman wrote: > This behavior was recently brought to my attention [1]: > > --> 1 in 'hello' > Traceback (most recent call last): > File "", line 1, in > TypeError: 'in ' requires string as left operand, not int > > However, in any other collection (set, dict, list, tuple, etc), the answer would be False. > > Does anyone remember the reason why an exception is raised in the string instance instead of returning False? > Well, strings aren't really a collection like set, etc, which can contain various types, even a mixture of types. A string can contain only strings (including codepoints). A bytestring can contain only bytestrings and ints (and there's been debate on whether the latter was a good idea!). From steve at pearwood.info Tue Apr 3 22:18:31 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 4 Apr 2018 12:18:31 +1000 Subject: [Python-Dev] Hi, I am amar :) In-Reply-To: References: Message-ID: <20180404021831.GN16661@ando.pearwood.info> Hi Amar, and welcome. You might find that you get a more productive response if you write a meaningful subject line and actually tell us what the PRs are about. Nobody has the time or interest to look at every single PR, so if you want to bring your PRs to the attention of those who will care about them, without annoy those who don't care, giving more than just a PR number and link is important. (Also, a subject line that looks less like spam and is more informative will help too.) Thanks. -- Steve From J.Demeyer at UGent.be Wed Apr 4 04:58:36 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Wed, 4 Apr 2018 10:58:36 +0200 Subject: [Python-Dev] Are undocumented functions part of the stable ABI? Message-ID: <5AC493BC.8050104@UGent.be> Hello, I have a question about PEP 384: can undocumented functions ever be considered as part of the stable ABI? With undocumented, I mean not appearing in the "Python/C API Reference Manual". Whatever the answer to this question is, it would be good to make it explicit in PEP 384. I am in particular asking about functions starting with PyCFunction_ appearing in Include/methodobject.h Thanks, Jeroen. From agnosticdev at gmail.com Wed Apr 4 07:21:03 2018 From: agnosticdev at gmail.com (Agnostic Dev) Date: Wed, 4 Apr 2018 06:21:03 -0500 Subject: [Python-Dev] Python Bug Tracker Message-ID: Apologies if this is not the correct place to report this, but I am seeing outages on bugs.python.org. I am in the Chicago area so maybe this is a regional issue or possibly the site is being update, but I wanted to report it. Matt Eaton (Agnosticdev) -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bpo_down.png Type: image/png Size: 30411 bytes Desc: not available URL: From guido at python.org Wed Apr 4 11:56:30 2018 From: guido at python.org (Guido van Rossum) Date: Wed, 4 Apr 2018 08:56:30 -0700 Subject: [Python-Dev] Are undocumented functions part of the stable ABI? In-Reply-To: <5AC493BC.8050104@UGent.be> References: <5AC493BC.8050104@UGent.be> Message-ID: It would be helpful if you explained the context of your request. E.g. did you have some code that broke due to a change in one of these functions, or are you contemplating a change to one of them? And specifics of what broke or what you're contemplating. On Wed, Apr 4, 2018 at 1:58 AM, Jeroen Demeyer wrote: > Hello, > > I have a question about PEP 384: can undocumented functions ever be > considered as part of the stable ABI? With undocumented, I mean not > appearing in the "Python/C API Reference Manual". > > Whatever the answer to this question is, it would be good to make it > explicit in PEP 384. > > I am in particular asking about functions starting with PyCFunction_ > appearing in Include/methodobject.h > > > Thanks, > Jeroen. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido% > 40python.org > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From nad at python.org Wed Apr 4 12:59:03 2018 From: nad at python.org (Ned Deily) Date: Wed, 4 Apr 2018 12:59:03 -0400 Subject: [Python-Dev] Python Bug Tracker In-Reply-To: References: Message-ID: <9E588E46-00DC-455A-8810-697A9F31F720@python.org> On Apr 4, 2018, at 07:21, Agnostic Dev wrote: > Apologies if this is not the correct place to report this, but I am seeing outages on bugs.python.org. > I am in the Chicago area so maybe this is a regional issue or possibly the site is being update, but I wanted to report it. You are not alone. There have been intermittent TLS connection issues with bugs.python.org for some time now. The issue is being tracked here: https://github.com/python/psf-infra-meta/issues/4 -- Ned Deily nad at python.org -- [] From nad at python.org Wed Apr 4 13:14:19 2018 From: nad at python.org (Ned Deily) Date: Wed, 4 Apr 2018 13:14:19 -0400 Subject: [Python-Dev] Python Bug Tracker In-Reply-To: <9E588E46-00DC-455A-8810-697A9F31F720@python.org> References: <9E588E46-00DC-455A-8810-697A9F31F720@python.org> Message-ID: <2970429C-37E8-4B09-AD4B-3C34E7B1D957@python.org> On Apr 4, 2018, at 12:59, Ned Deily wrote: > On Apr 4, 2018, at 07:21, Agnostic Dev wrote: >> Apologies if this is not the correct place to report this, but I am seeing outages on bugs.python.org. >> I am in the Chicago area so maybe this is a regional issue or possibly the site is being update, but I wanted to report it. > You are not alone. There have been intermittent TLS connection issues with bugs.python.org for some time now. The issue is being tracked here: > > https://github.com/python/psf-infra-meta/issues/4 Ah, but beyond that, I see now that bugs.python.org had a major outage earlier today. Should be OK now. -- Ned Deily nad at python.org -- [] From nad at python.org Wed Apr 4 14:09:04 2018 From: nad at python.org (Ned Deily) Date: Wed, 4 Apr 2018 14:09:04 -0400 Subject: [Python-Dev] gdb support could use some love Message-ID: <08353CAE-4322-4B6E-9881-8D6AF96FFEEE@python.org> There are a bunch of open issues regarding gdb support including one with a PR in need of review for 3.6+. I imagine that there are duplicates at least among the more recent open issues. If you use and are familiar with the gdb macros, it would be really swell if you could take some time to review some of those open issues. Thanks! --Ned https://bugs.python.org/issue29673 https://github.com/python/cpython/pull/6126 https://bugs.python.org/issue?%40search_text=&ignore=file%3Acontent&title=gdb&%40columns=title&id=&%40columns=id&stage=&creation=&creator=&activity=&%40columns=activity&%40sort=activity&actor=&nosy=&type=&components=&versions=&dependencies=&assignee=&keywords=&priority=&status=1&%40columns=status&resolution=&nosy_count=&message_count=&%40group=&%40pagesize=50&%40startwith=0&%40sortdir=on&%40action=search -- Ned Deily nad at python.org -- [] From ethan at stoneleaf.us Wed Apr 4 14:32:34 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 04 Apr 2018 11:32:34 -0700 Subject: [Python-Dev] Enum, Flag, __contains__, and False vs TypeError Message-ID: <5AC51A42.8080600@stoneleaf.us> API question. Background: ---------- When doing checks such as --> 3 in [4, 5, 6] --> 'test' in {'test':True, 'live':False} the result is True or False. When doing checks such as --> 3 in 'hello world' --> [4, 5, 6] in {'test':True, 'live':False} the result is a TypeError. The general rule seems to be that if it is impossible for the in-question object to be in the container object then a TypeError is raised, otherwise True or False is returned. Question 1: ---------- (A) A standard Enum class is a container of Enum members. It cannot hold anything else. However, it has been returning False both in cases where the in-question object was an Enum of a different class (a Fruit in a Color, for example) and when the in-question object was not even an Enum ('apple' in Fruit, for example). (B) The waters get even more muddied when Fruit has a str mixin, so `Fruit.APPLE == 'apple' is True` -- in that case, should `'orange' in Fruit` return True, False, or raise TypeError? Question 2: ---------- (A) The new Flag type allows `in` tests on the members themselves; so, for example: --> SomeFlag.ONE in SomeFlag.One|SomeFlag.TWO True The question, of course, is what to do when a non-Flag member is tested for: --> 'apple' in SomeFlag.ONE # False or TypeError? --> 2 in SomeFlag.TWO # True or TypeError? (B) And, of course, the same muddier question arises with IntFlag, where SomeFlag.TWO == 2 is True. My thoughts: ----------- For question 1A (pure Enum): I'm thinking a TypeError should be raised when the in-question object is not an Enum member of any kind -- it simply is not possible for that object to ever be in an Enum, and is surely a bug. For question 1B (mixed Enum): if 1A is TypeError, then 1B should also be TypeError at least for non-mixin in-question types (so checking for 1 in StrEnum would be a TypeError), but I'm torn between TypeError and True/False for cases where the in-question type matches the mixin type ('apple' in StrEnum).... On the one hand, even though an Enum member might be equal to some other type, that other type will not have the Enum attributes, etc, and a True answer would lead one to believe you could access `.name` and `.value`, etc., while a False answer would lead one to believe there was no match even though equality tests pass; on the other hand, how strong is the "container" aspect of a mixed Enum? How often is the test `'apple' in Fruit` meant to discover if you have a Fruit member vs whether you have something that could be a Fruit member? Also, how important is it to be consistent with IntFlag, which I definitely think should return True/False for int checks? For question 2A (pure Flag): I'm comfortable sticking with a TypeError (assuming we switch to TypeError for 1A). For question 2B (int Flag): I think a TypeError if the in-question object is not an int is appropriate (assuming TypeError for 1A and 2A), but if it is an int, True/False seems the better path. My reasoning being that Flag and IntFlag are more similar to sets than lists, and IntFlag is specifically meant to work with ints, and a test of `2 in some_int_flags` is more concerned with a flag being set than with .name or .value attributes that may or may not exist on the in-question object. Any and all thoughts appreciated. -- ~Ethan~ From brett at python.org Wed Apr 4 15:00:11 2018 From: brett at python.org (Brett Cannon) Date: Wed, 04 Apr 2018 19:00:11 +0000 Subject: [Python-Dev] Enum, Flag, __contains__, and False vs TypeError In-Reply-To: <5AC51A42.8080600@stoneleaf.us> References: <5AC51A42.8080600@stoneleaf.us> Message-ID: On Wed, 4 Apr 2018 at 11:30 Ethan Furman wrote: > API question. > > Background: > ---------- > > When doing checks such as > > --> 3 in [4, 5, 6] > --> 'test' in {'test':True, 'live':False} > > the result is True or False. > > When doing checks such as > > --> 3 in 'hello world' > --> [4, 5, 6] in {'test':True, 'live':False} > > the result is a TypeError. > > The general rule seems to be that if it is impossible for the in-question > object to be in the container object then a > TypeError is raised, otherwise True or False is returned. > > > Question 1: > ---------- > > (A) A standard Enum class is a container of Enum members. It cannot hold > anything else. However, it has been returning > False both in cases where the in-question object was an Enum of a > different class (a Fruit in a Color, for example) and > when the in-question object was not even an Enum ('apple' in Fruit, for > example). > > (B) The waters get even more muddied when Fruit has a str mixin, so > `Fruit.APPLE == 'apple' is True` -- in that case, > should `'orange' in Fruit` return True, False, or raise TypeError? > > > Question 2: > ---------- > > (A) The new Flag type allows `in` tests on the members themselves; so, for > example: > > --> SomeFlag.ONE in SomeFlag.One|SomeFlag.TWO > True > > The question, of course, is what to do when a non-Flag member is tested > for: > > --> 'apple' in SomeFlag.ONE > # False or TypeError? > > --> 2 in SomeFlag.TWO > # True or TypeError? > > (B) And, of course, the same muddier question arises with IntFlag, where > SomeFlag.TWO == 2 is True. > > > My thoughts: > ----------- > > For question 1A (pure Enum): I'm thinking a TypeError should be raised > when the in-question object is not an Enum > member of any kind -- it simply is not possible for that object to ever be > in an Enum, and is surely a bug. > Yes, although it sounds like it already does the other thing, so do you want to start raising TypeError immediately or go for a deprecation period? > > For question 1B (mixed Enum): if 1A is TypeError, then 1B should also be > TypeError at least for non-mixin in-question > types (so checking for 1 in StrEnum would be a TypeError), but I'm torn > between TypeError and True/False for cases where > the in-question type matches the mixin type ('apple' in StrEnum).... On > the one hand, even though an Enum member might > be equal to some other type, that other type will not have the Enum > attributes, etc, and a True answer would lead one to > believe you could access `.name` and `.value`, etc., while a False answer > would lead one to believe there was no match > even though equality tests pass; on the other hand, how strong is the > "container" aspect of a mixed Enum? How often is > the test `'apple' in Fruit` meant to discover if you have a Fruit member > vs whether you have something that could be a > Fruit member? Also, how important is it to be consistent with IntFlag, > which I definitely think should return > True/False for int checks? > I think this is a design question as to how interchangeable you want these kind of enums to be, and I personally have no opinion as I have never needed to use them. > > For question 2A (pure Flag): I'm comfortable sticking with a TypeError > (assuming we switch to TypeError for 1A). > Makes sense. > > For question 2B (int Flag): I think a TypeError if the in-question object > is not an int is appropriate (assuming > TypeError for 1A and 2A), but if it is an int, True/False seems the better > path. My reasoning being that Flag and > IntFlag are more similar to sets than lists, and IntFlag is specifically > meant to work with ints, and a test of `2 in > some_int_flags` is more concerned with a flag being set than with .name or > .value attributes that may or may not exist > on the in-question object. > Same as above; no experience so no opinion. > > Any and all thoughts appreciated. > > -- > ~Ethan~ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Wed Apr 4 15:40:09 2018 From: barry at python.org (Barry Warsaw) Date: Wed, 4 Apr 2018 12:40:09 -0700 Subject: [Python-Dev] Enum, Flag, __contains__, and False vs TypeError In-Reply-To: <5AC51A42.8080600@stoneleaf.us> References: <5AC51A42.8080600@stoneleaf.us> Message-ID: On Apr 4, 2018, at 11:32, Ethan Furman wrote: > > (A) A standard Enum class is a container of Enum members. It cannot hold anything else. However, it has been returning False both in cases where the in-question object was an Enum of a different class (a Fruit in a Color, for example) and when the in-question object was not even an Enum ('apple' in Fruit, for example). > > (B) The waters get even more muddied when Fruit has a str mixin, so `Fruit.APPLE == 'apple' is True` -- in that case, should `'orange' in Fruit` return True, False, or raise TypeError? Are you proposing to change current behavior, and if so, what?s your deprecation plan? I?m not sure I feel that the purity is important enough to change how it currently works, especially since you?ll have to be prepared to catch exceptions rather than just handle the boolean value. OTOH, since most of my use cases are comparisons against explicit enum values, I?m not sure how often people write code to check for enum values contained in the Enum (I do it in one or two places where I?m deserializing the actual value object, e.g. from a pickle). (FWIW, I encourage individual comparisons use `is` rather than `==`.) > Question 2: > ---------- > > (A) The new Flag type allows `in` tests on the members themselves; so, for example: > > --> SomeFlag.ONE in SomeFlag.One|SomeFlag.TWO > True > > The question, of course, is what to do when a non-Flag member is tested for: > > --> 'apple' in SomeFlag.ONE > # False or TypeError? > > --> 2 in SomeFlag.TWO > # True or TypeError? > > (B) And, of course, the same muddier question arises with IntFlag, where SomeFlag.TWO == 2 is True. Well, now I?m confused: Python 3.7.0b2+ (heads/3.7:f328caf4ca, Mar 26 2018, 19:57:33) [Clang 9.0.0 (clang-900.0.39.2)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from enum import IntFlag >>> class Flag(IntFlag): ... a = 1 ... b = 2 ... c = 4 ... d = 8 ... >>> 'foo' in (Flag.a|Flag.b) True Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From storchaka at gmail.com Wed Apr 4 16:11:42 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Wed, 4 Apr 2018 23:11:42 +0300 Subject: [Python-Dev] Enum, Flag, __contains__, and False vs TypeError In-Reply-To: <5AC51A42.8080600@stoneleaf.us> References: <5AC51A42.8080600@stoneleaf.us> Message-ID: I expect that for IntFlag `x in flags` is equivalent either to bool(x | flags.value) or to (x | flags.value) == x It should return some result only if x is an integer (or compatible with integers) and raise a TypeError otherwise. Don't add any special checks unless there are good reasons for this. Keep the code as simple as possible. From ethan at stoneleaf.us Wed Apr 4 16:25:42 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 04 Apr 2018 13:25:42 -0700 Subject: [Python-Dev] Enum, Flag, __contains__, and False vs TypeError In-Reply-To: References: <5AC51A42.8080600@stoneleaf.us> Message-ID: <5AC534C6.1090601@stoneleaf.us> On 04/04/2018 12:40 PM, Barry Warsaw wrote: > On Apr 4, 2018, at 11:32, Ethan Furman wrote: >> (A) A standard Enum class is a container of Enum members. It cannot hold anything else... >> >> (B) The waters get even more muddied when Fruit has a str mixin, so `Fruit.APPLE == 'apple' is True`... > > Are you proposing to change current behavior, and if so, what?s your deprecation plan? The first problem is to determine what the correct behavior should be -- if that turns out to be different than what is we can then decide the time frame for changing it. ;) > I?m not sure I feel that the purity is important enough to change how it > currently works, especially since you?ll have to be prepared to catch > exceptions rather than just handle the boolean value. It's not so much about purity as about bugs not passing silently. But whether or not it's a bug depends on how "containerish" an Enum class is (or is perceived to be). > OTOH, since most of my > use cases are comparisons against explicit enum values, I?m not sure how often > people write code to check for enum values contained in the Enum (I do it in > one or two places where I?m deserializing the actual value object, e.g. from a > pickle). I write a bunch of cli scripts for work, so I'm often comparing a string value with an Enum value. As often as not, I don't need the member itself, just to know that the string I have can be converted to a valid Enum -- `if "quick" in Method`, for example -- but just converting to an Enum is undoubtedly the better course of action . > (FWIW, I encourage individual comparisons use `is` rather than `==`.) Which is great for pure Enums, not so much for mixed. >> Question 2: >> ---------- >> >> (A) The new Flag type allows `in` tests on the members themselves; so, for example: >> >> --> SomeFlag.ONE in SomeFlag.One|SomeFlag.TWO >> True >> >> The question, of course, is what to do when a non-Flag member is tested for: >> >> --> 'apple' in SomeFlag.ONE >> # False or TypeError? >> >> --> 2 in SomeFlag.TWO >> # True or TypeError? >> >> (B) And, of course, the same muddier question arises with IntFlag, where SomeFlag.TWO == 2 is True. > > Well, now I?m confused: > > Python 3.7.0b2+ (heads/3.7:f328caf4ca, Mar 26 2018, 19:57:33) > [Clang 9.0.0 (clang-900.0.39.2)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > --> from enum import IntFlag > --> class Flag(IntFlag): > ... a = 1 > ... b = 2 > ... c = 4 > ... d = 8 > ... > --> 'foo' in (Flag.a|Flag.b) > True Already being tracked at https://bugs.python.org/issue33217, which is what got me thinking about this whole issue. True is obviously wrong, but should the correct answer be False, or TypeError? -- ~Ethan~ From christian at python.org Wed Apr 4 16:38:31 2018 From: christian at python.org (Christian Heimes) Date: Wed, 4 Apr 2018 22:38:31 +0200 Subject: [Python-Dev] ssl module and LibreSSL CVE-2018-8970 Message-ID: Hi, I like to share the story of a critical security bug with you. Contrary to other issues in TLS/SSL, it's a story with happy ending. Nobody was harmed. The bug was fixed before it affected the general population. Introduction ------------ Python's ssl.match_hostname() function was a source of several CVEs and other security bugs. After a long struggle, I decided to drop support for old OpenSSL releases and uses a new OpenSSL method to offload host name verification to OpenSSL. The improvement [1] eventually landed in Python 3.7. Nowadays OpenSSL verifies host name or IP address during the TLS/SSL handshake. Later I discovered that LibreSSL <= 2.6 did not have X509_VERIFY_PARAM_set1_host() [2]. We had to temporarily suspend support for LibreSSL. About two months later, LibreSSL caught up and released version 2.7.0 with support for the function. The bug ------- One day after the release of LibreSSL 2.7.0, I started to port Python 3.7 to LibreSSL. In matter of minutes I got the ssl module to compile and work with LibreSSL. All tests were passing -- except for negative the host name verification tests. LibreSSL was accepting all invalid host names as correct! Python's vigorous test suite had discovered a critical security bug in LibreSSL. It turned out that LibreSSL copied the implementation of X509_VERIFY_PARAM_set1_host(param, name, namelen) from BoringSSL and the documentation from OpenSSL. BoringSSL's implementation didn't support the special case of 0 as namelen parammeter. OpenSSL supports namelen = 0, which is interpreted as namelen=strlen(name). It is documented in OpenSSL's man page and was even recommended on OpenSSL's wiki as preferred way. Happy Ending ------------ So I got in contact with LibreSSL's security team and BoringSSL's security team [3]. Less than a day later, both libraries released fixes for the bug [4]. Mitre has assigned CVE-2018-8970 [5] to the bug. Disaster averted! BoringSSL's security issue [3] contains more information. Adam Langley lifted the restriction about an hour ago. I like to thank Bob Beck (LibreSSL), Adam Langley (Google) and David Benjamin (Google) for their assistance and cooperation. Regards, Christian [1] https://bugs.python.org/issue31399 [2] https://github.com/libressl-portable/portable/issues/381 [3] https://bugs.chromium.org/p/chromium/issues/detail?id=824799 [4] https://www.libressl.org/releases.html [5] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-8970 From v+python at g.nevcal.com Wed Apr 4 16:24:15 2018 From: v+python at g.nevcal.com (Glenn Linderman) Date: Wed, 4 Apr 2018 13:24:15 -0700 Subject: [Python-Dev] Enum, Flag, __contains__, and False vs TypeError In-Reply-To: <5AC51A42.8080600@stoneleaf.us> References: <5AC51A42.8080600@stoneleaf.us> Message-ID: On 4/4/2018 11:32 AM, Ethan Furman wrote: > API question. > > Background: > ---------- > > When doing checks such as > > --> 3 in [4, 5, 6] > --> 'test' in {'test':True, 'live':False} > > the result is True or False. > > When doing checks such as > > --> 3 in 'hello world' > --> [4, 5, 6] in {'test':True, 'live':False} > > the result is a TypeError. > > The general rule seems to be that if it is impossible for the > in-question object to be in the container object then a TypeError is > raised, otherwise True or False is returned. > > > Question 1: > ---------- > > (A) A standard Enum class is a container of Enum members.? It cannot > hold anything else.? However, it has been returning False both in > cases where the in-question object was an Enum of a different class (a > Fruit in a Color, for example) and when the in-question object was not > even an Enum ('apple' in Fruit, for example). > > (B) The waters get even more muddied when Fruit has a str mixin, so > `Fruit.APPLE == 'apple' is True` -- in that case, should `'orange' in > Fruit` return True, False, or raise TypeError? > > > Question 2: > ---------- > > (A) The new Flag type allows `in` tests on the members themselves; so, > for example: > > --> SomeFlag.ONE in SomeFlag.One|SomeFlag.TWO > True > > The question, of course, is what to do when a non-Flag member is > tested for: > > --> 'apple' in SomeFlag.ONE > # False or TypeError? > > --> 2 in SomeFlag.TWO > # True or TypeError? > > (B) And, of course, the same muddier question arises with IntFlag, > where SomeFlag.TWO == 2 is True. > > > My thoughts: > ----------- > > For question 1A (pure Enum):? I'm thinking a TypeError should be > raised when the in-question object is not an Enum member of any kind > -- it simply is not possible for that object to ever be in an Enum, > and is surely a bug. > > For question 1B (mixed Enum):? if 1A is TypeError, then 1B should also > be TypeError at least for non-mixin in-question types (so checking for > 1 in StrEnum would be a TypeError), but I'm torn between TypeError and > True/False for cases where the in-question type matches the mixin type > ('apple' in StrEnum)....? On the one hand, even though an Enum member > might be equal to some other type, that other type will not have the > Enum attributes, etc, and a True answer would lead one to believe you > could access `.name` and `.value`, etc., while a False answer would > lead one to believe there was no match even though equality tests > pass; on the other hand, how strong is the "container" aspect of a > mixed Enum?? How often is the test `'apple' in Fruit` meant to > discover if you have a Fruit member vs whether you have something that > could be a Fruit member?? Also, how important is it to be consistent > with IntFlag, which I definitely think should return True/False for > int checks? > > For question 2A (pure Flag):? I'm comfortable sticking with a > TypeError (assuming we switch to TypeError for 1A). > > For question 2B (int Flag):? I think a TypeError if the in-question > object is not an int is appropriate (assuming TypeError for 1A and > 2A), but if it is an int, True/False seems the better path.? My > reasoning being that Flag and IntFlag are more similar to sets than > lists, and IntFlag is specifically meant to work with ints, and a test > of `2 in some_int_flags` is more concerned with a flag being set than > with .name or .value attributes that may or may not exist on the > in-question object. > > Any and all thoughts appreciated. I think the "in" test should raise TypeError if tested against _anything_ that is not an Enum member. Why? I see a parallel between Enum and mappings. x = {'test':True, 'live':False} So it is True that? 'test' in x?? and? 'live' in x? and False that True in x and False in x. It is False that? 'foo' in x? and? 3 in x It is TypeError that [4,5,6] in x, or {'foo': 'bar'} in x. Note that it is False that (4,5,6) in x? which is a little surprising given the above two, but not when you realize the differences between this and the above two. So with mappings, you can have any hashable type as a key: with Enum, you can only have Enum members as keys. So I have no idea why you would want to return False, rather than TypeError, other than (1) the distinction probably doesn't matter to most people (2) backward compatibility. I would find?? 2 in some_int_flags? being True surprising. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Wed Apr 4 17:07:50 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 04 Apr 2018 14:07:50 -0700 Subject: [Python-Dev] Enum, Flag, __contains__, and False vs TypeError In-Reply-To: References: <5AC51A42.8080600@stoneleaf.us> Message-ID: <5AC53EA6.5050301@stoneleaf.us> On 04/04/2018 01:24 PM, Glenn Linderman wrote: > I think the "in" test should raise TypeError if tested against _anything_ that is not an Enum member. > > Why? I see a parallel between Enum and mappings. > > x = {'test':True, 'live':False} > > So it is True that 'test' in x and 'live' in x and False that True in x and False in x. > > It is False that 'foo' in x and 3 in x > > It is TypeError that [4,5,6] in x, or {'foo': 'bar'} in x. > > Note that it is False that (4,5,6) in x which is a little surprising given the above two, but not when you realize the > differences between this and the above two. > > So with mappings, you can have any hashable type as a key: with Enum, you can only have Enum members as keys. Technically, an Enum has strings as keys (the member names), while the members themselves are the values; but list()ing an Enum returns the values (members) while a mapping would return the names (keys); Enums are definitely a bit strange! > So I have no idea why you would want to return False, rather than TypeError, other than (1) the distinction probably > doesn't matter to most people (2) backward compatibility. To me, `in` is an equality test, and equality tests should never, ever raise exceptions -- they should return True or False. Which is why I was surprised that `1 in "hello"` raised instead of returning False. However, I have learned that in certain situations, such as str or dict or set, if it is impossible to contain the item being searched for then raising an exception is appropriate. Hence my questions now on deciding what the behavior /should/ be, and whether practicality has a log to stand on in this case. ;) -- ~Ethan~ From guido at python.org Wed Apr 4 17:12:17 2018 From: guido at python.org (Guido van Rossum) Date: Wed, 4 Apr 2018 14:12:17 -0700 Subject: [Python-Dev] Enum, Flag, __contains__, and False vs TypeError In-Reply-To: <5AC53EA6.5050301@stoneleaf.us> References: <5AC51A42.8080600@stoneleaf.us> <5AC53EA6.5050301@stoneleaf.us> Message-ID: I don't think of "in" as an equality test -- it's special just like <, <=, >, >= are special, and those can certainly raise TypeError. On Wed, Apr 4, 2018 at 2:07 PM, Ethan Furman wrote: > On 04/04/2018 01:24 PM, Glenn Linderman wrote: > > I think the "in" test should raise TypeError if tested against _anything_ >> that is not an Enum member. >> >> Why? I see a parallel between Enum and mappings. >> >> x = {'test':True, 'live':False} >> >> So it is True that 'test' in x and 'live' in x and False that True >> in x and False in x. >> >> It is False that 'foo' in x and 3 in x >> >> It is TypeError that [4,5,6] in x, or {'foo': 'bar'} in x. >> >> Note that it is False that (4,5,6) in x which is a little surprising >> given the above two, but not when you realize the >> differences between this and the above two. >> >> So with mappings, you can have any hashable type as a key: with Enum, you >> can only have Enum members as keys. >> > > Technically, an Enum has strings as keys (the member names), while the > members themselves are the values; but list()ing an Enum returns the values > (members) while a mapping would return the names (keys); Enums are > definitely a bit strange! > > So I have no idea why you would want to return False, rather than >> TypeError, other than (1) the distinction probably >> doesn't matter to most people (2) backward compatibility. >> > > To me, `in` is an equality test, and equality tests should never, ever > raise exceptions -- they should return True or False. Which is why I was > surprised that `1 in "hello"` raised instead of returning False. However, > I have learned that in certain situations, such as str or dict or set, if > it is impossible to contain the item being searched for then raising an > exception is appropriate. > > Hence my questions now on deciding what the behavior /should/ be, and > whether practicality has a log to stand on in this case. ;) > > > -- > ~Ethan~ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido% > 40python.org > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Wed Apr 4 17:36:10 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 04 Apr 2018 14:36:10 -0700 Subject: [Python-Dev] Enum, Flag, __contains__, and False vs TypeError In-Reply-To: References: <5AC51A42.8080600@stoneleaf.us> <5AC53EA6.5050301@stoneleaf.us> Message-ID: <5AC5454A.9020704@stoneleaf.us> On 04/04/2018 02:12 PM, Guido van Rossum wrote: > I don't think of "in" as an equality test -- it's special just like <, <=, > >, >= are special, and those can certainly raise TypeError. Good to know, thank you! -- ~Ethan~ From benjamin at python.org Thu Apr 5 00:46:26 2018 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 04 Apr 2018 21:46:26 -0700 Subject: [Python-Dev] 2.7.15 release schedule Message-ID: <1522903586.3328981.1327160888.2F1CF382@webmail.messagingengine.com> Hi all, It's that time yet again: I'm planning to release 2.7.15 release candidate 1 on April 14 and a final release two weeks later on April 28. From J.Demeyer at UGent.be Thu Apr 5 01:34:55 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Thu, 5 Apr 2018 07:34:55 +0200 Subject: [Python-Dev] Are undocumented functions part of the stable ABI? In-Reply-To: <43a821a6be244016816c02de52fe0126@xmail102.UGent.be> References: <5AC493BC.8050104@UGent.be> <43a821a6be244016816c02de52fe0126@xmail102.UGent.be> Message-ID: <5AC5B57F.1040402@UGent.be> On 2018-04-04 17:56, Guido van Rossum wrote: > It would be helpful if you explained the context of your request. The context is PEP 575. I guess my question is mostly about PyCFunction_Check(). I will not be able to keep it 100% backwards compatible simply because the goal of that PEP is precisely changing the classes of some objects. Now the question is: am I allowed to change the implementation of PyCFunction_Check()? If it's considered part of the stable ABI, then the answer is immediately "no". By the way, does anybody happen to know why the PyCFunction_* functions are undocumented? Is it just an oversight in the docs or is it intentional? But regardless of the context, I think that the question "Are undocumented functions part of the stable ABI?" should be answered in PEP 384. Jeroen. From skip.montanaro at gmail.com Thu Apr 5 05:47:39 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Thu, 5 Apr 2018 04:47:39 -0500 Subject: [Python-Dev] gdb support could use some love In-Reply-To: <08353CAE-4322-4B6E-9881-8D6AF96FFEEE@python.org> References: <08353CAE-4322-4B6E-9881-8D6AF96FFEEE@python.org> Message-ID: > There are a bunch of open issues regarding gdb support including one with a PR in need of review for 3.6+. I rejected one (which assumed everyone now uses a python-aware gdb), commented on another (ceval.c-related name changes in several commands), and created a PR for third (documentation for the user-defined commands). Unfortunately, it's been so long since I contributed, I don't quite understand the ins and outs of the workflow anymore. In particular, I could find no way to add the "skip news" label. I'm afraid someone might have to intervene here: https://github.com/python/cpython/pull/6384 Skip From brett at python.org Thu Apr 5 11:12:41 2018 From: brett at python.org (Brett Cannon) Date: Thu, 05 Apr 2018 15:12:41 +0000 Subject: [Python-Dev] ssl module and LibreSSL CVE-2018-8970 In-Reply-To: References: Message-ID: Nice work! Something to add to our "finding C compiler bugs" list of accomplishments. ? On Wed, Apr 4, 2018, 13:39 Christian Heimes, wrote: > Hi, > > I like to share the story of a critical security bug with you. Contrary > to other issues in TLS/SSL, it's a story with happy ending. Nobody was > harmed. The bug was fixed before it affected the general population. > > > Introduction > ------------ > > Python's ssl.match_hostname() function was a source of several CVEs and > other security bugs. After a long struggle, I decided to drop support > for old OpenSSL releases and uses a new OpenSSL method to offload host > name verification to OpenSSL. The improvement [1] eventually landed in > Python 3.7. Nowadays OpenSSL verifies host name or IP address during the > TLS/SSL handshake. > > Later I discovered that LibreSSL <= 2.6 did not have > X509_VERIFY_PARAM_set1_host() [2]. We had to temporarily suspend support > for LibreSSL. About two months later, LibreSSL caught up and released > version 2.7.0 with support for the function. > > > The bug > ------- > > One day after the release of LibreSSL 2.7.0, I started to port Python > 3.7 to LibreSSL. In matter of minutes I got the ssl module to compile > and work with LibreSSL. All tests were passing -- except for negative > the host name verification tests. LibreSSL was accepting all invalid > host names as correct! Python's vigorous test suite had discovered a > critical security bug in LibreSSL. > > It turned out that LibreSSL copied the implementation of > X509_VERIFY_PARAM_set1_host(param, name, namelen) from BoringSSL and the > documentation from OpenSSL. BoringSSL's implementation didn't support > the special case of 0 as namelen parammeter. OpenSSL supports namelen = > 0, which is interpreted as namelen=strlen(name). It is documented in > OpenSSL's man page and was even recommended on OpenSSL's wiki as > preferred way. > > > Happy Ending > ------------ > > So I got in contact with LibreSSL's security team and BoringSSL's > security team [3]. Less than a day later, both libraries released fixes > for the bug [4]. Mitre has assigned CVE-2018-8970 [5] to the bug. > Disaster averted! > > BoringSSL's security issue [3] contains more information. Adam Langley > lifted the restriction about an hour ago. > > I like to thank Bob Beck (LibreSSL), Adam Langley (Google) and David > Benjamin (Google) for their assistance and cooperation. > > Regards, > Christian > > [1] https://bugs.python.org/issue31399 > [2] https://github.com/libressl-portable/portable/issues/381 > [3] https://bugs.chromium.org/p/chromium/issues/detail?id=824799 > [4] https://www.libressl.org/releases.html > [5] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-8970 > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Thu Apr 5 12:35:48 2018 From: brett at python.org (Brett Cannon) Date: Thu, 05 Apr 2018 16:35:48 +0000 Subject: [Python-Dev] gdb support could use some love In-Reply-To: References: <08353CAE-4322-4B6E-9881-8D6AF96FFEEE@python.org> Message-ID: On Thu, 5 Apr 2018 at 02:49 Skip Montanaro wrote: > > There are a bunch of open issues regarding gdb support including one > with a PR in need of review for 3.6+. > > I rejected one (which assumed everyone now uses a python-aware gdb), > commented on another (ceval.c-related name changes in several > commands), and created a PR for third (documentation for the > user-defined commands). Unfortunately, it's been so long since I > contributed, I don't quite understand the ins and outs of the workflow > anymore. In particular, I could find no way to add the "skip news" > label. There's a "Labels" section in the right-hand column. You can click "Labels" and a drop-down of available and selected labels will show up. -Brett > I'm afraid someone might have to intervene here: > > https://github.com/python/cpython/pull/6384 > > Skip > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nad at python.org Thu Apr 5 13:38:08 2018 From: nad at python.org (Ned Deily) Date: Thu, 5 Apr 2018 13:38:08 -0400 Subject: [Python-Dev] gdb support could use some love In-Reply-To: References: <08353CAE-4322-4B6E-9881-8D6AF96FFEEE@python.org> Message-ID: <7669EEBA-F4FF-442D-99FD-425FA206D315@python.org> On Apr 5, 2018, at 12:35, Brett Cannon wrote: > On Thu, 5 Apr 2018 at 02:49 Skip Montanaro wrote: >> > There are a bunch of open issues regarding gdb support including one with a PR in need of review for 3.6+. >> >> I rejected one (which assumed everyone now uses a python-aware gdb), >> commented on another (ceval.c-related name changes in several >> commands), and created a PR for third (documentation for the >> user-defined commands). Unfortunately, it's been so long since I >> contributed, I don't quite understand the ins and outs of the workflow >> anymore. In particular, I could find no way to add the "skip news" >> label. > There's a "Labels" section in the right-hand column. You can click "Labels" and a drop-down of available and selected labels will show up. Modifying GitHub Labels is only available to people with commit privs and, IIRC, Skip asked to drop his commit privs a few years ago (although I'm sure we would all be happy to welcome him back!). > I'm afraid someone might have to intervene here: > > https://github.com/python/cpython/pull/6384 Thanks for pushing that! It's now merged. -- Ned Deily nad at python.org -- [] From wes.turner at gmail.com Thu Apr 5 13:40:03 2018 From: wes.turner at gmail.com (Wes Turner) Date: Thu, 5 Apr 2018 13:40:03 -0400 Subject: [Python-Dev] ssl module and LibreSSL CVE-2018-8970 In-Reply-To: References: Message-ID: +1. Thanks! Which tests? On Wednesday, April 4, 2018, Christian Heimes wrote: > Hi, > > I like to share the story of a critical security bug with you. Contrary > to other issues in TLS/SSL, it's a story with happy ending. Nobody was > harmed. The bug was fixed before it affected the general population. > > > Introduction > ------------ > > Python's ssl.match_hostname() function was a source of several CVEs and > other security bugs. After a long struggle, I decided to drop support > for old OpenSSL releases and uses a new OpenSSL method to offload host > name verification to OpenSSL. The improvement [1] eventually landed in > Python 3.7. Nowadays OpenSSL verifies host name or IP address during the > TLS/SSL handshake. > > Later I discovered that LibreSSL <= 2.6 did not have > X509_VERIFY_PARAM_set1_host() [2]. We had to temporarily suspend support > for LibreSSL. About two months later, LibreSSL caught up and released > version 2.7.0 with support for the function. > > > The bug > ------- > > One day after the release of LibreSSL 2.7.0, I started to port Python > 3.7 to LibreSSL. In matter of minutes I got the ssl module to compile > and work with LibreSSL. All tests were passing -- except for negative > the host name verification tests. LibreSSL was accepting all invalid > host names as correct! Python's vigorous test suite had discovered a > critical security bug in LibreSSL. > > It turned out that LibreSSL copied the implementation of > X509_VERIFY_PARAM_set1_host(param, name, namelen) from BoringSSL and the > documentation from OpenSSL. BoringSSL's implementation didn't support > the special case of 0 as namelen parammeter. OpenSSL supports namelen = > 0, which is interpreted as namelen=strlen(name). It is documented in > OpenSSL's man page and was even recommended on OpenSSL's wiki as > preferred way. > > > Happy Ending > ------------ > > So I got in contact with LibreSSL's security team and BoringSSL's > security team [3]. Less than a day later, both libraries released fixes > for the bug [4]. Mitre has assigned CVE-2018-8970 [5] to the bug. > Disaster averted! > > BoringSSL's security issue [3] contains more information. Adam Langley > lifted the restriction about an hour ago. > > I like to thank Bob Beck (LibreSSL), Adam Langley (Google) and David > Benjamin (Google) for their assistance and cooperation. > > Regards, > Christian > > [1] https://bugs.python.org/issue31399 > [2] https://github.com/libressl-portable/portable/issues/381 > [3] https://bugs.chromium.org/p/chromium/issues/detail?id=824799 > [4] https://www.libressl.org/releases.html > [5] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-8970 > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > wes.turner%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Thu Apr 5 14:48:57 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 5 Apr 2018 14:48:57 -0400 Subject: [Python-Dev] gdb support could use some love In-Reply-To: References: <08353CAE-4322-4B6E-9881-8D6AF96FFEEE@python.org> Message-ID: On 4/5/2018 5:47 AM, Skip Montanaro wrote: >> There are a bunch of open issues regarding gdb support including one with a PR in need of review for 3.6+. > > I rejected one (which assumed everyone now uses a python-aware gdb), > commented on another (ceval.c-related name changes in several > commands), and created a PR for third (documentation for the > user-defined commands). Unfortunately, it's been so long since I > contributed, I don't quite understand the ins and outs of the workflow > anymore. In particular, I could find no way to add the "skip news" > label. I'm afraid someone might have to intervene here: > > https://github.com/python/cpython/pull/6384 You created the PR from your local python repository master branch. I have done this, with negative consequences. I believe you will find life with git easier if you never edit your master branch, or at least, never make local commits to it, and only commit to and create PRs from special-purpose patch branches, as described in the devguide. -- Terry Jan Reedy From nad at python.org Thu Apr 5 15:13:03 2018 From: nad at python.org (Ned Deily) Date: Thu, 5 Apr 2018 15:13:03 -0400 Subject: [Python-Dev] gdb support could use some love In-Reply-To: References: <08353CAE-4322-4B6E-9881-8D6AF96FFEEE@python.org> Message-ID: <0061AC7C-A426-40B4-A511-A1F9145F37EE@python.org> On Apr 5, 2018, at 14:48, Terry Reedy wrote: > On 4/5/2018 5:47 AM, Skip Montanaro wrote: >>> There are a bunch of open issues regarding gdb support including one with a PR in need of review for 3.6+. >> I rejected one (which assumed everyone now uses a python-aware gdb), >> commented on another (ceval.c-related name changes in several >> commands), and created a PR for third (documentation for the >> user-defined commands). Unfortunately, it's been so long since I >> contributed, I don't quite understand the ins and outs of the workflow >> anymore. In particular, I could find no way to add the "skip news" >> label. I'm afraid someone might have to intervene here: >> https://github.com/python/cpython/pull/6384 > > You created the PR from your local python repository master branch. I have done this, with negative consequences. I believe you will find life with git easier if you never edit your master branch, or at least, never make local commits to it, and only commit to and create PRs from special-purpose patch branches, as described in the devguide. That's a good observation, Terry. The main problem is that, when a core developer merges the PR from your local repo to the main python/cpython repo, we do a squash merge which means that the change or changes committed in the python/cpython and included in the PR will end up with a single new hash id that differs from what you originally committed in your cpython fork. So the next time you pull to your cpython fork (conventionally "origin") from the main cpython repo ("upstream"), there will be a conflict on the "normal" branch, i.e. master, 3.7, etc, requiring a merge or rebase, but you don't want to do that since the change is already upstream. Once the PR is merged into the main cpython repo, you can fix the problem by force removing the original commit(s) from the "normal" branch in your forked repo. See for example: https://stackoverflow.com/questions/9646167/clean-up-a-fork-and-restart-it-from-the-upstream -- Ned Deily nad at python.org -- [] From skip.montanaro at gmail.com Thu Apr 5 16:34:00 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Thu, 5 Apr 2018 15:34:00 -0500 Subject: [Python-Dev] gdb support could use some love In-Reply-To: References: <08353CAE-4322-4B6E-9881-8D6AF96FFEEE@python.org> Message-ID: > You created the PR from your local python repository master branch. I have > done this, with negative consequences. I believe you will find life with > git easier if you never edit your master branch, or at least, never make > local commits to it, and only commit to and create PRs from special-purpose > patch branches, as described in the devguide. Thanks. I clearly need to brush up on workflow and etiquette. Skip From skip.montanaro at gmail.com Thu Apr 5 16:36:20 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Thu, 5 Apr 2018 15:36:20 -0500 Subject: [Python-Dev] gdb support could use some love In-Reply-To: <7669EEBA-F4FF-442D-99FD-425FA206D315@python.org> References: <08353CAE-4322-4B6E-9881-8D6AF96FFEEE@python.org> <7669EEBA-F4FF-442D-99FD-425FA206D315@python.org> Message-ID: > Modifying GitHub Labels is only available to people with commit privs and, IIRC, Skip asked to drop his commit privs a few years ago (although I'm sure we would all be happy to welcome him back!). Alas, then I would feel some obligation to be semi-responsive to buggy things in areas where I have some interest. I'd really rather be out on my bike. :-) S From ethan at stoneleaf.us Thu Apr 5 17:17:17 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 05 Apr 2018 14:17:17 -0700 Subject: [Python-Dev] GH-6392: Enum, __contains__, TypeError, and DeprecationWarnings Message-ID: <5AC6925D.40502@stoneleaf.us> Oh my! ;) So, after the previous discussion about when __contains__ should raise TypeError instead of returning False, it seems there are a couple cases where a TypeError should be raised now: 1) non-enum-instance in EnumClass 2) anything in enum_instance 3) non-flag-instance in flag instance (1) can be True/False if an Enum instance is checked for; (2) should (and does already) always raise TypeError; (3) can be True/False when a Flag instance is checked for in another flag instance. I suspect these are corner cases, and infrequently, if ever, being used incorrectly now, but I would rather have a deprecation period in 3.7 instead of just changing in 3.8 to TypeErrors. To that end I have a PR [1] readay, and would appreciate any review, thoughts, and/or comments. -- ~Ethan~ [1] https://github.com/python/cpython/pull/6392 From anthony.flury at btinternet.com Thu Apr 5 22:57:09 2018 From: anthony.flury at btinternet.com (Anthony Flury) Date: Fri, 6 Apr 2018 03:57:09 +0100 Subject: [Python-Dev] Timeline for Pull request reviews in 3.8 In-Reply-To: References: Message-ID: Can anyone enlighten me on what the expected time-line is for reviewing pull requests made on 3.8. I made a few simple fixes in Early March - and I understand everyone is busy. What is the time line and cut off dates for backports to 3.7 and 3.6. I also made a documentation change (based on a open bug report) into 2.7, and I am keen to understand the planned time-line for those too. -- Anthony Flury email : *Anthony.flury at btinternet.com* Twitter : *@TonyFlury * From guido at python.org Fri Apr 6 00:45:38 2018 From: guido at python.org (Guido van Rossum) Date: Thu, 5 Apr 2018 21:45:38 -0700 Subject: [Python-Dev] Are undocumented functions part of the stable ABI? In-Reply-To: <5AC5B57F.1040402@UGent.be> References: <5AC493BC.8050104@UGent.be> <43a821a6be244016816c02de52fe0126@xmail102.UGent.be> <5AC5B57F.1040402@UGent.be> Message-ID: As you may guess from the silence, it may be hard to get a definitive answer to this question -- PEP 384's author has stopped actively participating in the Python community and I'm not sure if any core developers currently consider themselves to be the "guardians of the ABI". That said, from a quick skim of PEP 384 it seems to be quite strict in its position that anything not explicitly included by the PEP should be considered not part of the ABI, which makes me think that only a few PyCFunction related items are considered part of the ABI (searching for PyCFunction only finds three hits). OTOH it states that if Py_LIMITED_API is defined, all definitions that are not part of the ABI will be hidden. So from that (plus the source code) you should be able to tell which PyCFunction_* functions are part of the ABI -- again it does not appear the documentation status of a function matters for this rule. On the third hand, PEP 384 references a file "python3.def" ( https://www.python.org/dev/peps/pep-0384/#id5) and that file contains several PyCFunction_* names. Maybe this is the hard rule you're looking for? Again, being documented is not a requirement. Another observation would be that (AFAICT) PEP 384 strictly forbids signature changes, but is mostly silent on semantics. On Wed, Apr 4, 2018 at 10:34 PM, Jeroen Demeyer wrote: > On 2018-04-04 17:56, Guido van Rossum wrote: > >> It would be helpful if you explained the context of your request. >> > > The context is PEP 575. I guess my question is mostly about > PyCFunction_Check(). I will not be able to keep it 100% backwards > compatible simply because the goal of that PEP is precisely changing the > classes of some objects. > > Now the question is: am I allowed to change the implementation of > PyCFunction_Check()? If it's considered part of the stable ABI, then the > answer is immediately "no". > > By the way, does anybody happen to know why the PyCFunction_* functions > are undocumented? Is it just an oversight in the docs or is it intentional? > > But regardless of the context, I think that the question "Are undocumented > functions part of the stable ABI?" should be answered in PEP 384. > > > > Jeroen. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido% > 40python.org > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From turnbull.stephen.fw at u.tsukuba.ac.jp Fri Apr 6 00:39:13 2018 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Fri, 6 Apr 2018 13:39:13 +0900 Subject: [Python-Dev] Python version numbers In-Reply-To: References: <54D86C28-26EA-4503-BB1E-CDD926BE5B1A@langa.pl> <20180402231329.48e78d77@fsol> Message-ID: <23238.63985.739539.388145@turnbull.sk.tsukuba.ac.jp> Barry Warsaw writes: > Python 21.12 anyone? :) Well, for one thing we know that version 42 will be perfect! With current versioning policy, it will take a loooooong time to get there.... Steve -- Associate Professor Division of Policy and Planning Science http://turnbull/sk.tsukuba.ac.jp/ Faculty of Systems and Information Email: turnbull at sk.tsukuba.ac.jp University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN From eric at trueblade.com Fri Apr 6 08:37:51 2018 From: eric at trueblade.com (Eric V. Smith) Date: Fri, 6 Apr 2018 08:37:51 -0400 Subject: [Python-Dev] Timeline for Pull request reviews in 3.8 In-Reply-To: References: Message-ID: <1ced810a-d96c-decb-7f8b-a91889516d24@trueblade.com> Hi, Anthony. On 4/5/2018 10:57 PM, Anthony Flury via Python-Dev wrote: > Can anyone enlighten me on what the expected time-line is for reviewing > pull requests made on 3.8. I can give you the timeline for releases. Unfortunately, volunteer time is short for reviews. > I made a few simple fixes in Early March - and I understand everyone is > busy. If you mention the issues, someone might take a look at them. > What is the time line and cut off dates for backports to 3.7 and 3.6. Both are closed to new features and open for bug fixes. PEP 494 has the 3.6 schedule and PEP 537 has the 3.7 schedule. For 3.6 the last binary release is currently scheduled for 2018-12-03. > I also made a documentation change (based on a open bug report) into > 2.7, and I am keen to understand the planned time-line for those too. PEP 373 has the 2.7 release schedule. Eric. From skip.montanaro at gmail.com Fri Apr 6 12:06:11 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Fri, 6 Apr 2018 11:06:11 -0500 Subject: [Python-Dev] Can't open standard streams with "a" Message-ID: In Python 2 you can open the standard output and error streams in append mode, despite the fact that they aren't technically seekable. This changed somewhere along the way in (I think) io.open. There's an open bug report about this: https://bugs.python.org/issue27805 I just stumbled on it porting an application from Py2 to Py3 where my default logfile is /dev/stderr. I don't ever want to truncate a user's logfile, so "a" is a nice default, but in case the user doesn't name a logfile, I have to test, just in case /dev/stderr is the candidate. This seems error-prone, and must be done in all applications. The Py2 behavior seems less error-prone. I know explicit is better than implicit, but practicality also beats purity. Also, os.open is fine with O_APPEND on these streams. Any chance of this getting into 3.7 or will a fix have to wait for 3.8 at this point? (I'm guessing "no" as I don't see a patch.) Skip From status at bugs.python.org Fri Apr 6 12:09:57 2018 From: status at bugs.python.org (Python tracker) Date: Fri, 6 Apr 2018 18:09:57 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20180406160957.A8EE05672B@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2018-03-30 - 2018-04-06) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 6568 (+25) closed 38420 (+27) total 44988 (+52) Open issues with patches: 2558 Issues opened (40) ================== #23403: Use pickle protocol 4 by default? https://bugs.python.org/issue23403 reopened by lukasz.langa #31455: ElementTree.XMLParser() mishandles exceptions https://bugs.python.org/issue31455 reopened by zach.ware #33187: Document ElementInclude (XInclude) support in ElementTree https://bugs.python.org/issue33187 opened by scoder #33188: dataclass MRO entry resolution for type variable metaclasses: https://bugs.python.org/issue33188 opened by Ricyteach #33189: pygettext doesn't work with f-strings https://bugs.python.org/issue33189 opened by Riccardo Polignieri #33190: problem with ABCMeta.__prepare__ when called after types.new_c https://bugs.python.org/issue33190 opened by Ricyteach #33191: Refleak in posix_spawn https://bugs.python.org/issue33191 opened by zach.ware #33192: asyncio should use signal.set_wakeup_fd on Windows https://bugs.python.org/issue33192 opened by njs #33193: Cannot create a venv on Windows when directory path contains d https://bugs.python.org/issue33193 opened by Stuart Cuthbertson #33194: Path-file objects does not have method to delete itself if its https://bugs.python.org/issue33194 opened by rougeth #33196: SEGV in mp.synchronize.Lock.__repr__ in spawn'ed proc if ctx m https://bugs.python.org/issue33196 opened by arcivanov #33197: Confusing error message when constructing invalid inspect.Para https://bugs.python.org/issue33197 opened by Antony.Lee #33198: Build on Linux with --enable-optimizations fails https://bugs.python.org/issue33198 opened by fschulze #33200: Optimize the empty set "literal" https://bugs.python.org/issue33200 opened by serhiy.storchaka #33201: Modernize "Extension types" documentation https://bugs.python.org/issue33201 opened by pitrou #33204: IDLE: remove \b from colorizer string prefix https://bugs.python.org/issue33204 opened by terry.reedy #33205: GROWTH_RATE prevents dict shrinking https://bugs.python.org/issue33205 opened by inada.naoki #33210: pkgutil.walk_packages "prefix" option docs are misleading https://bugs.python.org/issue33210 opened by cykerway #33211: lineno and col_offset are wrong on function definitions with d https://bugs.python.org/issue33211 opened by gforcada #33212: add several options to msgfmt.py https://bugs.python.org/issue33212 opened by umedoblock #33213: crypt function not hashing properly on Mac (uses a specific sa https://bugs.python.org/issue33213 opened by Ron Reiter #33214: join method for list and tuple https://bugs.python.org/issue33214 opened by Javier Dehesa #33216: Wrong order of stack for CALL_FUNCTION_VAR and CALL_FUNCTION_V https://bugs.python.org/issue33216 opened by mvaled #33217: x in enum.Flag member is True when x is not a Flag https://bugs.python.org/issue33217 opened by Dutcho #33219: x in IntFlag should test raise TypeError if x is not an IntFla https://bugs.python.org/issue33219 opened by Dutcho #33220: Antivirus hits on python-2.7.14.amd64.msi file https://bugs.python.org/issue33220 opened by brett.rasmussen at inl.gov #33221: Add stats for asyncio task usage. https://bugs.python.org/issue33221 opened by asvetlov #33222: Various test failures if PYTHONUSERBASE is not canonicalized https://bugs.python.org/issue33222 opened by jdemeyer #33223: test_posix fails ERRNO 0 https://bugs.python.org/issue33223 opened by wizofe #33225: imaplib module IMAP4.append() unexpected response BAD Command https://bugs.python.org/issue33225 opened by yuy #33227: Cmd do_something only accepts one argument https://bugs.python.org/issue33227 opened by Oz.Tiram #33228: Use Random.choices in tempfile https://bugs.python.org/issue33228 opened by wolma #33229: Documentation - io ??? Core tools for working with streams - https://bugs.python.org/issue33229 opened by Mikhail Zakharov #33230: _decimal build failure (unsupported platform for that module) https://bugs.python.org/issue33230 opened by Hubert Holin #33232: Segmentation fault in operator.attrgetter https://bugs.python.org/issue33232 opened by asturm #33233: Suggest third-party cmd2 module as alternative to cmd https://bugs.python.org/issue33233 opened by ned.deily #33234: Improve list() pre-sizing for inputs with known lengths https://bugs.python.org/issue33234 opened by rhettinger #33235: Better help text for dict.setdefault https://bugs.python.org/issue33235 opened by Paddy McCarthy #33236: MagicMock().__iter__.return_value is different from MagicMock( https://bugs.python.org/issue33236 opened by mrh1997 #33237: Improve AttributeError message for partially initialized modul https://bugs.python.org/issue33237 opened by serhiy.storchaka Most recent 15 issues with no replies (15) ========================================== #33236: MagicMock().__iter__.return_value is different from MagicMock( https://bugs.python.org/issue33236 #33235: Better help text for dict.setdefault https://bugs.python.org/issue33235 #33234: Improve list() pre-sizing for inputs with known lengths https://bugs.python.org/issue33234 #33233: Suggest third-party cmd2 module as alternative to cmd https://bugs.python.org/issue33233 #33225: imaplib module IMAP4.append() unexpected response BAD Command https://bugs.python.org/issue33225 #33222: Various test failures if PYTHONUSERBASE is not canonicalized https://bugs.python.org/issue33222 #33220: Antivirus hits on python-2.7.14.amd64.msi file https://bugs.python.org/issue33220 #33201: Modernize "Extension types" documentation https://bugs.python.org/issue33201 #33198: Build on Linux with --enable-optimizations fails https://bugs.python.org/issue33198 #33192: asyncio should use signal.set_wakeup_fd on Windows https://bugs.python.org/issue33192 #33187: Document ElementInclude (XInclude) support in ElementTree https://bugs.python.org/issue33187 #33174: error building the _sha3 module with Intel 2018 compilers https://bugs.python.org/issue33174 #33173: GzipFile's .seekable() returns True even if underlying buffer https://bugs.python.org/issue33173 #33171: multiprocessing won't utilize all of platform resources https://bugs.python.org/issue33171 #33168: distutils build/build_ext and --debug https://bugs.python.org/issue33168 Most recent 15 issues waiting for review (15) ============================================= #33237: Improve AttributeError message for partially initialized modul https://bugs.python.org/issue33237 #33228: Use Random.choices in tempfile https://bugs.python.org/issue33228 #33227: Cmd do_something only accepts one argument https://bugs.python.org/issue33227 #33217: x in enum.Flag member is True when x is not a Flag https://bugs.python.org/issue33217 #33216: Wrong order of stack for CALL_FUNCTION_VAR and CALL_FUNCTION_V https://bugs.python.org/issue33216 #33212: add several options to msgfmt.py https://bugs.python.org/issue33212 #33205: GROWTH_RATE prevents dict shrinking https://bugs.python.org/issue33205 #33204: IDLE: remove \b from colorizer string prefix https://bugs.python.org/issue33204 #33201: Modernize "Extension types" documentation https://bugs.python.org/issue33201 #33200: Optimize the empty set "literal" https://bugs.python.org/issue33200 #33191: Refleak in posix_spawn https://bugs.python.org/issue33191 #33190: problem with ABCMeta.__prepare__ when called after types.new_c https://bugs.python.org/issue33190 #33189: pygettext doesn't work with f-strings https://bugs.python.org/issue33189 #33188: dataclass MRO entry resolution for type variable metaclasses: https://bugs.python.org/issue33188 #33176: Allow memoryview.cast(readonly=...) https://bugs.python.org/issue33176 Top 10 most discussed issues (10) ================================= #33204: IDLE: remove \b from colorizer string prefix https://bugs.python.org/issue33204 11 msgs #20104: expose posix_spawn(p) https://bugs.python.org/issue20104 10 msgs #33188: dataclass MRO entry resolution for type variable metaclasses: https://bugs.python.org/issue33188 10 msgs #33213: crypt function not hashing properly on Mac (uses a specific sa https://bugs.python.org/issue33213 10 msgs #33181: SimpleHTTPRequestHandler shouldn't redirect to directories wit https://bugs.python.org/issue33181 7 msgs #33228: Use Random.choices in tempfile https://bugs.python.org/issue33228 6 msgs #33185: Python 3.7.0b3 fails in pydoc where b2 did not. https://bugs.python.org/issue33185 5 msgs #33217: x in enum.Flag member is True when x is not a Flag https://bugs.python.org/issue33217 5 msgs #33219: x in IntFlag should test raise TypeError if x is not an IntFla https://bugs.python.org/issue33219 5 msgs #33221: Add stats for asyncio task usage. https://bugs.python.org/issue33221 5 msgs Issues closed (28) ================== #14119: Ability to adjust queue size in Executors https://bugs.python.org/issue14119 closed by pitrou #15257: Misc/.gdbinit:pystack is too brittle https://bugs.python.org/issue15257 closed by skip.montanaro #15817: Misc/gdbinit: Expose command documentation to gdb help https://bugs.python.org/issue15817 closed by ned.deily #27212: Doc for itertools, 'islice()' implementation have unwanted beh https://bugs.python.org/issue27212 closed by rhettinger #30071: Duck-typing inspect.isfunction() https://bugs.python.org/issue30071 closed by jdemeyer #30760: configparse module in python3 can not write '%' to config file https://bugs.python.org/issue30760 closed by lukasz.langa #31814: subprocess_fork_exec more stable with vfork https://bugs.python.org/issue31814 closed by gregory.p.smith #32337: Dict order is now guaranteed, so add tests and doc for it https://bugs.python.org/issue32337 closed by ned.deily #32360: Save OrderedDict imports in various stdlibs. https://bugs.python.org/issue32360 closed by inada.naoki #32713: tarfile.itn breaks if n is a negative float https://bugs.python.org/issue32713 closed by serhiy.storchaka #33096: ttk.Treeview.insert() does not allow to insert item with "Fals https://bugs.python.org/issue33096 closed by serhiy.storchaka #33132: Possible refcount issues in the compiler https://bugs.python.org/issue33132 closed by serhiy.storchaka #33152: Use list comprehension in timeit module instead of loop with a https://bugs.python.org/issue33152 closed by eric.araujo #33162: TimedRotatingFileHandler in logging module https://bugs.python.org/issue33162 closed by vinay.sajip #33186: Memory corruption with urllib.parse https://bugs.python.org/issue33186 closed by steven.daprano #33195: PyArg_Parse* should deprecate 'u' and 'z' family. https://bugs.python.org/issue33195 closed by inada.naoki #33199: PyDict_Copy() can leave 'ma_version_tag' uninitialized https://bugs.python.org/issue33199 closed by inada.naoki #33202: os.walk mentions os.listdir instead of os.scandir https://bugs.python.org/issue33202 closed by benjamin.peterson #33203: random.choice: raise IndexError on empty sequence even when no https://bugs.python.org/issue33203 closed by rhettinger #33206: Windows inet_pton(socket.AF_INET6, 'localhost') raises ValueEr https://bugs.python.org/issue33206 closed by zach.ware #33207: typing.Generic does not correctly call super().__init_subclass https://bugs.python.org/issue33207 closed by levkivskyi #33208: Lib2to3 grammar.txt error https://bugs.python.org/issue33208 closed by benjamin.peterson #33209: Repeated Pickler.dump() doesn't reset the state in the C imple https://bugs.python.org/issue33209 closed by serhiy.storchaka #33215: PyPI API wiki pages should be mutable https://bugs.python.org/issue33215 closed by ned.deily #33218: Fix instances in documentation where dictionaries are referenc https://bugs.python.org/issue33218 closed by inada.naoki #33224: "RuntimeError: generator raised StopIteration" in difflib.mdif https://bugs.python.org/issue33224 closed by rhettinger #33226: In some envrionment using unicode, formatwarning shows ascii e https://bugs.python.org/issue33226 closed by xtrusia #33231: Potential memory leak in normalizestring() https://bugs.python.org/issue33231 closed by inada.naoki From anthony.flury at btinternet.com Fri Apr 6 12:32:53 2018 From: anthony.flury at btinternet.com (Anthony Flury) Date: Fri, 6 Apr 2018 17:32:53 +0100 Subject: [Python-Dev] Timeline for Pull request reviews in 3.8 In-Reply-To: References: Message-ID: <01949ebd-4bbb-3540-06ee-399d4b9b1542@btinternet.com> All, The three pull requests are : Python 2.7 - doc string fix : https://github.com/python/cpython/pull/6015 Python 3.8 - documentation fix : https://github.com/python/cpython/pull/5982 Python 3.8 - Small bug fix on unittest.mock.mock_open : https://github.com/python/cpython/pull/5974 The Py2.7 change does not need to be rolled forward to Python3 documentation The two Py3.8 fixes could/should/can ? be backported to earlier versions These are all trivial with no conflicts with their target branch (or at least there wasn't when I made the requests). -- Anthony Flury email : *Anthony.flury at btinternet.com* Twitter : *@TonyFlury * From storchaka at gmail.com Fri Apr 6 12:56:46 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Fri, 6 Apr 2018 19:56:46 +0300 Subject: [Python-Dev] Timeline for Pull request reviews in 3.8 In-Reply-To: <01949ebd-4bbb-3540-06ee-399d4b9b1542@btinternet.com> References: <01949ebd-4bbb-3540-06ee-399d4b9b1542@btinternet.com> Message-ID: 06.04.18 19:32, Anthony Flury via Python-Dev ????: > The three pull requests are : > > Python 2.7 - doc string fix : https://github.com/python/cpython/pull/6015 There are several open issues for "sequence" vs "iterable" in docstrings and documentation for different functions. It would be worth to merge all these issues and and make a general decision. > Python 3.8 - documentation fix : > https://github.com/python/cpython/pull/5982 It needs a review from one of our floating point experts. > Python 3.8 - Small bug fix on unittest.mock.mock_open : > https://github.com/python/cpython/pull/5974 It needs a review from one of our unittest.mock experts. From berker.peksag at gmail.com Fri Apr 6 12:58:39 2018 From: berker.peksag at gmail.com (=?UTF-8?Q?Berker_Peksa=C4=9F?=) Date: Fri, 6 Apr 2018 19:58:39 +0300 Subject: [Python-Dev] Timeline for Pull request reviews in 3.8 In-Reply-To: <01949ebd-4bbb-3540-06ee-399d4b9b1542@btinternet.com> References: <01949ebd-4bbb-3540-06ee-399d4b9b1542@btinternet.com> Message-ID: On Fri, Apr 6, 2018 at 7:32 PM, Anthony Flury via Python-Dev wrote: > All, > > The three pull requests are : > > Python 2.7 - doc string fix : https://github.com/python/cpython/pull/6015 > > Python 3.8 - documentation fix : https://github.com/python/cpython/pull/5982 Hi Anthony, I've just reviewed this. > Python 3.8 - Small bug fix on unittest.mock.mock_open : > https://github.com/python/cpython/pull/5974 > > The Py2.7 change does not need to be rolled forward to Python3 documentation > > The two Py3.8 fixes could/should/can ? be backported to earlier versions Yes, PR 5982 will be backported to 3.6 and 3.7. I'm not sure about PR 5974 though (I may be wrong because I don't have the time to triage the issue at the moment) --Berker From cbryan at rapitasystems.com Fri Apr 6 07:56:49 2018 From: cbryan at rapitasystems.com (Chris Bryan) Date: Fri, 6 Apr 2018 12:56:49 +0100 Subject: [Python-Dev] Error embedding python Message-ID: <6f0e1c40-ef93-03e3-09a2-29f6ff5143a4@rapitasystems.com> Hello list, I am embedding a python 3.6 environment in another executable, which is compiling and executing ok. However I get an error on Windows when I try to import any module except sys: Traceback (most recent call last): ? File "args", line 1, in ? File "", line 971, in _find_and_load ? File "", line 951, in _find_and_load_unlocked ? File "", line 894, in _find_spec ? File "", line 1157, in find_spec ? File "", line 1129, in _get_spec ? File "", line 1245, in find_spec ? File "", line 1302, in _fill_cache TypeError: a bytes-like object is required, not 'str' Looking on line 1302, it would appear that the call to _os.listdir() (line 1285) is returning a list of byte objects. I can confirm that I get the same error doing the following from a normal python interactive session: >>> x = b'a.b.c' >>> x.partition('.') Traceback (most recent call last): ? File "", line 1, in TypeError: a bytes-like object is required, not 'str' Is this a bug in the bootstrap module, or am I doing something wrong which is causing listdir() to return bytes? Chris From nad at python.org Fri Apr 6 17:17:58 2018 From: nad at python.org (Ned Deily) Date: Fri, 6 Apr 2018 17:17:58 -0400 Subject: [Python-Dev] Can't open standard streams with "a" In-Reply-To: References: Message-ID: <900869CA-0CC6-46CB-8D16-A7C69237852D@python.org> On Apr 6, 2018, at 12:06, Skip Montanaro wrote: > [...] > https://bugs.python.org/issue27805 > [...] > Any chance of this getting into 3.7 or will a fix have to wait for 3.8 > at this point? (I'm guessing "no" as I don't see a patch.) It seems like most commenters on the issue consider the behavior to be a bug so a case could be made for a fix to go into a maintenance release, not just a feature release. But, obviously, someone needs to produce a PR for review first. -- Ned Deily nad at python.org -- [] From solipsis at pitrou.net Fri Apr 6 17:31:15 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 6 Apr 2018 23:31:15 +0200 Subject: [Python-Dev] Can't open standard streams with "a" References: <900869CA-0CC6-46CB-8D16-A7C69237852D@python.org> Message-ID: <20180406233115.0a4c63f7@fsol> On Fri, 6 Apr 2018 17:17:58 -0400 Ned Deily wrote: > On Apr 6, 2018, at 12:06, Skip Montanaro wrote: > > [...] > > https://bugs.python.org/issue27805 > > [...] > > Any chance of this getting into 3.7 or will a fix have to wait for 3.8 > > at this point? (I'm guessing "no" as I don't see a patch.) > > It seems like most commenters on the issue consider the behavior to be a bug so a case could be made for a fix to go into a maintenance release, not just a feature release. But, obviously, someone needs to produce a PR for review first. OTOH this has never worked on Python 3, so it's hardly an important fix. If it's delicate enough, we may prefer to only ship it in 3.8. PS : it's the first time I encounter "/dev/stderr". Apparently it's a Linux-only symlink to "/proc/self/fd/2", i.e. the current process' file descriptor number 2 (which is conventionally the current process' stderr stream)! Regards Antoine. From nad at python.org Fri Apr 6 18:02:18 2018 From: nad at python.org (Ned Deily) Date: Fri, 6 Apr 2018 18:02:18 -0400 Subject: [Python-Dev] Soliciting comments on the future of the cmd module (bpo-33233) Message-ID: <7CAA437E-0734-41F4-83AB-0FAD07262DB1@python.org> In https://bugs.python.org/issue33233, I have proposed considering deprecation for the cmd module: "The cmd module in the standard library has languished for many years. In the mean time, third-party replacements for it have arisen. Perhaps the most popular is cmd2 which seems to be actively maintained, provides upward compatibility from cmd along with many desirable new features, and is permissively licensed. I suggest we consider at a minimum adding a "See also:" note referencing cmd2 to the cmd documentation in the Standard Library document, similar to what we do for the third-party "requests" module in the "urllib.request" documentation. We could be even bolder and officially deprecate "cmd" and consider closing open enhancement issues for it on b.p.o." What triggered this suggestion is that Oz Tiram opened an enhancement request for cmd with suggested code (https://bugs.python.org/issue33227) that implements a feature already in cmd2. As I replied there, it seems to me to be almost a disservice to our users to add piecemeal enhancements to cmd at this point when it is essentially unmaintained and a superior alternative exists - that's assuming a core developer stepped up to shepherd the proposed change. I don't have a strong feeling one way or another about the change but I think this might be one instance where we can and should make a decision to prioritize our limited volunteer resources and acknowledge the current state of affairs re cmd. We've done similar things with other standard library modules in the past. If you have an opinion about either recommending cmd2 in the cmd docs and/or deprecating cmd in 3.8, please comment on https://bugs.python.org/issue33233. Thanks! --Ned https://docs.python.org/3.8/library/cmd.html https://pypi.org/project/cmd2/ https://cmd2.readthedocs.io/ =- Ned Deily nad at python.org -- [] From steve at pearwood.info Fri Apr 6 20:13:32 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 7 Apr 2018 10:13:32 +1000 Subject: [Python-Dev] Soliciting comments on the future of the cmd module (bpo-33233) In-Reply-To: <7CAA437E-0734-41F4-83AB-0FAD07262DB1@python.org> References: <7CAA437E-0734-41F4-83AB-0FAD07262DB1@python.org> Message-ID: <20180407001331.GZ16661@ando.pearwood.info> On Fri, Apr 06, 2018 at 06:02:18PM -0400, Ned Deily wrote: > I suggest we consider at a minimum adding a "See also:" note > referencing cmd2 to the cmd documentation in the Standard Library > document, similar to what we do for the third-party "requests" module > in the "urllib.request" documentation. I think the documentation note is a good idea. But I disagree with deprecating "cmd" unless it is actively falling apart and no longer working, not just languishing with no feature improvements. Just in the last week, I've been reminded twice that many people using Python do so where they cannot just arbitarily pip install , and if a library isn't in the std lib, they can't use it without a lot of pain: https://mail.python.org/pipermail/tutor/2018-April/112817.html https://mail.python.org/pipermail/tutor/2018-April/112818.html [...] > What triggered this suggestion is that Oz Tiram opened an enhancement > request for cmd with suggested code > (https://bugs.python.org/issue33227) that implements a feature already > in cmd2. As I replied there, it seems to me to be almost a disservice > to our users to add piecemeal enhancements to cmd at this point when > it is essentially unmaintained and a superior alternative exists - > that's assuming a core developer stepped up to shepherd the proposed > change. It seems especially perverse to complain that the module is not being maintained, and then when the first person steps up with a feature request that might encourage giving the module some attention, to deprecate it instead! *wink* > If you have an opinion about either recommending cmd2 in the cmd docs > and/or deprecating cmd in 3.8, please comment on > https://bugs.python.org/issue33233. I will follow up there as well. -- Steve From raymond.hettinger at gmail.com Fri Apr 6 20:57:04 2018 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Fri, 6 Apr 2018 17:57:04 -0700 Subject: [Python-Dev] Soliciting comments on the future of the cmd module (bpo-33233) In-Reply-To: <7CAA437E-0734-41F4-83AB-0FAD07262DB1@python.org> References: <7CAA437E-0734-41F4-83AB-0FAD07262DB1@python.org> Message-ID: > On Apr 6, 2018, at 3:02 PM, Ned Deily wrote: > > We could be even bolder and officially deprecate "cmd" and consider closing open enhancement issues for it on b.p.o." FWIW, the pdb module depends on the cmd module. Also, I still teach people how to use cmd and I think it still serves a useful purpose. So, unless it is considered broken, I don't think it should be deprecated. Raymond From steve.dower at python.org Fri Apr 6 23:13:06 2018 From: steve.dower at python.org (Steve Dower) Date: Fri, 6 Apr 2018 20:13:06 -0700 Subject: [Python-Dev] Soliciting comments on the future of the cmdmodule (bpo-33233) In-Reply-To: References: <7CAA437E-0734-41F4-83AB-0FAD07262DB1@python.org> Message-ID: Better to deprecate it before it becomes broken, in my opinion. Having someone willing and able to review and merge changes is the best criteria for whether a module is still supported or not. Top-posted from my Windows phone From: Raymond Hettinger Sent: Friday, April 6, 2018 18:55 To: Ned Deily Cc: Python-Dev at Python. Org Subject: Re: [Python-Dev] Soliciting comments on the future of the cmdmodule (bpo-33233) > On Apr 6, 2018, at 3:02 PM, Ned Deily wrote: > > We could be even bolder and officially deprecate "cmd" and consider closing open enhancement issues for it on b.p.o." FWIW, the pdb module depends on the cmd module. Also, I still teach people how to use cmd and I think it still serves a useful purpose. So, unless it is considered broken, I don't think it should be deprecated. Raymond _______________________________________________ Python-Dev mailing list Python-Dev at python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Sat Apr 7 04:30:05 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 7 Apr 2018 09:30:05 +0100 Subject: [Python-Dev] Soliciting comments on the future of the cmdmodule (bpo-33233) In-Reply-To: <40J1sK5hgRzFr6k@mail.python.org> References: <7CAA437E-0734-41F4-83AB-0FAD07262DB1@python.org> <40J1sK5hgRzFr6k@mail.python.org> Message-ID: On 7 April 2018 at 04:13, Steve Dower wrote: > Better to deprecate it before it becomes broken, in my opinion. > > Having someone willing and able to review and merge changes is the best > criteria for whether a module is still supported or not. I think there's a difference between not being willing to add enhancements, and not fixing bugs. The issue that originally triggered this discussion was an enhancement request, and I don't think it's unreasonable to declare cmd as "stable - no further enhancements will be made or accepted" while still considering it as supported for bugfixes. If significant bugs in cmd are remaining unfixed, then that's a somewhat different matter. The fact that pdb uses it, and the advantage of having something in the stdlib for users without easy access to "pip install", *plus* the general principle of "if it isn't broken, don't fix it" make me feel that the best solution would be to document that extended replacements such as cmd2 exist in PyPI, but retain cmd as supported but not (in principle) accepting further enhancements (leaving the door open for interested core devs to merge enhancements on a case by case basis if they have a personal interest in doing so). Paul From steve at pearwood.info Sat Apr 7 05:08:44 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 7 Apr 2018 19:08:44 +1000 Subject: [Python-Dev] Soliciting comments on the future of the cmdmodule (bpo-33233) In-Reply-To: References: <7CAA437E-0734-41F4-83AB-0FAD07262DB1@python.org> <40J1sK5hgRzFr6k@mail.python.org> Message-ID: <20180407090841.GD16661@ando.pearwood.info> On Sat, Apr 07, 2018 at 09:30:05AM +0100, Paul Moore wrote: > On 7 April 2018 at 04:13, Steve Dower wrote: > > Better to deprecate it before it becomes broken, in my opinion. That argument could be applied to everything in the std lib. > > Having someone willing and able to review and merge changes is the best > > criteria for whether a module is still supported or not. I don't think "best" is justified -- and certainly it is not the *only* criteria. Modules can become stable simply because they have no known bugs and no new feature requests. Stable doesn't mean useless, and the urge to consider anything that isn't being regularly fiddled with as "obsolete" is a tendency to be resisted. If the module isn't broken, there's no need to fix it. That's a GOOD thing, not a reason to dump a perfectly good, useful, working module. Raymond has stated that he is happy to work on it if there are any bugs reported on it, and if he's not available, I'm sure somebody will. And if not, well, we don't have any sort of performance guarantees on fixes: sooner or later, *somebody* will provide a patch. That's the beauty of the Open Source model. There are plenty of potential upstream contributors who could contribute a patch. > I think there's a difference between not being willing to add > enhancements, and not fixing bugs. The issue that originally triggered > this discussion was an enhancement request, and I don't think it's > unreasonable to declare cmd as "stable - no further enhancements will > be made or accepted" while still considering it as supported for > bugfixes. Its not an unreasonable position to take, but I don't think it is justified in this case. The cmd module is not something so arcane or complicated that it requires a specialist to maintain it. Its about 400 lines, including blank lines and doc strings, with a single class and around twenty methods. Wasn't one of the major reasons for moving to git and Github to make it easier for non-core devs to contribute? A module as stable and simple as cmd seems to me to be the ideal place for people to begin contributing, whether it is fixing bugs or contributing any (hypothetical) feature enhancements. I don't think we need do anything here: so long as there is a core developer willing to review any PRs, and so long as new enhancements go through the same approval process on the bug tracker and/or Python- Ideas, I don't think we need to single cmd out as deprecated or "no new features". This isn't gopher, or something with serious unfixable security vulnerabilities. It works. What more needs to be said? -- Steve From mcepl at cepl.eu Sat Apr 7 09:54:01 2018 From: mcepl at cepl.eu (=?UTF-8?Q?Mat=C4=9Bj?= Cepl) Date: Sat, 07 Apr 2018 15:54:01 +0200 Subject: [Python-Dev] Soliciting comments on the future of the cmd module (bpo-33233) References: <7CAA437E-0734-41F4-83AB-0FAD07262DB1@python.org> <20180407001331.GZ16661@ando.pearwood.info> Message-ID: On 2018-04-07, 00:13 GMT, Steven D'Aprano wrote: > Just in the last week, I've been reminded twice that many > people using Python do so where they cannot just arbitarily > pip install , and if a library isn't in the std lib, > they can't use it without a lot of pain: 100% agree + one of the great advantages of Python is that the batteries actually are included and we are not forcing users to download one of twenty competing unstable versions somewhere on GitHub (I won't name any competing languages here). Also, I don't see a problem with having one more mature, slower version of library in the standard library, while the more alive version is still being developed from time to time rebasing the stdlib version (I thought, unittest was one such example.) Also, considering requests, I am still dreaming about somebody writing some requests-like API over the standard library. Best, Mat?j -- https://matej.ceplovi.cz/blog/, Jabber: mcepl at ceplovi.cz GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 My opinions may have changed, but not the fact that I am right. --Ashleigh Brilliant From brett at python.org Sat Apr 7 14:02:20 2018 From: brett at python.org (Brett Cannon) Date: Sat, 07 Apr 2018 18:02:20 +0000 Subject: [Python-Dev] Error embedding python In-Reply-To: <6f0e1c40-ef93-03e3-09a2-29f6ff5143a4@rapitasystems.com> References: <6f0e1c40-ef93-03e3-09a2-29f6ff5143a4@rapitasystems.com> Message-ID: Most likely your 'args' module is calling import with a bytes object and not a string and that it's getting that far into the process before you hit code that only works with strings (_os.listdir() returns bytes if you pass a bytes argument to it). At this point I would take the question to python-list or python-tutor to get more help with embedding. On Fri, 6 Apr 2018 at 10:29 Chris Bryan wrote: > Hello list, > > I am embedding a python 3.6 environment in another executable, which is > compiling and executing ok. However I get an error on Windows when I try > to import any module except sys: > > Traceback (most recent call last): > File "args", line 1, in > File "", line 971, in _find_and_load > File "", line 951, in > _find_and_load_unlocked > File "", line 894, in _find_spec > File "", line 1157, in find_spec > File "", line 1129, in _get_spec > File "", line 1245, in find_spec > File "", line 1302, in _fill_cache > TypeError: a bytes-like object is required, not 'str' > > Looking on line 1302, it would appear that the call to _os.listdir() > (line 1285) is returning a list of byte objects. I can confirm that I > get the same error doing the following from a normal python interactive > session: > > >>> x = b'a.b.c' > >>> x.partition('.') > Traceback (most recent call last): > File "", line 1, in > TypeError: a bytes-like object is required, not 'str' > > Is this a bug in the bootstrap module, or am I doing something wrong > which is causing listdir() to return bytes? > > Chris > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Sat Apr 7 14:50:00 2018 From: chris.barker at noaa.gov (Chris Barker - NOAA Federal) Date: Sat, 7 Apr 2018 14:50:00 -0400 Subject: [Python-Dev] Soliciting comments on the future of the cmd module (bpo-33233) In-Reply-To: References: <7CAA437E-0734-41F4-83AB-0FAD07262DB1@python.org> <20180407001331.GZ16661@ando.pearwood.info> Message-ID: Is bringing cmd2 into the standard library an option to be considered? That water get included batteries and a more featurefull and supported lib. It seems (on python-ideas) that people are often told, when they have a suggestion for the stdlib, that they put it on pypi and see if it gains usage, etc, and it it proves useful and popular, maybe then it could be brought in. That seems to be the case here. I understand that it only makes sense if the cmd2 developers want to support it and are willing to accept the responsibilities and restrictions that come with being in the core, but it?s worth asking. -CHB From brett at python.org Sat Apr 7 14:53:44 2018 From: brett at python.org (Brett Cannon) Date: Sat, 07 Apr 2018 18:53:44 +0000 Subject: [Python-Dev] Soliciting comments on the future of the cmdmodule (bpo-33233) In-Reply-To: <20180407090841.GD16661@ando.pearwood.info> References: <7CAA437E-0734-41F4-83AB-0FAD07262DB1@python.org> <40J1sK5hgRzFr6k@mail.python.org> <20180407090841.GD16661@ando.pearwood.info> Message-ID: On Sat, 7 Apr 2018 at 02:09 Steven D'Aprano wrote: > On Sat, Apr 07, 2018 at 09:30:05AM +0100, Paul Moore wrote: > > On 7 April 2018 at 04:13, Steve Dower wrote: > > > Better to deprecate it before it becomes broken, in my opinion. > > That argument could be applied to everything in the std lib. > Sure, but we all know none of this is as black-and-white as it's being portrayed either. > > > > > Having someone willing and able to review and merge changes is the best > > > criteria for whether a module is still supported or not. > > I don't think "best" is justified -- and certainly it is not the *only* > criteria. > But you have to admit it is an important one. > > Modules can become stable simply because they have no known bugs and no > new feature requests. Stable doesn't mean useless, and the urge to > consider anything that isn't being regularly fiddled with as "obsolete" > is a tendency to be resisted. > > If the module isn't broken, there's no need to fix it. That's a GOOD > thing, not a reason to dump a perfectly good, useful, working module. > I think part of the question is whether the module is also used enough to justify putting in scarce core dev time to maintain it. > > Raymond has stated that he is happy to work on it if there are any bugs > reported on it, and if he's not available, I'm sure somebody will. Actually Raymond said he *teaches* the module, not that he wanted to maintain it. And I definitely would not assume that someone will pick up to help maintain any module in the stdlib. > And > if not, well, we don't have any sort of performance guarantees on fixes: > sooner or later, *somebody* will provide a patch. Assuming someone does, do we really want to say, "eh, it's buggy but we can wait over 7 years for a fix" (the oldest open issue on b.p.o is from August 2009: https://bugs.python.org/issue6686). And even if they do, that doesn't mean someone will have the time or inclination to review it and eventually see it through to being merged. > That's the beauty of > the Open Source model. There are plenty of potential upstream > contributors who could contribute a patch. > But potential does not necessarily translate to action. > > > > I think there's a difference between not being willing to add > > enhancements, and not fixing bugs. The issue that originally triggered > > this discussion was an enhancement request, and I don't think it's > > unreasonable to declare cmd as "stable - no further enhancements will > > be made or accepted" while still considering it as supported for > > bugfixes. > > Its not an unreasonable position to take, but I don't think it is > justified in this case. The cmd module is not something so arcane or > complicated that it requires a specialist to maintain it. Its about 400 > lines, including blank lines and doc strings, with a single class and > around twenty methods. > > Wasn't one of the major reasons for moving to git and Github to > make it easier for non-core devs to contribute? Actually the major reason was to make it easier for core devs to review contributions. Easing the workflow for outside contributors was a side benefit. > A module as stable and > simple as cmd seems to me to be the ideal place for people to begin > contributing, whether it is fixing bugs or contributing any > (hypothetical) feature enhancements. > Perhaps, but that assumes someone wants that job. ;) > > I don't think we need do anything here: so long as there is a core > developer willing to review any PRs, and so long as new enhancements go > through the same approval process on the bug tracker and/or Python- > Ideas, I don't think we need to single cmd out as deprecated or "no new > features". > Those two "so long as" parts are I think the key reason Ned brought this up. > > This isn't gopher, or something with serious unfixable security > vulnerabilities. It works. What more needs to be said? > I think this all ties back to the usual discussion we have when it comes to the stdlib: what is the bar for what should be in there because nothing is maintenance-free? The cmd module itself has plenty of commits that were just standard code maintenance changes: https://github.com/python/cpython/commits/master/Lib/cmd.py . All of that eats up time over the 26 years of the module's existence. And feature enhancements are not free either even if you don't review the PR because even if you don't review them you still have to peruse the PR title to make the decision not to review it, which once again is a small amount of time in isolation but adds up at a macro level. We all have a limited amount of time to contribute, especially when we are almost all spending personal time on this project. And no matter how trivial a module is to keep around, it is not a non-zero amount of time for the group all-up, even if we all collectively choose to ignore it in hopes someone comes forward to help out. And I would also say that ignoring a module until someone comes forward to maintain it runs counter-productive to our overall goal to produce quality software as we are effectively saying we are okay with lower quality if it simply puts us out too much to maintain it. I personally would argue that the language itself is an amazingly solid piece of software but that the stdlib's quality varies. For me, that's where my personal desire to trim the stdlib a bit so we can have "more wood behind fewer arrows" and bring the stdlib's quality up a bit. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nad at python.org Sat Apr 7 14:59:32 2018 From: nad at python.org (Ned Deily) Date: Sat, 7 Apr 2018 14:59:32 -0400 Subject: [Python-Dev] Soliciting comments on the future of the cmd module (bpo-33233) In-Reply-To: References: <7CAA437E-0734-41F4-83AB-0FAD07262DB1@python.org> <20180407001331.GZ16661@ando.pearwood.info> Message-ID: Thanks for everyone's interest but, please, let's keep the discussion in one place as originally requested: > If you have an opinion about either recommending cmd2 in the cmd docs and/or > deprecating cmd in 3.8, please comment on https://bugs.python.org/issue33233. You'll find some answers to some of these questions there already. -- Ned Deily nad at python.org -- [] From brett at python.org Sat Apr 7 15:00:16 2018 From: brett at python.org (Brett Cannon) Date: Sat, 07 Apr 2018 19:00:16 +0000 Subject: [Python-Dev] Soliciting comments on the future of the cmd module (bpo-33233) In-Reply-To: References: <7CAA437E-0734-41F4-83AB-0FAD07262DB1@python.org> <20180407001331.GZ16661@ando.pearwood.info> Message-ID: On Sat, 7 Apr 2018 at 11:50 Chris Barker - NOAA Federal < chris.barker at noaa.gov> wrote: > Is bringing cmd2 into the standard library an option to be considered? > Anything can be considered. ;) > > That water get included batteries and a more featurefull and supported lib. > > It seems (on python-ideas) that people are often told, when they have > a suggestion for the stdlib, that they put it on pypi and see if it > gains usage, etc, and it it proves useful and popular, maybe then it > could be brought in. That seems to be the case here. > > I understand that it only makes sense if the cmd2 developers want to > support it and are willing to accept the responsibilities and > restrictions that come with being in the core, but it?s worth asking. > First I think we need to ask ourselves whether we would even want a module like cmd in the stdlib today. It's 26 years old, so it was added back when Guido has admitted the bar for entry was much lower. If we wouldn't add the module today, then there's no need to bring in a replacement. Plus we have instances in the stdlib where we brought it in with the maintainer becoming a core dev, and then they step away and no one picks the module up going forward. So I would argue that even if we thought we would add the cmd module today that we shouldn't bring it in unless we have at least 2 people in total willing to list themselves in the experts index for that module (preferably 3 people so we're never down to just one person when someone takes a break). -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephane.blondon at gmail.com Sat Apr 7 16:21:06 2018 From: stephane.blondon at gmail.com (=?UTF-8?Q?St=c3=a9phane_Blondon?=) Date: Sat, 7 Apr 2018 22:21:06 +0200 Subject: [Python-Dev] Is it useful to update cgitb module? Message-ID: Hello, I wonder if it's useful to update the cgitb module, in particular the html output. I see some possible improvements: 1. In both text and html versions: When a module is called, there are no parameters (displayed as '()'). I think they are unnecessary. Perhaps the parentheses should be removed? Perhaps it's better to keep them for backward compatibility? ### example for the text version ### $ python3 demo.py [...] /home/stephane/src/cgitest/demo.py in () 7 def func1(a, b): [...] ### end of example ### 2. In html version only: a. If the executed code is in : in this case, it is not shown in the html version because the square brackets are interpreted as a html tag (see the picture in attachement). b. Update the style of the html or/and using html5. It would be prettier but it will be a big change for probably too few benefits. What do you think about them? I can report bugs and send pull-requests for them but I would prefer to get feedbacks before. Regards, St?phane -------------- next part -------------- A non-text attachment was scrubbed... Name: traceback_cgi.gif Type: image/gif Size: 10238 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From skip.montanaro at gmail.com Sat Apr 7 18:31:47 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sat, 7 Apr 2018 17:31:47 -0500 Subject: [Python-Dev] Trying to build from source, test-poplib fails Message-ID: It's been a long while since I rebuilt Python from the Git source. I tried for the first time the other day. Everything passed except test_poplib and test_asyncio. The former just runs and runs and runs. Here's the first traceback I encounter when executing ./python Lib/test/test_poplib.py: test_stls_context (__main__.TestPOP3Class) ... Exception in thread Thread-16: Traceback (most recent call last): File "/home/skip/src/python/cpython/Lib/threading.py", line 917, in _bootstrap_inner self.run() File "Lib/test/test_poplib.py", line 227, in run asyncore.loop(timeout=0.1, count=1) File "/home/skip/src/python/cpython/Lib/asyncore.py", line 207, in loop poll_fun(timeout, map) File "/home/skip/src/python/cpython/Lib/asyncore.py", line 150, in poll read(obj) File "/home/skip/src/python/cpython/Lib/asyncore.py", line 87, in read obj.handle_error() File "/home/skip/src/python/cpython/Lib/asyncore.py", line 83, in read obj.handle_read_event() File "/home/skip/src/python/cpython/Lib/asyncore.py", line 422, in handle_read_event self.handle_read() File "Lib/test/test_poplib.py", line 194, in handle_read self._do_tls_handshake() File "Lib/test/test_poplib.py", line 174, in _do_tls_handshake self.socket.do_handshake() File "/home/skip/src/python/cpython/Lib/ssl.py", line 1108, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert certificate unknown (_ssl.c:1049) I thought perhaps I was missing something, but _ssl built just fine. I believe I have a recent enough version of libssl (1.0.2g), and I'm on a pretty vanilla system (up-to-date Ubuntu 17.10). SSL-wise: % apt search ssl | grep installed | egrep '^lib' | egrep ssl libgnutls-openssl27/artful,now 3.5.8-6ubuntu3 amd64 [installed] libio-socket-ssl-perl/artful,artful,now 2.050-1 all [installed] libnet-smtp-ssl-perl/artful,artful,now 1.04-1 all [installed] libnet-ssleay-perl/artful,now 1.80-1build1 amd64 [installed] libssl-dev/artful-updates,artful-security,now 1.0.2g-1ubuntu13.4 amd64 [installed] libssl-doc/artful-updates,artful-updates,artful-security,artful-security,now 1.0.2g-1ubuntu13.4 all [installed,automatic] libssl1.0.0/artful-updates,artful-security,now 1.0.2g-1ubuntu13.4 amd64 [installed] Any clues about what I might be missing from my setup would be appreciated. Not sure what was wrong with test_asyncio. I ran it in isolation and it passed. Thx, Skip From rymg19 at gmail.com Sat Apr 7 19:28:56 2018 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Sat, 07 Apr 2018 18:28:56 -0500 Subject: [Python-Dev] Trying to build from source, test-poplib fails In-Reply-To: References: Message-ID: <162a270fac0.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> Do you have ca-certificates installed? On April 7, 2018 5:33:35 PM Skip Montanaro wrote: > It's been a long while since I rebuilt Python from the Git source. I > tried for the first time the other day. Everything passed except > test_poplib and test_asyncio. The former just runs and runs and runs. > Here's the first traceback I encounter when executing ./python > Lib/test/test_poplib.py: > > test_stls_context (__main__.TestPOP3Class) ... Exception in thread Thread-16: > Traceback (most recent call last): > File "/home/skip/src/python/cpython/Lib/threading.py", line 917, in > _bootstrap_inner > self.run() > File "Lib/test/test_poplib.py", line 227, in run > asyncore.loop(timeout=0.1, count=1) > File "/home/skip/src/python/cpython/Lib/asyncore.py", line 207, in loop > poll_fun(timeout, map) > File "/home/skip/src/python/cpython/Lib/asyncore.py", line 150, in poll > read(obj) > File "/home/skip/src/python/cpython/Lib/asyncore.py", line 87, in read > obj.handle_error() > File "/home/skip/src/python/cpython/Lib/asyncore.py", line 83, in read > obj.handle_read_event() > File "/home/skip/src/python/cpython/Lib/asyncore.py", line 422, in > handle_read_event > self.handle_read() > File "Lib/test/test_poplib.py", line 194, in handle_read > self._do_tls_handshake() > File "Lib/test/test_poplib.py", line 174, in _do_tls_handshake > self.socket.do_handshake() > File "/home/skip/src/python/cpython/Lib/ssl.py", line 1108, in do_handshake > self._sslobj.do_handshake() > ssl.SSLError: [SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert > certificate unknown (_ssl.c:1049) > > I thought perhaps I was missing something, but _ssl built just fine. I > believe I have a recent enough version of libssl (1.0.2g), and I'm on > a pretty vanilla system (up-to-date Ubuntu 17.10). SSL-wise: > > % apt search ssl | grep installed | egrep '^lib' | egrep ssl > > libgnutls-openssl27/artful,now 3.5.8-6ubuntu3 amd64 [installed] > libio-socket-ssl-perl/artful,artful,now 2.050-1 all [installed] > libnet-smtp-ssl-perl/artful,artful,now 1.04-1 all [installed] > libnet-ssleay-perl/artful,now 1.80-1build1 amd64 [installed] > libssl-dev/artful-updates,artful-security,now 1.0.2g-1ubuntu13.4 amd64 > [installed] > libssl-doc/artful-updates,artful-updates,artful-security,artful-security,now > 1.0.2g-1ubuntu13.4 all [installed,automatic] > libssl1.0.0/artful-updates,artful-security,now 1.0.2g-1ubuntu13.4 > amd64 [installed] > > Any clues about what I might be missing from my setup would be appreciated. > > Not sure what was wrong with test_asyncio. I ran it in isolation and it passed. > > Thx, > > Skip > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com -- Ryan (????) Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone else https://refi64.com/ From skip.montanaro at gmail.com Sat Apr 7 19:33:59 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sat, 7 Apr 2018 18:33:59 -0500 Subject: [Python-Dev] Trying to build from source, test-poplib fails In-Reply-To: <162a270fac0.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> References: <162a270fac0.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> Message-ID: > Do you have ca-certificates installed? It seems so: % apt search ca-certificates | grep installed ca-certificates/artful,artful,now 20170717 all [installed] ca-certificates-mono/artful,artful,now 4.6.2.7+dfsg-1ubuntu1 all [installed,automatic] liblwp-protocol-https-perl/artful,artful,now 6.07-2 all [installed] S From steve at pearwood.info Sat Apr 7 20:51:53 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 8 Apr 2018 10:51:53 +1000 Subject: [Python-Dev] Soliciting comments on the future of the cmd module (bpo-33233) In-Reply-To: References: <7CAA437E-0734-41F4-83AB-0FAD07262DB1@python.org> <20180407001331.GZ16661@ando.pearwood.info> Message-ID: <20180408005153.GF16661@ando.pearwood.info> On Sat, Apr 07, 2018 at 02:50:00PM -0400, Chris Barker - NOAA Federal wrote: > Is bringing cmd2 into the standard library an option to be considered? That is discussed on the tracker. The short answer is, yes, it is considered, but no, cmd2 is not ready to come into the std lib. I recommend you check out the discussion here: https://bugs.python.org/issue33233 -- Steve From njs at pobox.com Sat Apr 7 21:28:08 2018 From: njs at pobox.com (Nathaniel Smith) Date: Sat, 7 Apr 2018 18:28:08 -0700 Subject: [Python-Dev] Soliciting comments on the future of the cmd module (bpo-33233) In-Reply-To: References: <7CAA437E-0734-41F4-83AB-0FAD07262DB1@python.org> <20180407001331.GZ16661@ando.pearwood.info> Message-ID: On Sat, Apr 7, 2018 at 6:54 AM, Mat?j Cepl wrote: > Also, considering requests, I am still dreaming about somebody > writing some requests-like API over the standard library. What would be the difference between that and... requests? Requests still uses http.client under the hood... -n -- Nathaniel J. Smith -- https://vorpus.org From ncoghlan at gmail.com Sat Apr 7 23:17:36 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 8 Apr 2018 13:17:36 +1000 Subject: [Python-Dev] Are undocumented functions part of the stable ABI? In-Reply-To: References: <5AC493BC.8050104@UGent.be> <43a821a6be244016816c02de52fe0126@xmail102.UGent.be> <5AC5B57F.1040402@UGent.be> Message-ID: On 6 April 2018 at 14:45, Guido van Rossum wrote: > As you may guess from the silence, it may be hard to get a definitive answer > to this question -- PEP 384's author has stopped actively participating in > the Python community and I'm not sure if any core developers currently > consider themselves to be the "guardians of the ABI". It's only been a few days, and there are definitely some of us that genuinely want the stable ABI to meet its guarantees (e.g. me, Steve Dower, Eric Snow, Victor Stinner). It just isn't likely to come fully into its own until more library developers are able to drop support for Python 2.7. > That said, from a quick skim of PEP 384 it seems to be quite strict in its > position that anything not explicitly included by the PEP should be > considered not part of the ABI, which makes me think that only a few > PyCFunction related items are considered part of the ABI (searching for > PyCFunction only finds three hits). OTOH it states that if Py_LIMITED_API is > defined, all definitions that are not part of the ABI will be hidden. So > from that (plus the source code) you should be able to tell which > PyCFunction_* functions are part of the ABI -- again it does not appear the > documentation status of a function matters for this rule. > > On the third hand, PEP 384 references a file "python3.def" > (https://www.python.org/dev/peps/pep-0384/#id5) and that file contains > several PyCFunction_* names. Maybe this is the hard rule you're looking for? > Again, being documented is not a requirement. > > Another observation would be that (AFAICT) PEP 384 strictly forbids > signature changes, but is mostly silent on semantics. Right, PEP 384 is mainly focused on low level ABI compatibility: whether or not the extension module loader will even be able to import the module without getting segfaults due signature mismatches or struct size changes. Semantic questions are a bit different, as the question there isn't "Will the module segfault?" it's "Will it work as expected?", and that question applies regardless of whether you recompile it or not. > On Wed, Apr 4, 2018 at 10:34 PM, Jeroen Demeyer wrote: >> The context is PEP 575. I guess my question is mostly about >> PyCFunction_Check(). I will not be able to keep it 100% backwards compatible >> simply because the goal of that PEP is precisely changing the classes of >> some objects. >> >> Now the question is: am I allowed to change the implementation of >> PyCFunction_Check()? If it's considered part of the stable ABI, then the >> answer is immediately "no". Changing macro definitions doesn't break the stable ABI, as long as the *old* macro expansions still do the right thing. So in the case of PEP 575, the following change would be OK (since the old macro expansion would still work): * add PyCFunction_CheckExact() to replace the current meaning of PyCFunction_Check * adjust PyCFunction_Check() to allow subclasses In a lot of ways, you're actually better off than if these were functions, as extension modules built against the stable ABI with an old version won't even see the semantic change - they'll implicitly keep PyCFunction_CheckExact semantics. >> By the way, does anybody happen to know why the PyCFunction_* functions >> are undocumented? Is it just an oversight in the docs or is it intentional? Just an oversight as far as I am aware. >> But regardless of the context, I think that the question "Are undocumented >> functions part of the stable ABI?" should be answered in PEP 384. We'd put any answer to that question in https://docs.python.org/3/c-api/stable.html, rather than in PEP 384 (although it may make sense to link from the PEP back to the docs for usage questions, for the benefit of folks that find the PEP first). There isn't really a universal answer, though - the closest we get to that is the fact that we default to "No, they're not part of the stable ABI" when they have an underscore prefix (and hence aren't even part of the public API), and "Yes, they are covered by the stable ABI" otherwise (it may be a bug that they escaped into the stable ABI in the first place, but once they're there, they're there). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From tritium-list at sdamon.com Sun Apr 8 00:45:41 2018 From: tritium-list at sdamon.com (Alex Walters) Date: Sun, 8 Apr 2018 00:45:41 -0400 Subject: [Python-Dev] Is it useful to update cgitb module? In-Reply-To: References: Message-ID: <030f01d3cef4$7303d750$590b85f0$@sdamon.com> Are there people still actively developing new cgi scripts in python? I know some modern HTTPDs don?t even support classic cgi without some kind of fastcgi daemon in between. I am aware that some parts of various wsgi tools use the cgi module, but is the cgitb module useful for them? Your suggestions might be good ideas, but I don?t know if they would be used. I feel like its kind of like updating the macpath module - sure you can make code improvements to it if you want, but its for a workflow that is very rarely used. > -----Original Message----- > From: Python-Dev list=sdamon.com at python.org> On Behalf Of St?phane Blondon > Sent: Saturday, April 7, 2018 4:21 PM > To: python-dev at python.org > Subject: [Python-Dev] Is it useful to update cgitb module? > > Hello, > > I wonder if it's useful to update the cgitb module, in particular the > html output. > I see some possible improvements: > > 1. In both text and html versions: > > When a module is called, there are no parameters (displayed as '()'). I > think they are unnecessary. Perhaps the parentheses should be removed? > Perhaps it's better to keep them for backward compatibility? > > ### example for the text version ### > $ python3 demo.py > [...] > /home/stephane/src/cgitest/demo.py in () > 7 def func1(a, b): > [...] > ### end of example ### > > 2. In html version only: > a. If the executed code is in : in this case, it is not shown > in the html version because the square brackets are interpreted as a > html tag (see the picture in attachement). > b. Update the style of the html or/and using html5. It would be > prettier but it will be a big change for probably too few benefits. > > What do you think about them? I can report bugs and send pull-requests > for them but I would prefer to get feedbacks before. > > Regards, > St?phane From v+python at g.nevcal.com Sun Apr 8 02:28:24 2018 From: v+python at g.nevcal.com (Glenn Linderman) Date: Sat, 7 Apr 2018 23:28:24 -0700 Subject: [Python-Dev] Is it useful to update cgitb module? In-Reply-To: <030f01d3cef4$7303d750$590b85f0$@sdamon.com> References: <030f01d3cef4$7303d750$590b85f0$@sdamon.com> Message-ID: On 4/7/2018 9:45 PM, Alex Walters wrote: > Are there people still actively developing new cgi scripts in python? I know some modern HTTPDs don?t even support classic cgi without some kind of fastcgi daemon in between. I am aware that some parts of various wsgi tools use the cgi module, but is the cgitb module useful for them? Yes. I have several web sites and applications built as Python CGI scripts.? cgitb is extremely useful. -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Sun Apr 8 04:43:23 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 8 Apr 2018 10:43:23 +0200 Subject: [Python-Dev] Are undocumented functions part of the stable ABI? References: <5AC493BC.8050104@UGent.be> <43a821a6be244016816c02de52fe0126@xmail102.UGent.be> <5AC5B57F.1040402@UGent.be> Message-ID: <20180408104323.6a2279a5@fsol> On Sun, 8 Apr 2018 13:17:36 +1000 Nick Coghlan wrote: > On 6 April 2018 at 14:45, Guido van Rossum wrote: > > As you may guess from the silence, it may be hard to get a definitive answer > > to this question -- PEP 384's author has stopped actively participating in > > the Python community and I'm not sure if any core developers currently > > consider themselves to be the "guardians of the ABI". > > It's only been a few days, and there are definitely some of us that > genuinely want the stable ABI to meet its guarantees (e.g. me, Steve > Dower, Eric Snow, Victor Stinner). It just isn't likely to come fully > into its own until more library developers are able to drop support > for Python 2.7. Note there are issues with PyType_FromSpec() which make the stable ABI rather delicate to use: https://bugs.python.org/issue15727 https://bugs.python.org/issue16690 Regards Antoine. From solipsis at pitrou.net Sun Apr 8 12:10:59 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 8 Apr 2018 18:10:59 +0200 Subject: [Python-Dev] Trying to build from source, test-poplib fails References: Message-ID: <20180408181059.67c339a5@fsol> On Sat, 7 Apr 2018 17:31:47 -0500 Skip Montanaro wrote: > It's been a long while since I rebuilt Python from the Git source. I > tried for the first time the other day. Everything passed except > test_poplib and test_asyncio. The former just runs and runs and runs. > Here's the first traceback I encounter when executing ./python > Lib/test/test_poplib.py: > > test_stls_context (__main__.TestPOP3Class) ... Exception in thread Thread-16: > Traceback (most recent call last): > File "/home/skip/src/python/cpython/Lib/threading.py", line 917, in > _bootstrap_inner > self.run() > File "Lib/test/test_poplib.py", line 227, in run > asyncore.loop(timeout=0.1, count=1) > File "/home/skip/src/python/cpython/Lib/asyncore.py", line 207, in loop > poll_fun(timeout, map) > File "/home/skip/src/python/cpython/Lib/asyncore.py", line 150, in poll > read(obj) > File "/home/skip/src/python/cpython/Lib/asyncore.py", line 87, in read > obj.handle_error() > File "/home/skip/src/python/cpython/Lib/asyncore.py", line 83, in read > obj.handle_read_event() > File "/home/skip/src/python/cpython/Lib/asyncore.py", line 422, in > handle_read_event > self.handle_read() > File "Lib/test/test_poplib.py", line 194, in handle_read > self._do_tls_handshake() > File "Lib/test/test_poplib.py", line 174, in _do_tls_handshake > self.socket.do_handshake() > File "/home/skip/src/python/cpython/Lib/ssl.py", line 1108, in do_handshake > self._sslobj.do_handshake() > ssl.SSLError: [SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert > certificate unknown (_ssl.c:1049) [...] I get the same issues too, and this has been happening for quite some time. The tests and/or poplib itself may be written in a fragile way. Regards Antoine. From python-dev at mgmiller.net Sun Apr 8 15:51:28 2018 From: python-dev at mgmiller.net (Mike Miller) Date: Sun, 8 Apr 2018 12:51:28 -0700 Subject: [Python-Dev] Soliciting comments on the future of the cmd module (bpo-33233) In-Reply-To: <20180407090841.GD16661@ando.pearwood.info> References: <7CAA437E-0734-41F4-83AB-0FAD07262DB1@python.org> <40J1sK5hgRzFr6k@mail.python.org> <20180407090841.GD16661@ando.pearwood.info> Message-ID: <7462c03a-6068-b804-faac-e37c8bdab968@mgmiller.net> On 2018-04-07 02:08, Steven D'Aprano wrote: > This isn't gopher, or something with serious unfixable security > vulnerabilities. It works. What more needs to be said? Interesting, I'd forgotten about the module but this thread brought it from dusty backup tape back into my brain. Part of the problem may be the generic name. I just used it for a programming challenge and it worked quite well. :-) The source is close to trivial, not a burden, and I found the module useful and easy to use. Only nitpick is that the "repeat last action on empty line" is a poor default I think. -Mike p.s. The bug listed above has the authors of cmd2 planning a simplified/compatible version that could become the next "cmd." From steve at pearwood.info Sun Apr 8 20:36:37 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 9 Apr 2018 10:36:37 +1000 Subject: [Python-Dev] Soliciting comments on the future of the cmdmodule (bpo-33233) In-Reply-To: References: <7CAA437E-0734-41F4-83AB-0FAD07262DB1@python.org> <40J1sK5hgRzFr6k@mail.python.org> <20180407090841.GD16661@ando.pearwood.info> Message-ID: <20180409003637.GJ16661@ando.pearwood.info> On Sat, Apr 07, 2018 at 06:53:44PM +0000, Brett Cannon wrote: > > Raymond has stated that he is happy to work on it if there are any bugs > > reported on it, and if he's not available, I'm sure somebody will. > > Actually Raymond said he *teaches* the module, not that he wanted to > maintain it. And I definitely would not assume that someone will pick up to > help maintain any module in the stdlib. Raymond said: "If a bug were reported for cmd, I would be happy to work on it." https://bugs.python.org/issue33233#msg315061 -- Steve From tritium-list at sdamon.com Mon Apr 9 04:00:05 2018 From: tritium-list at sdamon.com (Alex Walters) Date: Mon, 9 Apr 2018 04:00:05 -0400 Subject: [Python-Dev] Is it useful to update cgitb module? In-Reply-To: References: <030f01d3cef4$7303d750$590b85f0$@sdamon.com> Message-ID: <3a7c201d3cfd8$c6403aa0$52c0afe0$@sdamon.com> Extremely useful ? for CGI scripts. Now, my sample size is quite small ? mainly the #python irc channel on freenode, but I have been informally asking for a few years, and you are the first person I have encountered who admitted to writing and maintaining new CGI based sites. I still say the value of supporting such an obsolete model is? questionable. That said, I have no power to block commits. From: Python-Dev On Behalf Of Glenn Linderman Sent: Sunday, April 8, 2018 2:28 AM To: python-dev at python.org Subject: Re: [Python-Dev] Is it useful to update cgitb module? On 4/7/2018 9:45 PM, Alex Walters wrote: Are there people still actively developing new cgi scripts in python? I know some modern HTTPDs don?t even support classic cgi without some kind of fastcgi daemon in between. I am aware that some parts of various wsgi tools use the cgi module, but is the cgitb module useful for them? Yes. I have several web sites and applications built as Python CGI scripts. cgitb is extremely useful. -------------- next part -------------- An HTML attachment was scrubbed... URL: From storchaka at gmail.com Mon Apr 9 04:22:23 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Mon, 9 Apr 2018 11:22:23 +0300 Subject: [Python-Dev] Is it useful to update cgitb module? In-Reply-To: References: Message-ID: 07.04.18 23:21, St?phane Blondon ????: > I wonder if it's useful to update the cgitb module, in particular the > html output. > I see some possible improvements: > > 1. In both text and html versions: > > When a module is called, there are no parameters (displayed as '()'). I > think they are unnecessary. Perhaps the parentheses should be removed? > Perhaps it's better to keep them for backward compatibility? > > ### example for the text version ### > $ python3 demo.py > [...] > /home/stephane/src/cgitest/demo.py in () > 7 def func1(a, b): > [...] > ### end of example ### > > 2. In html version only: > a. If the executed code is in : in this case, it is not shown > in the html version because the square brackets are interpreted as a > html tag (see the picture in attachement). > b. Update the style of the html or/and using html5. It would be > prettier but it will be a big change for probably too few benefits. > > What do you think about them? I can report bugs and send pull-requests > for them but I would prefer to get feedbacks before. 2a is definitely a bug. Please open an issue on the bug tracker for. 1 can be considered as a bug or as a request for enhancement. Needed to look at the code closer. And 2b is a separate request for enhancement. All three should be separate issues. From storchaka at gmail.com Mon Apr 9 04:30:17 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Mon, 9 Apr 2018 11:30:17 +0300 Subject: [Python-Dev] Trying to build from source, test-poplib fails In-Reply-To: <20180408181059.67c339a5@fsol> References: <20180408181059.67c339a5@fsol> Message-ID: 08.04.18 19:10, Antoine Pitrou ????: > On Sat, 7 Apr 2018 17:31:47 -0500 > Skip Montanaro wrote: >> It's been a long while since I rebuilt Python from the Git source. I >> tried for the first time the other day. Everything passed except >> test_poplib and test_asyncio. The former just runs and runs and runs. >> Here's the first traceback I encounter when executing ./python >> Lib/test/test_poplib.py: >> >> test_stls_context (__main__.TestPOP3Class) ... Exception in thread Thread-16: >> Traceback (most recent call last): >> File "/home/skip/src/python/cpython/Lib/threading.py", line 917, in >> _bootstrap_inner >> self.run() >> File "Lib/test/test_poplib.py", line 227, in run >> asyncore.loop(timeout=0.1, count=1) >> File "/home/skip/src/python/cpython/Lib/asyncore.py", line 207, in loop >> poll_fun(timeout, map) >> File "/home/skip/src/python/cpython/Lib/asyncore.py", line 150, in poll >> read(obj) >> File "/home/skip/src/python/cpython/Lib/asyncore.py", line 87, in read >> obj.handle_error() >> File "/home/skip/src/python/cpython/Lib/asyncore.py", line 83, in read >> obj.handle_read_event() >> File "/home/skip/src/python/cpython/Lib/asyncore.py", line 422, in >> handle_read_event >> self.handle_read() >> File "Lib/test/test_poplib.py", line 194, in handle_read >> self._do_tls_handshake() >> File "Lib/test/test_poplib.py", line 174, in _do_tls_handshake >> self.socket.do_handshake() >> File "/home/skip/src/python/cpython/Lib/ssl.py", line 1108, in do_handshake >> self._sslobj.do_handshake() >> ssl.SSLError: [SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert >> certificate unknown (_ssl.c:1049) > [...] > > I get the same issues too, and this has been happening for quite some > time. The tests and/or poplib itself may be written in a fragile way. I get the different issue with test_poplib (running with the -j option). All tests except test_multiprocessing_fork are passed, and test_poplib hangs. I thought it is a misconfiguration on my computer. From christian at python.org Mon Apr 9 05:54:41 2018 From: christian at python.org (Christian Heimes) Date: Mon, 9 Apr 2018 11:54:41 +0200 Subject: [Python-Dev] Trying to build from source, test-poplib fails In-Reply-To: References: <162a270fac0.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> Message-ID: On 2018-04-08 01:33, Skip Montanaro wrote: >> Do you have ca-certificates installed? > > It seems so: > > % apt search ca-certificates | grep installed > > ca-certificates/artful,artful,now 20170717 all [installed] > ca-certificates-mono/artful,artful,now 4.6.2.7+dfsg-1ubuntu1 all > [installed,automatic] > liblwp-protocol-https-perl/artful,artful,now 6.07-2 all [installed] Skip, it's a red herring. Python's test_poplib suite doesn't depend on public CA certs. From songofacandy at gmail.com Mon Apr 9 05:59:41 2018 From: songofacandy at gmail.com (INADA Naoki) Date: Mon, 9 Apr 2018 18:59:41 +0900 Subject: [Python-Dev] Trying to build from source, test-poplib fails In-Reply-To: References: Message-ID: FYI, there is filed issue. https://bugs.python.org/issue33099 From christian at python.org Mon Apr 9 06:01:37 2018 From: christian at python.org (Christian Heimes) Date: Mon, 9 Apr 2018 12:01:37 +0200 Subject: [Python-Dev] Trying to build from source, test-poplib fails In-Reply-To: References: <20180408181059.67c339a5@fsol> Message-ID: On 2018-04-09 10:30, Serhiy Storchaka wrote: >> I get the same issues too, and this has been happening for quite some >> time.? The tests and/or poplib itself may be written in a fragile way. > > I get the different issue with test_poplib (running with the -j option). > All tests except test_multiprocessing_fork are passed, and test_poplib > hangs. I thought it is a misconfiguration on my computer. It's not your computer, it's the test suite. It uses the old asyncore library. The test has become unstable since the ssl module properly checks host names, see https://bugs.python.org/issue32706 and https://bugs.python.org/issue32753. In Python 3.6 and earlier, a hostname verification error resulted in a closed connection. Starting with Python 3.7, the ssl module checks the hostname earlier in the TLS handshake. A verification error results in a TLS handshake alert and an exception on the server side. Sometimes the old server code in asyncore runs into a race condition and the test fails. The whole test framework needs to be rewritten from scratch and replaced with something better. I haven't had time to work on it. Christian From tseaver at palladion.com Mon Apr 9 09:47:57 2018 From: tseaver at palladion.com (Tres Seaver) Date: Mon, 9 Apr 2018 09:47:57 -0400 Subject: [Python-Dev] Is it useful to update cgitb module? In-Reply-To: <030f01d3cef4$7303d750$590b85f0$@sdamon.com> References: <030f01d3cef4$7303d750$590b85f0$@sdamon.com> Message-ID: On 04/08/2018 12:45 AM, Alex Walters wrote: > Are there people still actively developing new cgi scripts in python? In addition to the "magic" support triggered by 'cgitb.enable()', the module also exposes useful utility functions ('cgitb.html()' and 'cgitb.text()') which are used by non-CGI third-party libraries to render tracebacks for debugging purposes. Enhancing those methods seems pretty reasonable to me. Tres. -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com From jmcs at jsantos.eu Mon Apr 9 02:20:41 2018 From: jmcs at jsantos.eu (=?UTF-8?B?Sm/Do28gU2FudG9z?=) Date: Mon, 09 Apr 2018 06:20:41 +0000 Subject: [Python-Dev] Is it useful to update cgitb module? In-Reply-To: References: <030f01d3cef4$7303d750$590b85f0$@sdamon.com> Message-ID: cgitb is useful even outside of cgi scripts. I've used "cgitb.enable(format='text')" in the several scripts to get "better" exceptions. On Sun, 8 Apr 2018 at 08:52 Glenn Linderman wrote: > On 4/7/2018 9:45 PM, Alex Walters wrote: > > Are there people still actively developing new cgi scripts in python? I know some modern HTTPDs don?t even support classic cgi without some kind of fastcgi daemon in between. I am aware that some parts of various wsgi tools use the cgi module, but is the cgitb module useful for them? > > > Yes. I have several web sites and applications built as Python CGI > scripts. cgitb is extremely useful. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/jmcs%40jsantos.eu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From J.Demeyer at UGent.be Tue Apr 10 01:34:30 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Tue, 10 Apr 2018 07:34:30 +0200 Subject: [Python-Dev] Are undocumented functions part of the stable ABI? In-Reply-To: <070379a116bc4cf2a96b3d4010623fcd@xmail102.UGent.be> References: <5AC493BC.8050104@UGent.be> <43a821a6be244016816c02de52fe0126@xmail102.UGent.be> <5AC5B57F.1040402@UGent.be> <070379a116bc4cf2a96b3d4010623fcd@xmail102.UGent.be> Message-ID: <5ACC4CE6.1010808@UGent.be> On 2018-04-08 05:17, Nick Coghlan wrote: > Changing macro definitions doesn't break the stable ABI, as long as > the *old* macro expansions still do the right thing. To me, it looks like a bad idea to change macros. Imagine that the PyCFunction_Check macro changes in Python 3.8. Then an extension module compiled on 3.7 (but run on 3.8) would behave differently from the same extension compiled on 3.8. I cannot imagine that this is in line with the "stable ABI" philosophy. Jeroen. From ncoghlan at gmail.com Tue Apr 10 07:49:03 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 10 Apr 2018 21:49:03 +1000 Subject: [Python-Dev] Are undocumented functions part of the stable ABI? In-Reply-To: <5ACC4CE6.1010808@UGent.be> References: <5AC493BC.8050104@UGent.be> <43a821a6be244016816c02de52fe0126@xmail102.UGent.be> <5AC5B57F.1040402@UGent.be> <070379a116bc4cf2a96b3d4010623fcd@xmail102.UGent.be> <5ACC4CE6.1010808@UGent.be> Message-ID: On 10 April 2018 at 15:34, Jeroen Demeyer wrote: > On 2018-04-08 05:17, Nick Coghlan wrote: >> >> Changing macro definitions doesn't break the stable ABI, as long as >> the *old* macro expansions still do the right thing. > > > To me, it looks like a bad idea to change macros. Imagine that the > PyCFunction_Check macro changes in Python 3.8. Then an extension module > compiled on 3.7 (but run on 3.8) would behave differently from the same > extension compiled on 3.8. I cannot imagine that this is in line with the > "stable ABI" philosophy. It's fine, as the stable ABI philosophy is simply that extension modules compiled against a given version of the stable ABI (as defined by Py_LIMITED_API) will be binary compatible with all interpreter versions that support that ABI. So if a macro is changed to reference a newly exported symbol, then the new definition *must* be put behind an "#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03080000" guard. If it's only a semantic level change in the way the macro gets expanded, then whether or not it needs an ABI version guard gets judged on a case-by-case basis, and in this particular case, my view would be that developers should be able to write extensions using the stable ABI that accept function subclasses on 3.8+, without having to *require* the use of 3.8+ to import their module. For folks that explicitly *don't* want to allow subclasses, even when compiled with 3.8+, we can offer the following snippet in the Python 3.8 porting guide: #ifndef PyCFunction_CheckExact #define PyCFunction_CheckExact PyCFunction_Check #endif And then doing a search-and-replace on their code to use the "*Exact" variant. Ideally stable ABI users won't need to do that though, since the stable ABI hides struct internals, and all the public PyCFunction APIs should be updated to cope with subclasses properly. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From J.Demeyer at UGent.be Wed Apr 11 04:46:18 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Wed, 11 Apr 2018 10:46:18 +0200 Subject: [Python-Dev] Are undocumented functions part of the stable ABI? In-Reply-To: References: <5AC493BC.8050104@UGent.be> <43a821a6be244016816c02de52fe0126@xmail102.UGent.be> <5AC5B57F.1040402@UGent.be> <070379a116bc4cf2a96b3d4010623fcd@xmail102.UGent.be> <5ACC4CE6.1010808@UGent.be> Message-ID: <5ACDCB5A.4030501@UGent.be> On 2018-04-10 13:49, Nick Coghlan wrote: > If it's only a semantic level change in the way the macro gets > expanded, then whether or not it needs an ABI version guard gets > judged on a case-by-case basis, and in this particular case, my view > would be that developers should be able to write extensions using the > stable ABI that accept function subclasses on 3.8+, without having to > *require* the use of 3.8+ to import their module. I don't really get this paragraph, but in any case I decided to *not* change PyCFunction_Check in PEP 575. It doesn't seem worth the trouble as this macro is probably not often used anyway. Also, it's hard to guess what it should be replaced with: why would extensions be calling PyCFunction_Check()? From jsbueno at python.org.br Wed Apr 11 07:21:01 2018 From: jsbueno at python.org.br (Joao S. O. Bueno) Date: Wed, 11 Apr 2018 08:21:01 -0300 Subject: [Python-Dev] Possible undefined behavior on creating a method named "__dict__" Message-ID: I just came across a code snippet that would define a method with the "__dict__" name - like in: class A: def __dict__(self): return () The resulting class's instances can be assigned dynamic attributes as usual, but one can never acess its actual local variables through instance.__dict__ - the method is retrieved instead. Calling "vars" will also fail on objects of this class. This behavior is weird, and I believe is actually a side-effect of implementation details on CPython. I am not sure whether it shoud just: 1 - be left as is - whoever reuses __dict__ as a method had it coming 2 - document CPythn behavior 3 - file that as a bug to disallow __dict__ override in class declaration 4 - file that as a bug to not-create class __dict__ when one is explictly created in Python code (the same that happens when one have "__slots__". I have the feeling that (1) is just good - but then, I am at least posting this e-mail here. Similar weird things go when one creates a method named "__class__", and possible other names. (I just checked that pypy3 mimics the behavior) js -><- From steve at pearwood.info Wed Apr 11 08:08:06 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 11 Apr 2018 22:08:06 +1000 Subject: [Python-Dev] Possible undefined behavior on creating a method named "__dict__" In-Reply-To: References: Message-ID: <20180411120806.GS16661@ando.pearwood.info> On Wed, Apr 11, 2018 at 08:21:01AM -0300, Joao S. O. Bueno wrote: > I just came across a code snippet that > would define a method with the "__dict__" name - like in: > > class A: > def __dict__(self): > return () That's a strange thing to do, but I don't think it ought to be illegal. Consenting adults and all that. > The resulting class's instances can be assigned > dynamic attributes as usual, but one can never acess > its actual local variables through instance.__dict__ - > the method is retrieved instead. Yes, I believe that is expected behaviour for attribute access since the descriptor protocol was added. Methods take priority over data attributes, if I recall correctly. > Calling "vars" will also fail on objects of this class. I consider that a pseudo-bug. I can't call it an actual bug, because vars() doesn't document that it will work even when __dict__ is shadowed in this way, but I think it should. So its a bug against a future feature :-) Attribute access still works correctly even with such a shadow: py> class W: ... def __dict__(self): ... return () ... py> obj = W() py> obj.spam = 1 py> obj.spam 1 so there is still an instance dict somewhere inside the instance, and the C attribute-access machinary can access it. I think vars() should be able to do the same. (I'm not saying this in order to encourage people to shadow __dict__.) > This behavior is weird, and I believe is actually a side-effect > of implementation details on CPython. Its certain a weird thing to do, but I don't believe it is an implementation detail. Apart from the behaviour of vars(), I think the behaviour here all follows from the documented behaviour of the descriptor protocol. > I am not sure whether it shoud just: > 1 - be left as is - whoever reuses __dict__ as a method had it coming > 2 - document CPythn behavior > 3 - file that as a bug to disallow __dict__ override in class declaration > 4 - file that as a bug to not-create class __dict__ when one is explictly > created in Python code (the same that happens when one have "__slots__". > > I have the feeling that (1) is just good - but then, I am at least > posting this e-mail here. I agree that (1) is the best, but vars() ought to work even in the precence of a method shadowing __dict__. > Similar weird things go when one creates a method named "__class__", > and possible other names. type(instance) still works correctly when instance.__class__ is shadowed by a method. -- Steve From larry at hastings.org Wed Apr 11 12:23:13 2018 From: larry at hastings.org (Larry Hastings) Date: Wed, 11 Apr 2018 09:23:13 -0700 Subject: [Python-Dev] Reminder: 2018 Python Language Summit signups close next week Message-ID: <0c45559d-bdcd-376a-056a-7718708707b5@hastings.org> The deadline is a week from today, April 18th 2018.? Original announcement below. -- It?s that time again: time to start thinking about the Python Language Summit!? The 2018 summit will be held on Wednesday, May 9, from 10am to 4pm, at the Huntington Convention Center in Cleveland, Ohio, USA.? Your befezzled and befuddled hosts Barry and Larry will once more be behind the big desk in front. The summit?s purpose is to disseminate information and spark conversation among core Python developers.? It?s our yearly opportunity to get together for an in-person discussion, to review interesting developments of the previous year and hash out where we?re going next.? And we have lots to talk about! 3.7 is in beta, and we've all collectively started work on 3.8 too. As before, we?re using Google Forms to collect signups.? Signups are open now; the deadline to sign up is Wednesday April 18th, 2018 (AoE).? Please do us a favor and sign up sooner rather than later.? The signup form is simpler this year--I bet most people can be done in less than two minutes! One difference from last year: there are now *two* forms.? The first form is for signing up to attend (the "Request For Invitation" form), and the second form is for proposing a talk. Please note: if you want to present, you still need to fill out the Request For Invitation form too.? (Yes, it's more complicated this way, sorry.? But having both on the same form kind of enforced a one-to-one mapping, and it's really a many-to-many mapping: one person might propose multiple talks, and one talk might have multiple presenters.? Overall this should be less complicated.) You can find links to *both* forms on the official Python Language Summit 2018 page: https://us.pycon.org/2018/events/language-summit/ A few notes: * There will be lightning talks!? Signups will only be available during the Language Summit itself. * You don?t need to be registered for PyCon in order to attend the summit! * We?ll have badge ribbons for Language Summit participants, which we?ll hand out at the summit room in the morning. * We're inviting Jake Edge from Linux Weekly News to attend the summit and provide press coverage again.? Jake?s done a phenomenal job of covering the last few summits, providing valuable information not just for summit attendees, but also for the Python community at large.? Jake?s coverage goes a long way toward demystifying the summit, while remaining respectful of confidential information that?s deemed ?off the record? ahead of time by participants. One big final note (please read this!): When using Google Forms, you /can/ edit your responses later!? When you fill out the form and hit Submit, the submission complete page (the one that says "Thanks for playing!") will have a link on it labeled "Edit your response".? BOOKMARK THIS LINK!? You can use this link at /any time/ to edit your response, up to the point that signups close on April 18th.? Keep in mind, you'll need to bookmark each response independently: once for signing up to attend ("Request For Invitation"), and once for each talk proposal you submit.? Again, /please/ be sure to save this bookmark yourself--we don't know how to find the link for you later if you don't save it. We hope to see you at the summit! [BL]arry -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Wed Apr 11 13:46:00 2018 From: brett at python.org (Brett Cannon) Date: Wed, 11 Apr 2018 17:46:00 +0000 Subject: [Python-Dev] Possible undefined behavior on creating a method named "__dict__" In-Reply-To: <20180411120806.GS16661@ando.pearwood.info> References: <20180411120806.GS16661@ando.pearwood.info> Message-ID: On Wed, 11 Apr 2018 at 05:09 Steven D'Aprano wrote: > On Wed, Apr 11, 2018 at 08:21:01AM -0300, Joao S. O. Bueno wrote: > > I just came across a code snippet that > > would define a method with the "__dict__" name - like in: > > > > class A: > > def __dict__(self): > > return () > > That's a strange thing to do, but I don't think it ought to be illegal. > Consenting adults and all that. > > > > The resulting class's instances can be assigned > > dynamic attributes as usual, but one can never acess > > its actual local variables through instance.__dict__ - > > the method is retrieved instead. > > Yes, I believe that is expected behaviour for attribute access since the > descriptor protocol was added. Methods take priority over data > attributes, if I recall correctly. > Yep, they do. > > > > Calling "vars" will also fail on objects of this class. > > I consider that a pseudo-bug. I can't call it an actual bug, because > vars() doesn't document that it will work even when __dict__ is shadowed > in this way, but I think it should. So its a bug against a future > feature :-) > > Attribute access still works correctly even with such a shadow: > > py> class W: > ... def __dict__(self): > ... return () > ... > py> obj = W() > py> obj.spam = 1 > py> obj.spam > 1 > > so there is still an instance dict somewhere inside the instance, and > the C attribute-access machinary can access it. I think vars() should be > able to do the same. > > (I'm not saying this in order to encourage people to shadow __dict__.) > > > > This behavior is weird, and I believe is actually a side-effect > > of implementation details on CPython. > > Its certain a weird thing to do, but I don't believe it is an > implementation detail. Apart from the behaviour of vars(), I think the > behaviour here all follows from the documented behaviour of the > descriptor protocol. > > > > I am not sure whether it should just: > > 1 - be left as is - whoever reuses __dict__ as a method had it coming > > 2 - document CPython behavior > > 3 - file that as a bug to disallow __dict__ override in class declaration > > 4 - file that as a bug to not-create class __dict__ when one is explictly > > created in Python code (the same that happens when one have > "__slots__". > > > > I have the feeling that (1) is just good - but then, I am at least > > posting this e-mail here. > > I agree that (1) is the best, but vars() ought to work even in the > presence of a method shadowing __dict__. > I say option 1 as well. -Brett > > > > Similar weird things go when one creates a method named "__class__", > > and possible other names. > > type(instance) still works correctly when instance.__class__ is shadowed > by a method. > > > -- > Steve > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Wed Apr 11 20:25:30 2018 From: guido at python.org (Guido van Rossum) Date: Wed, 11 Apr 2018 17:25:30 -0700 Subject: [Python-Dev] Possible undefined behavior on creating a method named "__dict__" In-Reply-To: <20180411120806.GS16661@ando.pearwood.info> References: <20180411120806.GS16661@ando.pearwood.info> Message-ID: On Wed, Apr 11, 2018 at 5:08 AM, Steven D'Aprano wrote: > On Wed, Apr 11, 2018 at 08:21:01AM -0300, Joao S. O. Bueno wrote: > > I just came across a code snippet that > > would define a method with the "__dict__" name - like in: > > > > class A: > > def __dict__(self): > > return () > > That's a strange thing to do, but I don't think it ought to be illegal. > Consenting adults and all that. Python's guarantee in this case goes no further than that it promises not to crash in C code. There's a rule in the language reference that says that all __dunder__ names are reserved for the implementation and they should only be used according to the documentation. So, indeed, it's not illegal, but you are not guaranteed that anything works, either. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at ethanhs.me Thu Apr 12 04:59:53 2018 From: ethan at ethanhs.me (Ethan Smith) Date: Thu, 12 Apr 2018 01:59:53 -0700 Subject: [Python-Dev] PEP 561 implemented and minor clarification Message-ID: Hello, I've updated PEP 561 to clarify that any installed stub package should supersede an installed inline package. In other words if there is: /global/site-packages/pkg/ /user/site-packages/pkg-stubs/ Even if pkg in the global site packages is found first and marks that it supports types, the stub package should supersede it. I also point to mypy's docs on its implementation of the PEP (which can be read about here: https://mypy.readthedocs.io/en/latest/installed_packages.html). The implementation has been merged into master and will be available in the 0.590 release. As always, the PEP text is replicated below. Cheers! Ethan ============== PEP: 561 Title: Distributing and Packaging Type Information Author: Ethan Smith Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 09-Sep-2017 Python-Version: 3.7 Post-History: 10-Sep-2017, 12-Sep-2017, 06-Oct-2017, 26-Oct-2017 Abstract ======== PEP 484 introduced type hinting to Python, with goals of making typing gradual and easy to adopt. Currently, typing information must be distributed manually. This PEP provides a standardized means to leverage existing tooling to package and distribute type information with minimal work and an ordering for type checkers to resolve modules and collect this information for type checking. Rationale ========= Currently, package authors wish to distribute code that has inline type information. Additionally, maintainers would like to distribute stub files to keep Python 2 compatibility while using newer annotation syntax. However, there is no standard method to distribute packages with type information. Also, if one wished to ship stub files privately the only method available would be via setting ``MYPYPATH`` or the equivalent to manually point to stubs. If the package can be released publicly, it can be added to typeshed [1]_. However, this does not scale and becomes a burden on the maintainers of typeshed. In addition, it ties bug fixes in stubs to releases of the tool using typeshed. PEP 484 has a brief section on distributing typing information. In this section [2]_ the PEP recommends using ``shared/typehints/pythonX.Y/`` for shipping stub files. However, manually adding a path to stub files for each third party library does not scale. The simplest approach people have taken is to add ``site-packages`` to their ``MYPYPATH``, but this causes type checkers to fail on packages that are highly dynamic (e.g. sqlalchemy and Django). Definition of Terms =================== The definition of "MAY", "MUST", and "SHOULD", and "SHOULD NOT" are to be interpreted as described in RFC 2119. "inline" - the types are part of the runtime code using PEP 526 and 3107 syntax. "stubs" - files containing only type information, empty of runtime code. "Distributions" are the packaged files which are used to publish and distribute a release. [3]_ "Module" a file containing Python runtime code or stubbed type information. "Package" a directory or directories that namespace Python modules. Specification ============= There are several motivations and methods of supporting typing in a package. This PEP recognizes three (3) types of packages that users of typing wish to create: 1. The package maintainer would like to add type information inline. 2. The package maintainer would like to add type information via stubs. 3. A third party or package maintainer would like to share stub files for a package, but the maintainer does not want to include them in the source of the package. This PEP aims to support these scenarios and make them simple to add to packaging and deployment. The two major parts of this specification are the packaging specifications and the resolution order for resolving module type information. The type checking spec is meant to replace the ``shared/typehints/pythonX.Y/`` spec of PEP 484 [2]_. New third party stub libraries SHOULD distribute stubs via the third party packaging methods proposed in this PEP in place of being added to typeshed. Typeshed will remain in use, but if maintainers are found, third party stubs in typeshed MAY be split into their own package. Packaging Type Information -------------------------- In order to make packaging and distributing type information as simple and easy as possible, packaging and distribution is done through existing frameworks. Package maintainers who wish to support type checking of their code MUST add a marker file named ``py.typed`` to their package supporting typing. This marker applies recursively: if a top-level package includes it, all its sub-packages MUST support type checking as well. To have this file installed with the package, maintainers can use existing packaging options such as ``package_data`` in distutils, shown below. Distutils option example:: setup( ..., package_data = { 'foopkg': ['py.typed'], }, ..., ) For namespace packages, the ``py.typed`` file should be in the submodules of the namespace, to avoid conflicts and for clarity. This PEP does not support distributing typing information as part of module-only distributions. The code should be refactored into a package-based distribution and indicate that the package supports typing as described above. Stub Only Packages '''''''''''''''''' For package maintainers wishing to ship stub files containing all of their type information, it is preferred that the ``*.pyi`` stubs are alongside the corresponding ``*.py`` files. However, the stubs can also be put in a separate package and distributed separately. Third parties can also find this method useful if they wish to distribute stub files. The name of the stub package MUST follow the scheme ``foopkg-stubs`` for type stubs for the package named ``foopkg``. Note that for stub only packages adding a py.typed marker is not needed since the name ``*-stubs`` is enough to indicate it is a source of typing information. Third parties seeking to distribute stub files are encouraged to contact the maintainer of the package about distribution alongside the package. If the maintainer does not wish to maintain or package stub files or type information inline, then a third party stub only package can be created. In addition, stub-only distributions SHOULD indicate which version(s) of the runtime package are supported by indicating the runtime distribution's version(s) through normal dependency data. For example, the stub package ``flyingcircus-stubs`` can indicate the versions of the runtime ``flyingcircus`` distribution it supports through ``install_requires`` in distutils-based tools, or the equivalent in other packaging tools. Note that in pip 9.0, if you update ``flyingcircus-stubs``, it will update ``flyingcircus``. In pip 9.0, you can use the ``--upgrade-strategy=only-if-needed`` flag. In pip 10.0 this is the default behavior. Type Checker Module Resolution Order ------------------------------------ The following is the order in which type checkers supporting this PEP SHOULD resolve modules containing type information: 1. User code - the files the type checker is running on. 2. Stubs or Python source manually put in the beginning of the path. Type checkers SHOULD provide this to allow the user complete control of which stubs to use, and patch broken stubs/inline types from packages. 3. Stub packages - these packages SHOULD supersede any installed inline package. They can be found at ``foopkg-stubs`` for package ``foopkg``. 4. Inline packages - if there is nothing overriding the installed package, and it opts into type checking, inline types SHOULD be used. 5. Typeshed (if used) - Provides the stdlib types and several third party libraries. Type checkers that check a different Python version than the version they run on MUST find the type information in the ``site-packages``/``dist-packages`` of that Python version. This can be queried e.g. ``pythonX.Y -c 'import site; print(site.getsitepackages())'``. It is also recommended that the type checker allow for the user to point to a particular Python binary, in case it is not in the path. The normal resolution order of checking ``*.pyi`` before ``*.py`` will be maintained. Implementation ============== The proposed scheme of indicating support for typing is completely backwards compatible, and requires no modification to package tooling. A sample package with inline types is available [typed_pkg]_, as well as a sample package checker [pkg_checker]_ which reads the metadata of installed packages and reports on their status as either not typed, inline typed, or a stub package. The mypy type checker has an implementation of PEP 561 searching which can be read about in the mypy docs [4]_. Acknowledgements ================ This PEP would not have been possible without the ideas, feedback, and support of Ivan Levkivskyi, Jelle Zijlstra, Nick Coghlan, Daniel F Moisset, Nathaniel Smith, and Guido van Rossum. Version History =============== * 2018-04-09 * Add reference to mypy implementation * Clarify stub package priority. * 2018-02-02 * Change stub only package suffix to be -stubs not _stubs. * Note that py.typed is not needed for stub only packages. * Add note about pip and upgrading stub packages. * 2017-11-12 * Rewritten to use existing tooling only * No need to indicate kind of type information in metadata * Name of marker file changed from ``.typeinfo`` to ``py.typed`` * 2017-11-10 * Specification re-written to use package metadata instead of distribution metadata. * Removed stub only packages and merged into third party packages spec. * Removed suggestion for typecheckers to consider checking runtime versions * Implementations updated to reflect PEP changes. * 2017-10-26 * Added implementation references. * Added acknowledgements and version history. * 2017-10-06 * Rewritten to use .distinfo/METADATA over a distutils specific command. * Clarify versioning of third party stub packages. * 2017-09-11 * Added information about current solutions and typeshed. * Clarify rationale. References ========== .. [1] Typeshed (https://github.com/python/typeshed) .. [2] PEP 484, Storing and Distributing Stub Files (https://www.python.org/dev/peps/pep-0484/#storing-and-distributing-stub-files) .. [3] PEP 426 definitions (https://www.python.org/dev/peps/pep-0426/) .. [4] Example implementation in a type checker (https://mypy.readthedocs.io/en/latest/installed_packages.html) .. [typed_pkg] Sample typed package (https://github.com/ethanhs/sample-typed-package) .. [pkg_checker] Sample package checker (https://github.com/ethanhs/check_typedpkg) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: -------------- next part -------------- An HTML attachment was scrubbed... URL: From nikitarulz at gmail.com Thu Apr 12 06:16:14 2018 From: nikitarulz at gmail.com (Nikita Gupta) Date: Thu, 12 Apr 2018 15:46:14 +0530 Subject: [Python-Dev] python3.5.3: xmlrpc_net unit test fail Message-ID: Hi All, Before installing python 3.5.3, i performed the test cases provided with the source tar ball. But 3 tests failed. I figured out the reason for other 2. But 1 test case is out of my understanding. Re-running test 'test_xmlrpc_net' in verbose mode test_python_builders (test.test_xmlrpc_net.PythonBuildersTest) ... ERROR ====================================================================== ERROR: test_python_builders (test.test_xmlrpc_net.PythonBuildersTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/pkgs/Python-3.5.3/Lib/test/test_xmlrpc_net.py", line 17, in test_python_builders builders = server.getAllBuilders() File "/pkgs/Python-3.5.3/Lib/xmlrpc/client.py", line 1092, in __call__ return self.__send(self.__name, args) File "/pkgs/Python-3.5.3/Lib/xmlrpc/client.py", line 1432, in __request verbose=self.__verbose File "/pkgs/Python-3.5.3/Lib/xmlrpc/client.py", line 1134, in request return self.single_request(host, handler, request_body, verbose) File "/pkgs/Python-3.5.3/Lib/xmlrpc/client.py", line 1167, in single_request dict(resp.getheaders()) xmlrpc.client.ProtocolError: ---------------------------------------------------------------------- Ran 1 test in 0.215s FAILED (errors=1) test test_xmlrpc_net failed Can somebody highlight the reason of failure. -- Regards, Nikita Gupta -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodrigc at freebsd.org Thu Apr 12 11:50:54 2018 From: rodrigc at freebsd.org (Craig Rodrigues) Date: Thu, 12 Apr 2018 08:50:54 -0700 Subject: [Python-Dev] python3.5.3: xmlrpc_net unit test fail In-Reply-To: References: Message-ID: That unit test was written when https://buildbot.python.org was running at Buildbot version 8 which had an XMLRPC interface. Last year, buildbot.python.org was upgraded to Buildbot version 9, which dropped the XMLRPC interface in favor of a REST API. So that link no longer works. The unit change was changed to skip: https://github.com/python/cpython/blob/master/Lib/test/test_xmlrpc_net.py#L8 -- Craig On Thu, Apr 12, 2018 at 3:16 AM, Nikita Gupta wrote: > Hi All, > > Before installing python 3.5.3, i performed the test cases provided with > the source tar ball. > But 3 tests failed. I figured out the reason for other 2. But 1 test case > is out of my understanding. > > Re-running test 'test_xmlrpc_net' in verbose mode > test_python_builders (test.test_xmlrpc_net.PythonBuildersTest) ... ERROR > > ====================================================================== > ERROR: test_python_builders (test.test_xmlrpc_net.PythonBuildersTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/pkgs/Python-3.5.3/Lib/test/test_xmlrpc_net.py", line 17, in > test_python_builders > builders = server.getAllBuilders() > File "/pkgs/Python-3.5.3/Lib/xmlrpc/client.py", line 1092, in __call__ > return self.__send(self.__name, args) > File "/pkgs/Python-3.5.3/Lib/xmlrpc/client.py", line 1432, in __request > verbose=self.__verbose > File "/pkgs/Python-3.5.3/Lib/xmlrpc/client.py", line 1134, in request > return self.single_request(host, handler, request_body, verbose) > File "/pkgs/Python-3.5.3/Lib/xmlrpc/client.py", line 1167, in > single_request > dict(resp.getheaders()) > xmlrpc.client.ProtocolError: xmlrpc/: 503 Service Unavailable> > > ---------------------------------------------------------------------- > Ran 1 test in 0.215s > > FAILED (errors=1) > test test_xmlrpc_net failed > > Can somebody highlight the reason of failure. > > > -- > Regards, > Nikita Gupta > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > rodrigc%40freebsd.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From J.Demeyer at UGent.be Thu Apr 12 12:12:02 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Thu, 12 Apr 2018 18:12:02 +0200 Subject: [Python-Dev] PEP 575: Unifying function/method classes Message-ID: <5ACF8552.6020607@UGent.be> Dear Python developers, I would like to request a review of PEP 575, which is about changing the classes used for built-in functions and Python functions and methods. The text of the PEP can be found at https://www.python.org/dev/peps/pep-0575/ No substantial changes to the contents of the PEP were made compared to the first posting. However, many details have been changed, clarified or added, based on comments from the initial discussion thread and the work on an implementation. My implementation is at https://github.com/jdemeyer/cpython/tree/pep575 This is certainly not meant to be ready to merge upstream; in particular, the Python test suite does not fully pass. Nevertheless, it should be good enough to review the PEP. If the PEP would be accepted, I plan to continue working on the implementation, including adding tests and documentation. Jeroen. From ncoghlan at gmail.com Fri Apr 13 09:23:44 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 13 Apr 2018 23:23:44 +1000 Subject: [Python-Dev] PEP 575: Unifying function/method classes In-Reply-To: <5ACF8552.6020607@UGent.be> References: <5ACF8552.6020607@UGent.be> Message-ID: On 13 April 2018 at 02:12, Jeroen Demeyer wrote: > Dear Python developers, > > I would like to request a review of PEP 575, which is about changing the > classes used for built-in functions and Python functions and methods. The > text of the PEP can be found at > > https://www.python.org/dev/peps/pep-0575/ > > No substantial changes to the contents of the PEP were made compared to the > first posting. However, many details have been changed, clarified or added, > based on comments from the initial discussion thread and the work on an > implementation. I'm personally +1 on this version of the PEP (thank you for putting it together!), but have one request for clarification related to `__doc__`: at first glance, it isn't obvious why there are two different ways of storing the docstring (either m_ml->ml_doc or a dedicated func_code field). Those technically *could* be consolidated by always using m_ml->ml_doc, and changing the definition of base_function such that m_ml always pointed to a PyMethodDef instance. However, that's less than ideal when the doc string is already a Python object, since you need to convert between "const char *" and "PyObject *" when accessing the field from Python (and vice versa when setting it). There's also a section in the rationale which refers to METH_USRx flags, which I'm guessing from context are an idea you were considering proposing, but eventually dropped from the rest of the PEP. These remaining references to the concept just need to be cleaned up. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From songofacandy at gmail.com Fri Apr 13 09:27:16 2018 From: songofacandy at gmail.com (INADA Naoki) Date: Fri, 13 Apr 2018 22:27:16 +0900 Subject: [Python-Dev] Timing for removing legacy Unicode APIs deprecated by PEP 393 Message-ID: Hi, PEP 393 [1] deprecates some Unicode APIs relating to Py_UNICODE. The PEP doesn't provide schedule for removing them. But the APIs are marked "will be removed in 4.0" in the document. When removing them, we can reduce `wchar_t *` member of unicode object. It takes 8 bytes on 64bit platform. [1]: "Flexible String Representation" https://www.python.org/dev/peps/pep-0393/ I thought Python 4.0 is the next version of 3.9. But Guido has different idea. He said following at Zulip chat (we're trying it for now). > No, 4.0 is not just what comes after 3.9 -- the major number change would indicate some kind of major change somewhere (like possibly the Gilectomy, which changes a lot of the C APIs). If we have more than 10 3.x versions, we'll just live with 3.10, 3.11 etc. And he said about these APIs: >> Unicode objects has some "Deprecated since version 3.3, will be removed in version 4.0" APIs (pep-393). >> When removing them, we can reduce PyUnicode size about 8~12byte. > > We should be able to deprecate these sooner by updating the docs. Then, I want to reschedule the removal of these APIs. Can we remove them in 3.8? 3.9? or 3.10? I prefer sooner as possible. --- Slightly off topic, there are 4bytes alignment gap in the unicode object, on 64bit platform. typedef struct { .... struct { unsigned int interned:2; unsigned int kind:3; unsigned int compact:1; unsigned int ascii:1; unsigned int ready:1; unsigned int :24; } state; // 4 bytes // implicit 4 bytes gap here. wchar_t *wstr; // 8 bytes } PyASCIIObject; So, I think we can reduce 12 bytes instead of 8 bytes when removing wstr. Or we can reduce 4 bytes soon by moving `wstr` before `state`. Off course, it needs siphash support 4byte aligned data instead of 8byte. Regards, -- INADA Naoki From J.Demeyer at UGent.be Fri Apr 13 09:30:27 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Fri, 13 Apr 2018 15:30:27 +0200 Subject: [Python-Dev] PEP 575: Unifying function/method classes In-Reply-To: References: <5ACF8552.6020607@UGent.be> Message-ID: <5AD0B0F3.5010005@UGent.be> To make it easier to test and try out PEP 575, I created a binder repo: https://mybinder.org/v2/gh/jdemeyer/pep575binder.git /master?filepath=index.ipynb From status at bugs.python.org Fri Apr 13 12:09:55 2018 From: status at bugs.python.org (Python tracker) Date: Fri, 13 Apr 2018 18:09:55 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20180413160955.7ACFD56856@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2018-04-06 - 2018-04-13) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 6572 ( +4) closed 38452 (+32) total 45024 (+36) Open issues with patches: 2560 Issues opened (24) ================== #33238: AssertionError on await of Future returned by asyncio.wrap_fut https://bugs.python.org/issue33238 opened by Jason Haydaman #33239: tempfile module: functions with the 'buffering' option are inc https://bugs.python.org/issue33239 opened by MartyMacGyver #33240: shutil.rmtree fails when the inner floder is opened in Explore https://bugs.python.org/issue33240 opened by yuliu #33245: Unable to send CTRL_BREAK_EVENT https://bugs.python.org/issue33245 opened by Ofekmeister #33249: Unpickling objects with recursive references and partials fail https://bugs.python.org/issue33249 opened by aleneum #33250: Move VERSION attribute output up in pydoc (via help() builtin) https://bugs.python.org/issue33250 opened by handle #33251: ConfigParser.items returns items present in vars https://bugs.python.org/issue33251 opened by timster #33252: Clarify ResourceWarning documentation https://bugs.python.org/issue33252 opened by edmorley #33254: importlib.resources.contents() incorrectly yields an empty lis https://bugs.python.org/issue33254 opened by brett.cannon #33255: json.dumps has different behaviour if encoding='utf-8' or enco https://bugs.python.org/issue33255 opened by nhatcher #33256: module is not displayed by cgitb.html https://bugs.python.org/issue33256 opened by sblondon #33257: Race conditions in Tkinter with non-threaded Tcl https://bugs.python.org/issue33257 opened by Ivan.Pozdeev #33258: Unable to install 3.6.5 on Windows Server 2008 https://bugs.python.org/issue33258 opened by hpo0016 #33259: Encoding issue in the name of the local DST timezone https://bugs.python.org/issue33259 opened by maggyero #33261: inspect.isgeneratorfunction fails on hand-created methods https://bugs.python.org/issue33261 opened by jdemeyer #33262: Deprecate shlex.split(None) to read from stdin. https://bugs.python.org/issue33262 opened by christian.heimes #33263: Asyncio server enters an invalid state after a request with SO https://bugs.python.org/issue33263 opened by drtyrsa #33264: Remove to-be-deprecated urllib.request.urlretrieve function re https://bugs.python.org/issue33264 opened by adelfino #33266: 2to3 doesn't parse all valid string literals https://bugs.python.org/issue33266 opened by Zsolt Dollenstein #33267: ctypes array types create reference cycles https://bugs.python.org/issue33267 opened by Eric.Wieser #33269: InteractiveConsole behaves differently when used on terminal a https://bugs.python.org/issue33269 opened by Kadir Haldenbilen #33270: tags for anonymous code objects should be interned https://bugs.python.org/issue33270 opened by Daniel Moisset #33271: Exception handling matches subtypes, not subclasses https://bugs.python.org/issue33271 opened by Michael McCoy #33272: Which are reasonable reason for recursion limit in function _v https://bugs.python.org/issue33272 opened by mv.gavrilov Most recent 15 issues with no replies (15) ========================================== #33272: Which are reasonable reason for recursion limit in function _v https://bugs.python.org/issue33272 #33271: Exception handling matches subtypes, not subclasses https://bugs.python.org/issue33271 #33270: tags for anonymous code objects should be interned https://bugs.python.org/issue33270 #33263: Asyncio server enters an invalid state after a request with SO https://bugs.python.org/issue33263 #33261: inspect.isgeneratorfunction fails on hand-created methods https://bugs.python.org/issue33261 #33259: Encoding issue in the name of the local DST timezone https://bugs.python.org/issue33259 #33257: Race conditions in Tkinter with non-threaded Tcl https://bugs.python.org/issue33257 #33256: module is not displayed by cgitb.html https://bugs.python.org/issue33256 #33255: json.dumps has different behaviour if encoding='utf-8' or enco https://bugs.python.org/issue33255 #33251: ConfigParser.items returns items present in vars https://bugs.python.org/issue33251 #33250: Move VERSION attribute output up in pydoc (via help() builtin) https://bugs.python.org/issue33250 #33236: MagicMock().__iter__.return_value is different from MagicMock( https://bugs.python.org/issue33236 #33234: Improve list() pre-sizing for inputs with known lengths https://bugs.python.org/issue33234 #33225: imaplib module IMAP4.append() unexpected response BAD Command https://bugs.python.org/issue33225 #33220: Antivirus hits on python-2.7.14.amd64.msi file https://bugs.python.org/issue33220 Most recent 15 issues waiting for review (15) ============================================= #33271: Exception handling matches subtypes, not subclasses https://bugs.python.org/issue33271 #33266: 2to3 doesn't parse all valid string literals https://bugs.python.org/issue33266 #33264: Remove to-be-deprecated urllib.request.urlretrieve function re https://bugs.python.org/issue33264 #33263: Asyncio server enters an invalid state after a request with SO https://bugs.python.org/issue33263 #33261: inspect.isgeneratorfunction fails on hand-created methods https://bugs.python.org/issue33261 #33257: Race conditions in Tkinter with non-threaded Tcl https://bugs.python.org/issue33257 #33256: module is not displayed by cgitb.html https://bugs.python.org/issue33256 #33251: ConfigParser.items returns items present in vars https://bugs.python.org/issue33251 #33239: tempfile module: functions with the 'buffering' option are inc https://bugs.python.org/issue33239 #33237: Improve AttributeError message for partially initialized modul https://bugs.python.org/issue33237 #33228: Use Random.choices in tempfile https://bugs.python.org/issue33228 #33227: Cmd do_something only accepts one argument https://bugs.python.org/issue33227 #33222: Various test failures if PYTHONUSERBASE is not canonicalized https://bugs.python.org/issue33222 #33217: x in enum.Flag member is True when x is not a Flag https://bugs.python.org/issue33217 #33216: Wrong order of stack for CALL_FUNCTION_VAR and CALL_FUNCTION_V https://bugs.python.org/issue33216 Top 10 most discussed issues (10) ================================= #33233: Suggest third-party cmd2 module as alternative to cmd https://bugs.python.org/issue33233 15 msgs #33240: shutil.rmtree fails when the inner floder is opened in Explore https://bugs.python.org/issue33240 7 msgs #33099: test_poplib hangs with the changes done in PR https://bugs.python.org/issue33099 6 msgs #11122: bdist_rpm should use rpmbuild, not rpm https://bugs.python.org/issue11122 5 msgs #33227: Cmd do_something only accepts one argument https://bugs.python.org/issue33227 4 msgs #33237: Improve AttributeError message for partially initialized modul https://bugs.python.org/issue33237 4 msgs #33264: Remove to-be-deprecated urllib.request.urlretrieve function re https://bugs.python.org/issue33264 4 msgs #23403: Use pickle protocol 4 by default? https://bugs.python.org/issue23403 3 msgs #24882: ThreadPoolExecutor doesn't reuse threads until #threads == max https://bugs.python.org/issue24882 3 msgs #33065: IDLE debugger: problem importing user created module https://bugs.python.org/issue33065 3 msgs Issues closed (31) ================== #8243: curses writing to window's bottom right position raises: `_cur https://bugs.python.org/issue8243 closed by ned.deily #24068: statistics module - incorrect results with boolean input https://bugs.python.org/issue24068 closed by wolma #28832: Reduce memset in dict creation https://bugs.python.org/issue28832 closed by inada.naoki #29613: Support for SameSite Cookies https://bugs.python.org/issue29613 closed by alex #29673: Some gdb macros are broken in 3.6 https://bugs.python.org/issue29673 closed by ned.deily #30686: make `make install` faster https://bugs.python.org/issue30686 closed by inada.naoki #31201: make test: module test that failed doesn't exist https://bugs.python.org/issue31201 closed by ned.deily #31584: Documentation Language mixed up https://bugs.python.org/issue31584 closed by mdk #31920: pygettext ignores directories as inputfile argument https://bugs.python.org/issue31920 closed by serhiy.storchaka #32759: multiprocessing.Array do not release shared memory https://bugs.python.org/issue32759 closed by pitrou #33031: Questionable code in OrderedDict definition https://bugs.python.org/issue33031 closed by serhiy.storchaka #33097: concurrent futures Executors accept tasks after interpreter sh https://bugs.python.org/issue33097 closed by pitrou #33169: importlib.invalidate_caches() doesn't clear all caches https://bugs.python.org/issue33169 closed by brett.cannon #33188: dataclass MRO entry resolution for type variable metaclasses: https://bugs.python.org/issue33188 closed by levkivskyi #33200: Optimize the empty set "literal" https://bugs.python.org/issue33200 closed by serhiy.storchaka #33201: Modernize "Extension types" documentation https://bugs.python.org/issue33201 closed by pitrou #33219: x in IntFlag should test raise TypeError if x is not an IntFla https://bugs.python.org/issue33219 closed by ethan.furman #33229: Documentation - io ??? Core tools for working with streams - https://bugs.python.org/issue33229 closed by Mikhail Zakharov #33230: _decimal build failure (unsupported platform for that module) https://bugs.python.org/issue33230 closed by skrah #33241: Tooltip not display with macOS 64-bit installer 3.6.5 but work https://bugs.python.org/issue33241 closed by ned.deily #33242: Support binary symbol names https://bugs.python.org/issue33242 closed by eryksun #33243: nltk is not working properly https://bugs.python.org/issue33243 closed by serhiy.storchaka #33244: Overflow error https://bugs.python.org/issue33244 closed by terry.reedy #33246: Note in 18.2. json ??? JSON encoder and decoder is incorrect https://bugs.python.org/issue33246 closed by inada.naoki #33247: File "" is opened and read unconditionally https://bugs.python.org/issue33247 closed by serhiy.storchaka #33248: __await__ behaves different with or without PYTHONASYNCIODEBUG https://bugs.python.org/issue33248 closed by Jonas Obrist #33253: xxsubtype.bench does not function correctly on CPython 3+ https://bugs.python.org/issue33253 closed by serhiy.storchaka #33260: Update token.py etc for ASYNC and AWAIT https://bugs.python.org/issue33260 closed by serhiy.storchaka #33265: contextlib.ExitStack abuses __self__ https://bugs.python.org/issue33265 closed by ncoghlan #33268: iteration over attrs in metaclass' __new__ affects class' __na https://bugs.python.org/issue33268 closed by eric.smith #33273: Allow multiple imports from one module while preserving its na https://bugs.python.org/issue33273 closed by zach.ware From raymond.hettinger at gmail.com Fri Apr 13 15:30:13 2018 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Fri, 13 Apr 2018 12:30:13 -0700 Subject: [Python-Dev] PEP 575: Unifying function/method classes In-Reply-To: <5ACF8552.6020607@UGent.be> References: <5ACF8552.6020607@UGent.be> Message-ID: > On Apr 12, 2018, at 9:12 AM, Jeroen Demeyer wrote: > > I would like to request a review of PEP 575, which is about changing the classes used for built-in functions and Python functions and methods. The text of the PEP can be found at > > https://www.python.org/dev/peps/pep-0575/ Thanks for doing this work. The PEP is well written and I'm +1 on the general idea of what it's trying to do (I'm still taking in all the details). It would be nice to have a section that specifically discusses the implications with respect to other existing function-like tooling: classmethod, staticmethod, partial, itemgetter, attrgetter, methodgetter, etc. Also, please mention the backward compatibility issue that will arise for code that currently relies on types.MethodType, types.BuiltinFunctionType, types.BuiltinMethodType, etc. For example, I would need to update the code in random._randbelow(). That code uses the existing builtin-vs-pure-python type distinctions to determine whether either the random() or getrandbits() methods have been overridden. This is likely an easy change for me to make, but there may be code like it the wild, code that would be broken if the distinction is lost. Raymond From J.Demeyer at UGent.be Sat Apr 14 05:17:25 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Sat, 14 Apr 2018 11:17:25 +0200 Subject: [Python-Dev] PEP 575: Unifying function/method classes In-Reply-To: References: <5ACF8552.6020607@UGent.be> Message-ID: <5AD1C725.5090603@UGent.be> On 2018-04-13 21:30, Raymond Hettinger wrote: > It would be nice to have a section that specifically discusses the implications with respect to other existing function-like tooling: classmethod, staticmethod, partial, itemgetter, attrgetter, methodgetter, etc. My hope is that there are no such implications. An important design goal of this PEP (which I believe I achieved) is that as long as you're doing duck typing, you should be safe. I believe that the tools in your list do exactly that. It's only when you use inspect or when you do type checks that you will see the difference with this PEP. After implementing the C code part of my PEP, there were only a relatively small number of test failures. You can look at this commit which contains all Python code changes of my implementation, it doesn't look so bad: https://github.com/jdemeyer/cpython/commit/c404a8f1b7d9525dd2842712fe183a051a4b5094 > For example, I would need to update the code in random._randbelow(). For the record, there are no test failures related to this, but maybe that's just because tests for this are missing. From J.Demeyer at UGent.be Sat Apr 14 09:49:37 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Sat, 14 Apr 2018 15:49:37 +0200 Subject: [Python-Dev] PEP 575: Unifying function/method classes In-Reply-To: References: <5ACF8552.6020607@UGent.be> Message-ID: <5AD206F1.3040305@UGent.be> On 2018-04-13 15:23, Nick Coghlan wrote: > There's also a section in the rationale which refers to METH_USRx > flags, which I'm guessing from context are an idea you were > considering proposing, but eventually dropped from the rest of the > PEP. No, I actually still want to propose it. In my latest update of the PEP, I hope to have made it more clear. From guido at python.org Sat Apr 14 17:14:40 2018 From: guido at python.org (Guido van Rossum) Date: Sat, 14 Apr 2018 14:14:40 -0700 Subject: [Python-Dev] PEP 575: Unifying function/method classes In-Reply-To: <5AD1C725.5090603@UGent.be> References: <5ACF8552.6020607@UGent.be> <5AD1C725.5090603@UGent.be> Message-ID: On Sat, Apr 14, 2018 at 2:17 AM, Jeroen Demeyer wrote: > On 2018-04-13 21:30, Raymond Hettinger wrote: > >> It would be nice to have a section that specifically discusses the >> implications with respect to other existing function-like tooling: >> classmethod, staticmethod, partial, itemgetter, attrgetter, methodgetter, >> etc. >> > > My hope is that there are no such implications. An important design goal > of this PEP (which I believe I achieved) is that as long as you're doing > duck typing, you should be safe. I believe that the tools in your list do > exactly that. > > It's only when you use inspect or when you do type checks that you will > see the difference with this PEP. > That actually sounds like a pretty big problem. I'm sure there is lots of code that doesn't *just* duck-type nor calls inspect but uses isinstance() to decide how to extract the desired information. > After implementing the C code part of my PEP, there were only a relatively > small number of test failures. You can look at this commit which contains > all Python code changes of my implementation, it doesn't look so bad: > > https://github.com/jdemeyer/cpython/commit/c404a8f1b7d9525dd > 2842712fe183a051a4b5094 > > For example, I would need to update the code in random._randbelow(). >> > > For the record, there are no test failures related to this, but maybe > that's just because tests for this are missing. > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Sat Apr 14 22:41:19 2018 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 14 Apr 2018 19:41:19 -0700 Subject: [Python-Dev] [RELEASE] Python 2.7.15 release candidate 1 Message-ID: <1523760079.2728906.1338273840.27DC2547@webmail.messagingengine.com> I'm pleased to announce the immediate availability of Python 2.7.15 release candidate 1. Python 2.7.15rc1 is a preview release of the next bug fix release in the Python 2.7.x series. Python 2.7.15rc1 may be downloaded in source and binary forms from https://www.python.org/downloads/release/python-2715rc1/ Please consider testing this release with your applications and libraries and reporting bugs to https://bugs.python.org/ A final release is expected in 2 weeks time. 2.7.15 includes some noteworthy changes to the macOS installers: All python.org macOS installers now ship with a built-in copy of OpenSSL. Additionally, there is a new additional installer variant for macOS 10.9+ that includes a built-in version of Tcl/Tk 8.6. See the installer README for more information. Thank you, Benjamin (on behalf of 2.7's release team and contributors) From J.Demeyer at UGent.be Sun Apr 15 08:50:14 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Sun, 15 Apr 2018 14:50:14 +0200 Subject: [Python-Dev] PEP 575: Unifying function/method classes In-Reply-To: <3b065fca4bbd416b9d71afab455a4c2c@xmail101.UGent.be> References: <5ACF8552.6020607@UGent.be> <5AD1C725.5090603@UGent.be> <3b065fca4bbd416b9d71afab455a4c2c@xmail101.UGent.be> Message-ID: <5AD34A86.7030404@UGent.be> On 2018-04-14 23:14, Guido van Rossum wrote: > That actually sounds like a pretty big problem. I'm sure there is lots > of code that doesn't *just* duck-type nor calls inspect but uses > isinstance() to decide how to extract the desired information. In the CPython standard library, the *only* fixes that are needed because of this are in: - inspect (obviously) - doctest (to figure out the __module__ of an arbitrary object) - multiprocessing.reduction (something to do with pickling) - xml.etree.ElementTree (to determine whether a certain method was overridden) - GDB support I've been told that there might also be a problem with Random._randbelow, even though it doesn't cause test failures. The fact that there is so little breakage in the standard library makes me confident that the problem is not so bad. And in the cases where it does break, it's usually pretty easy to fix. Finally: changing the classes of certain objects is exactly the point of this PEP, so it's impossible to achieve 100% backwards compatibility. Jeroen. From ncoghlan at gmail.com Sun Apr 15 09:09:16 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 15 Apr 2018 23:09:16 +1000 Subject: [Python-Dev] PEP 575: Unifying function/method classes In-Reply-To: <5AD34A86.7030404@UGent.be> References: <5ACF8552.6020607@UGent.be> <5AD1C725.5090603@UGent.be> <3b065fca4bbd416b9d71afab455a4c2c@xmail101.UGent.be> <5AD34A86.7030404@UGent.be> Message-ID: On 15 April 2018 at 22:50, Jeroen Demeyer wrote: > On 2018-04-14 23:14, Guido van Rossum wrote: >> >> That actually sounds like a pretty big problem. I'm sure there is lots >> of code that doesn't *just* duck-type nor calls inspect but uses >> isinstance() to decide how to extract the desired information. > > In the CPython standard library, the *only* fixes that are needed because of > this are in: > > - inspect (obviously) > - doctest (to figure out the __module__ of an arbitrary object) > - multiprocessing.reduction (something to do with pickling) > - xml.etree.ElementTree (to determine whether a certain method was > overridden) > - GDB support > > I've been told that there might also be a problem with Random._randbelow, > even though it doesn't cause test failures. >From Raymond's description (and from an initial reading of the code), the _randbelow case seems like it may be a latent defect anyway, as it wouldn't detect cases where the replacement implementation came from an extension module (e.g. if someone used Cython to compile a module that subclassed Random and overrode either Random.random or Random.getrandbits). ("Builtin" is unfortunately a bit of a misnomer in the existing type names, since extension modules also create instances of those types) In a post-PEP-575 world, I believe those checks could be replaced with the more robust "if random.__func__ is __class__.random or getrandbits.__func__ is not __class__.getrandbits:" (since unbound methods go away even for builtin functions, it becomes easier to check method identity against a baseline implementation). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From storchaka at gmail.com Sun Apr 15 09:49:26 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sun, 15 Apr 2018 16:49:26 +0300 Subject: [Python-Dev] PEP 575: Unifying function/method classes In-Reply-To: References: <5ACF8552.6020607@UGent.be> <5AD1C725.5090603@UGent.be> <3b065fca4bbd416b9d71afab455a4c2c@xmail101.UGent.be> <5AD34A86.7030404@UGent.be> Message-ID: 15.04.18 16:09, Nick Coghlan ????: > On 15 April 2018 at 22:50, Jeroen Demeyer wrote: >> I've been told that there might also be a problem with Random._randbelow, >> even though it doesn't cause test failures. > > From Raymond's description (and from an initial reading of the code), > the _randbelow case seems like it may be a latent defect anyway, as it > wouldn't detect cases where the replacement implementation came from > an extension module (e.g. if someone used Cython to compile a module > that subclassed Random and overrode either Random.random or > Random.getrandbits). ("Builtin" is unfortunately a bit of a misnomer > in the existing type names, since extension modules also create > instances of those types) > > In a post-PEP-575 world, I believe those checks could be replaced with > the more robust "if random.__func__ is __class__.random or > getrandbits.__func__ is not __class__.getrandbits:" (since unbound > methods go away even for builtin functions, it becomes easier to check > method identity against a baseline implementation). See https://bugs.python.org/issue33144. From fromwheretowhere.service at gmail.com Sun Apr 15 12:41:16 2018 From: fromwheretowhere.service at gmail.com (Xuan Wu) Date: Sun, 15 Apr 2018 09:41:16 -0700 Subject: [Python-Dev] Translating sample programs in documentation Message-ID: Excuse me if this was discussed before, but in French and Japanese translations, all the sample programs seem to have identifiers in English still. According to "PEP 545 -- Python Documentation Translations", as I understand .po files are used for translations. May I ask if there's technical restrictions causing translations being only applied to the text parts? For example, here's the first sample program in 4.2: >>># Measure some strings: ... words = ['cat', 'window', 'defenestrate'] >>>for w in words: ... print(w, len(w)) ... cat 3 window 6 defenestrate 12 Here's a possible translation in Chinese: >>> # ??????? ... ?? = ['?', '??', '????'] >>> for ? in ??: ... print(?, len(?)) ... ? 1 ?? 2 ???? 4 As you may notice the strings differ in size if they are translated directly. Obviously that does add extra burden to review the new sample programs to assure effectiveness and readability. Any suggestion or comments are welcome. Thanks, Xuan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.hettinger at gmail.com Sun Apr 15 20:32:32 2018 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Sun, 15 Apr 2018 17:32:32 -0700 Subject: [Python-Dev] PEP 575: Unifying function/method classes In-Reply-To: <5AD34A86.7030404@UGent.be> References: <5ACF8552.6020607@UGent.be> <5AD1C725.5090603@UGent.be> <3b065fca4bbd416b9d71afab455a4c2c@xmail101.UGent.be> <5AD34A86.7030404@UGent.be> Message-ID: > On Apr 15, 2018, at 5:50 AM, Jeroen Demeyer wrote: > > On 2018-04-14 23:14, Guido van Rossum wrote: >> That actually sounds like a pretty big problem. I'm sure there is lots >> of code that doesn't *just* duck-type nor calls inspect but uses >> isinstance() to decide how to extract the desired information. > > In the CPython standard library, the *only* fixes that are needed because of this are in: > > - inspect (obviously) > - doctest (to figure out the __module__ of an arbitrary object) > - multiprocessing.reduction (something to do with pickling) > - xml.etree.ElementTree (to determine whether a certain method was overridden) > - GDB support > > I've been told that there might also be a problem with Random._randbelow, even though it doesn't cause test failures. Don't worry about Random._randbelow, we're already working on it and it is an easy fix. Instead, focus on Guido's comment. > The fact that there is so little breakage in the standard library makes > me confident that the problem is not so bad. And in the cases where it > does break, it's usually pretty easy to fix. I don't think that confidence is warranted. The world of Python is very large. When public APIs (such as that in the venerable types module) get changed, is virtually assured that some code will break. Raymond From shell909090 at gmail.com Sun Apr 15 22:49:04 2018 From: shell909090 at gmail.com (Shell Xu) Date: Mon, 16 Apr 2018 10:49:04 +0800 Subject: [Python-Dev] Translating sample programs in documentation In-Reply-To: References: Message-ID: Well, I'm not sure weather or not this is what you're looking for, but pep-8 (https://www.python.org/dev/peps/pep-0008/) suggest like this: For Python 3.0 and beyond, the following policy is prescribed for the standard library (see PEP 3131): All identifiers in the Python standard library MUST use ASCII-only identifiers, and SHOULD use English words wherever feasible (in many cases, abbreviations and technical terms are used which aren't English). In addition, string literals and comments must also be in ASCII. The only exceptions are (a) test cases testing the non-ASCII features, and (b) names of authors. Authors whose names are not based on the Latin alphabet (latin-1, ISO/IEC 8859-1 character set) MUST provide a transliteration of their names in this character set. So, I guess translate symbols to Chinese are not gonna help reader to figure out what kind of code should they writing... On Mon, Apr 16, 2018 at 12:41 AM, Xuan Wu < fromwheretowhere.service at gmail.com> wrote: > Excuse me if this was discussed before, but in French and Japanese > translations, all the sample programs seem to have identifiers in English > still. According to "PEP 545 -- Python Documentation Translations", as I > understand .po files are used for translations. May I ask if there's > technical restrictions causing translations being only applied to the text > parts? > > For example, here's the first sample program in 4.2: > > >>> # Measure some strings:... words = ['cat', 'window', 'defenestrate']>>> for w in words:... print(w, len(w))...cat 3window 6defenestrate 12 > > Here's a possible translation in Chinese: > > >>> # ??????? > ... ?? = ['?', '??', '????'] > >>> for ? in ??: > ... print(?, len(?)) > ... > ? 1 > ?? 2 > ???? 4 > > As you may notice the strings differ in size if they are translated > directly. Obviously that does add extra burden to review the new sample > programs to assure effectiveness and readability. > Any suggestion or comments are welcome. > > > Thanks, > Xuan. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > shell909090%40gmail.com > > -- ????????????????????????????????? blog: http://shell909090.org/ twitter: @shell909090 about.me: http://about.me/shell909090 -------------- next part -------------- An HTML attachment was scrubbed... URL: From J.Demeyer at UGent.be Mon Apr 16 05:12:06 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Mon, 16 Apr 2018 11:12:06 +0200 Subject: [Python-Dev] PEP 575: Unifying function/method classes In-Reply-To: <28b31643e2fb43978c4e8e3f60490711@xmail101.UGent.be> References: <5ACF8552.6020607@UGent.be> <5AD1C725.5090603@UGent.be> <3b065fca4bbd416b9d71afab455a4c2c@xmail101.UGent.be> <5AD34A86.7030404@UGent.be> <28b31643e2fb43978c4e8e3f60490711@xmail101.UGent.be> Message-ID: <5AD468E6.9040709@UGent.be> On 2018-04-16 02:32, Raymond Hettinger wrote: > I don't think that confidence is warranted. The world of Python is very large. When public APIs (such as that in the venerable types module) get changed, is virtually assured that some code will break. Yes, *some* code will break, I never denied that. I just think that there is not much existing code which needs to distinguish between different kinds of methods, such that not much code will break. And if existing code does need to make that distinction, the reason for it might go away after PEP 575 (this is what Nick Coghlan also alluded to). The hard question is whether the expected breakage is bad enough to reject this PEP. This is my first PEP, so I honestly don't have a good idea how high the bar for backwards compatibility is. Jeroen. From erik.m.bray at gmail.com Mon Apr 16 08:35:23 2018 From: erik.m.bray at gmail.com (Erik Bray) Date: Mon, 16 Apr 2018 14:35:23 +0200 Subject: [Python-Dev] Translating sample programs in documentation In-Reply-To: References: Message-ID: On Mon, Apr 16, 2018 at 4:49 AM, Shell Xu wrote: > Well, I'm not sure weather or not this is what you're looking for, but pep-8 > (https://www.python.org/dev/peps/pep-0008/) suggest like this: > > For Python 3.0 and beyond, the following policy is prescribed for the > standard library (see PEP 3131): All identifiers in the Python standard > library MUST use ASCII-only identifiers, and SHOULD use English words > wherever feasible (in many cases, abbreviations and technical terms are used > which aren't English). In addition, string literals and comments must also > be in ASCII. The only exceptions are (a) test cases testing the non-ASCII > features, and (b) names of authors. Authors whose names are not based on the > Latin alphabet (latin-1, ISO/IEC 8859-1 character set) MUST provide a > transliteration of their names in this character set. > > So, I guess translate symbols to Chinese are not gonna help reader to figure > out what kind of code should they writing... That only applies to the Python stdlib itself. It's a feature that Python allows unicode identifiers, and there's nothing about that against PEP-8 for sure. I think it's a great idea; I'm not sure how it works out technically in terms of providing .po files for .rst documentation or if there's some better mechanism for that... Best, E > On Mon, Apr 16, 2018 at 12:41 AM, Xuan Wu > wrote: >> >> Excuse me if this was discussed before, but in French and Japanese >> translations, all the sample programs seem to have identifiers in English >> still. According to "PEP 545 -- Python Documentation Translations", as I >> understand .po files are used for translations. May I ask if there's >> technical restrictions causing translations being only applied to the text >> parts? >> >> For example, here's the first sample program in 4.2: >> >> >>> # Measure some strings: >> ... words = ['cat', 'window', 'defenestrate'] >> >>> for w in words: >> ... print(w, len(w)) >> ... >> cat 3 >> window 6 >> defenestrate 12 >> >> Here's a possible translation in Chinese: >> >> >>> # ??????? >> ... ?? = ['?', '??', '????'] >> >>> for ? in ??: >> ... print(?, len(?)) >> ... >> ? 1 >> ?? 2 >> ???? 4 >> >> As you may notice the strings differ in size if they are translated >> directly. Obviously that does add extra burden to review the new sample >> programs to assure effectiveness and readability. >> Any suggestion or comments are welcome. >> >> >> Thanks, >> Xuan. >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/shell909090%40gmail.com >> > > > > -- > ????????????????????????????????? > blog: http://shell909090.org/ > twitter: @shell909090 > about.me: http://about.me/shell909090 > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/erik.m.bray%40gmail.com > From fromwheretowhere.service at gmail.com Mon Apr 16 05:11:12 2018 From: fromwheretowhere.service at gmail.com (Xuan Wu) Date: Mon, 16 Apr 2018 02:11:12 -0700 Subject: [Python-Dev] Translating sample programs in documentation In-Reply-To: References: Message-ID: <7a61be9e-1538-2854-ed29-1a944dc70fed@gmail.com> FYI I already post this in Doc-SIG mailing list, as it seems to be more relevant there. @Shell, thanks for the reference. Please correct me if I'm wrong, but I don't think the code standard for Python standard library applies to sample programs in the tutorials. Best, Xuan. On 4/15/18 7:49 PM, Shell Xu wrote: > Well, I'm not sure weather or not this is what you're looking for, but > pep-8 (https://www.python.org/dev/peps/pep-0008/) suggest like this: > > For Python 3.0 and beyond, the following policy is prescribed for the > standard library (see PEP 3131): All identifiers in the Python > standard library MUST use ASCII-only identifiers, and SHOULD use > English words wherever feasible (in many cases, abbreviations and > technical terms are used which aren't English). In addition, string > literals and comments must also be in ASCII. The only exceptions are > (a) test cases testing the non-ASCII features, and (b) names of > authors. Authors whose names are not based on the Latin alphabet > (latin-1, ISO/IEC 8859-1 character set) MUST provide a transliteration > of their names in this character set. > > So, I guess translate symbols to Chinese are not gonna help reader to > figure out what kind of code should they writing... > > On Mon, Apr 16, 2018 at 12:41 AM, Xuan Wu > > wrote: > > Excuse me if this was discussed before, but in French and Japanese > translations, all the sample programs seem to have identifiers in > English still. According to "PEP 545 -- Python Documentation > Translations", as I understand .po files are used for > translations. May I ask if there's technical restrictions causing > translations being only applied to the text parts? > > For example, here's the first sample program in 4.2: > > >>># Measure some strings: > ... words = ['cat', 'window', 'defenestrate'] > >>>for w in words: > ... print(w, len(w)) > ... > cat 3 > window 6 > defenestrate 12 > > Here's a possible translation in Chinese: > > >>> # ??????? > ... ?? = ['?', '??', '????'] > >>> for ? in ??: > ... print(?, len(?)) > ... > ? 1 > ?? 2 > ???? 4 > > As you may notice the strings differ in size if they are > translated directly. Obviously that does add extra burden to > review the new sample programs to assure effectiveness and > readability. > Any suggestion or comments are welcome. > > > Thanks, > Xuan. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/shell909090%40gmail.com > > > > > > -- > ????????????????????????????????? > blog: http://shell909090.org/ > twitter: @shell909090 > about.me : http://about.me/shell909090 -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Tue Apr 17 03:46:20 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 17 Apr 2018 17:46:20 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions Message-ID: Having survived four rounds in the boxing ring at python-ideas, PEP 572 is now ready to enter the arena of python-dev. I'll let the proposal speak for itself. Be aware that the reference implementation currently has a few test failures, which I'm still working on, but to my knowledge nothing will prevent the proposal itself from being successfully implemented. For those who have seen the most recent iteration on -ideas, the only actual change to the core proposal is that chaining is fully supported now. Formatted version: https://www.python.org/dev/peps/pep-0572/ ChrisA PEP: 572 Title: Assignment Expressions Author: Chris Angelico Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 28-Feb-2018 Python-Version: 3.8 Post-History: 28-Feb-2018, 02-Mar-2018, 23-Mar-2018, 04-Apr-2018, 17-Apr-2018 Abstract ======== This is a proposal for creating a way to assign to names within an expression. Additionally, the precise scope of comprehensions is adjusted, to maintain consistency and follow expectations. Rationale ========= Naming the result of an expression is an important part of programming, allowing a descriptive name to be used in place of a longer expression, and permitting reuse. Currently, this feature is available only in statement form, making it unavailable in list comprehensions and other expression contexts. Merely introducing a way to assign as an expression would create bizarre edge cases around comprehensions, though, and to avoid the worst of the confusions, we change the definition of comprehensions, causing some edge cases to be interpreted differently, but maintaining the existing behaviour in the majority of situations. Syntax and semantics ==================== In any context where arbitrary Python expressions can be used, a **named expression** can appear. This is of the form ``target := expr`` where ``expr`` is any valid Python expression, and ``target`` is any valid assignment target. The value of such a named expression is the same as the incorporated expression, with the additional side-effect that the target is assigned that value:: # Handle a matched regex if (match := pattern.search(data)) is not None: ... # A more explicit alternative to the 2-arg form of iter() invocation while (value := read_next_item()) is not None: ... # Share a subexpression between a comprehension filter clause and its output filtered_data = [y for x in data if (y := f(x)) is not None] Differences from regular assignment statements ---------------------------------------------- Most importantly, since ``:=`` is an expression, it can be used in contexts where statements are illegal, including lambda functions and comprehensions. An assignment statement can assign to multiple targets, left-to-right:: x = y = z = 0 The equivalent assignment expression is parsed as separate binary operators, and is therefore processed right-to-left, as if it were spelled thus:: assert 0 == (x := (y := (z := 0))) Augmented assignment is not supported in expression form:: >>> x +:= 1 File "", line 1 x +:= 1 ^ SyntaxError: invalid syntax Otherwise, the semantics of assignment are identical in statement and expression forms. Alterations to comprehensions ----------------------------- The current behaviour of list/set/dict comprehensions and generator expressions has some edge cases that would behave strangely if an assignment expression were to be used. Therefore the proposed semantics are changed, removing the current edge cases, and instead altering their behaviour *only* in a class scope. As of Python 3.7, the outermost iterable of any comprehension is evaluated in the surrounding context, and then passed as an argument to the implicit function that evaluates the comprehension. Under this proposal, the entire body of the comprehension is evaluated in its implicit function. Names not assigned to within the comprehension are located in the surrounding scopes, as with normal lookups. As one special case, a comprehension at class scope will **eagerly bind** any name which is already defined in the class scope. A list comprehension can be unrolled into an equivalent function. With Python 3.7 semantics:: numbers = [x + y for x in range(3) for y in range(4)] # Is approximately equivalent to def (iterator): result = [] for x in iterator: for y in range(4): result.append(x + y) return result numbers = (iter(range(3))) Under the new semantics, this would instead be equivalent to:: def (): result = [] for x in range(3): for y in range(4): result.append(x + y) return result numbers = () When a class scope is involved, a naive transformation into a function would prevent name lookups (as the function would behave like a method):: class X: names = ["Fred", "Barney", "Joe"] prefix = "> " prefixed_names = [prefix + name for name in names] With Python 3.7 semantics, this will evaluate the outermost iterable at class scope, which will succeed; but it will evaluate everything else in a function:: class X: names = ["Fred", "Barney", "Joe"] prefix = "> " def (iterator): result = [] for name in iterator: result.append(prefix + name) return result prefixed_names = (iter(names)) The name ``prefix`` is thus searched for at global scope, ignoring the class name. Under the proposed semantics, this name will be eagerly bound; and the same early binding then handles the outermost iterable as well. The list comprehension is thus approximately equivalent to:: class X: names = ["Fred", "Barney", "Joe"] prefix = "> " def (names=names, prefix=prefix): result = [] for name in names: result.append(prefix + name) return result prefixed_names = () With list comprehensions, this is unlikely to cause any confusion. With generator expressions, this has the potential to affect behaviour, as the eager binding means that the name could be rebound between the creation of the genexp and the first call to ``next()``. It is, however, more closely aligned to normal expectations. The effect is ONLY seen with names that are looked up from class scope; global names (eg ``range()``) will still be late-bound as usual. One consequence of this change is that certain bugs in genexps will not be detected until the first call to ``next()``, where today they would be caught upon creation of the generator. See 'open questions' below. Recommended use-cases ===================== Simplifying list comprehensions ------------------------------- These list comprehensions are all approximately equivalent:: stuff = [[y := f(x), x/y] for x in range(5)] # There are a number of less obvious ways to spell this in current # versions of Python. # External helper function def pair(x, value): return [value, x/value] stuff = [pair(x, f(x)) for x in range(5)] # Inline helper function stuff = [(lambda y: [y,x/y])(f(x)) for x in range(5)] # Extra 'for' loop - potentially could be optimized internally stuff = [[y, x/y] for x in range(5) for y in [f(x)]] # Iterating over a genexp stuff = [[y, x/y] for x, y in ((x, f(x)) for x in range(5))] # Expanding the comprehension into a loop stuff = [] for x in range(5): y = f(x) stuff.append([y, x/y]) # Wrapping the loop in a generator function def g(): for x in range(5): y = f(x) yield [y, x/y] stuff = list(g()) # Using a mutable cache object (various forms possible) c = {} stuff = [[c.update(y=f(x)) or c['y'], x/c['y']] for x in range(5)] If calling ``f(x)`` is expensive or has side effects, the clean operation of the list comprehension gets muddled. Using a short-duration name binding retains the simplicity; while the extra ``for`` loop does achieve this, it does so at the cost of dividing the expression visually, putting the named part at the end of the comprehension instead of the beginning. Similarly, a list comprehension can map and filter efficiently by capturing the condition:: results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0] Capturing condition values -------------------------- Assignment expressions can be used to good effect in the header of an ``if`` or ``while`` statement:: # Proposed syntax while (command := input("> ")) != "quit": print("You entered:", command) # Capturing regular expression match objects # See, for instance, Lib/pydoc.py, which uses a multiline spelling # of this effect if match := re.search(pat, text): print("Found:", match.group(0)) # Reading socket data until an empty string is returned while data := sock.read(): print("Received data:", data) # Equivalent in current Python, not caring about function return value while input("> ") != "quit": print("You entered a command.") # To capture the return value in current Python demands a four-line # loop header. while True: command = input("> "); if command == "quit": break print("You entered:", command) Particularly with the ``while`` loop, this can remove the need to have an infinite loop, an assignment, and a condition. It also creates a smooth parallel between a loop which simply uses a function call as its condition, and one which uses that as its condition but also uses the actual value. Rejected alternative proposals ============================== Proposals broadly similar to this one have come up frequently on python-ideas. Below are a number of alternative syntaxes, some of them specific to comprehensions, which have been rejected in favour of the one given above. Alternative spellings --------------------- Broadly the same semantics as the current proposal, but spelled differently. 1. ``EXPR as NAME``, with or without parentheses:: stuff = [[f(x) as y, x/y] for x in range(5)] Omitting the parentheses in this form of the proposal introduces many syntactic ambiguities. Requiring them in all contexts leaves open the option to make them optional in specific situations where the syntax is unambiguous (cf generator expressions as sole parameters in function calls), but there is no plausible way to make them optional everywhere. With the parentheses, this becomes a viable option, with its own tradeoffs in syntactic ambiguity. Since ``EXPR as NAME`` already has meaning in ``except`` and ``with`` statements (with different semantics), this would create unnecessary confusion or require special-casing (most notably of ``with`` and ``except`` statements, where a nearly-identical syntax has different semantics). 2. ``EXPR -> NAME``:: stuff = [[f(x) -> y, x/y] for x in range(5)] This syntax is inspired by languages such as R and Haskell, and some programmable calculators. (Note that a left-facing arrow ``y <- f(x)`` is not possible in Python, as it would be interpreted as less-than and unary minus.) This syntax has a slight advantage over 'as' in that it does not conflict with ``with`` and ``except`` statements, but otherwise is equivalent. 3. Adorning statement-local names with a leading dot:: stuff = [[(f(x) as .y), x/.y] for x in range(5)] # with "as" stuff = [[(.y := f(x)), x/.y] for x in range(5)] # with ":=" This has the advantage that leaked usage can be readily detected, removing some forms of syntactic ambiguity. However, this would be the only place in Python where a variable's scope is encoded into its name, making refactoring harder. This syntax is quite viable, and could be promoted to become the current recommendation if its advantages are found to outweigh its cost. 4. Adding a ``where:`` to any statement to create local name bindings:: value = x**2 + 2*x where: x = spam(1, 4, 7, q) Execution order is inverted (the indented body is performed first, followed by the "header"). This requires a new keyword, unless an existing keyword is repurposed (most likely ``with:``). See PEP 3150 for prior discussion on this subject (with the proposed keyword being ``given:``). 5. ``TARGET from EXPR``:: stuff = [[y from f(x), x/y] for x in range(5)] This syntax has fewer conflicts than ``as`` does (conflicting only with the ``raise Exc from Exc`` notation), but is otherwise comparable to it. Instead of paralleling ``with expr as target:`` (which can be useful but can also be confusing), this has no parallels, but is evocative. Special-casing conditional statements ------------------------------------- One of the most popular use-cases is ``if`` and ``while`` statements. Instead of a more general solution, this proposal enhances the syntax of these two statements to add a means of capturing the compared value:: if re.search(pat, text) as match: print("Found:", match.group(0)) This works beautifully if and ONLY if the desired condition is based on the truthiness of the captured value. It is thus effective for specific use-cases (regex matches, socket reads that return `''` when done), and completely useless in more complicated cases (eg where the condition is ``f(x) < 0`` and you want to capture the value of ``f(x)``). It also has no benefit to list comprehensions. Advantages: No syntactic ambiguities. Disadvantages: Answers only a fraction of possible use-cases, even in ``if``/``while`` statements. Special-casing comprehensions ----------------------------- Another common use-case is comprehensions (list/set/dict, and genexps). As above, proposals have been made for comprehension-specific solutions. 1. ``where``, ``let``, or ``given``:: stuff = [(y, x/y) where y = f(x) for x in range(5)] stuff = [(y, x/y) let y = f(x) for x in range(5)] stuff = [(y, x/y) given y = f(x) for x in range(5)] This brings the subexpression to a location in between the 'for' loop and the expression. It introduces an additional language keyword, which creates conflicts. Of the three, ``where`` reads the most cleanly, but also has the greatest potential for conflict (eg SQLAlchemy and numpy have ``where`` methods, as does ``tkinter.dnd.Icon`` in the standard library). 2. ``with NAME = EXPR``:: stuff = [(y, x/y) with y = f(x) for x in range(5)] As above, but reusing the `with` keyword. Doesn't read too badly, and needs no additional language keyword. Is restricted to comprehensions, though, and cannot as easily be transformed into "longhand" for-loop syntax. Has the C problem that an equals sign in an expression can now create a name binding, rather than performing a comparison. Would raise the question of why "with NAME = EXPR:" cannot be used as a statement on its own. 3. ``with EXPR as NAME``:: stuff = [(y, x/y) with f(x) as y for x in range(5)] As per option 2, but using ``as`` rather than an equals sign. Aligns syntactically with other uses of ``as`` for name binding, but a simple transformation to for-loop longhand would create drastically different semantics; the meaning of ``with`` inside a comprehension would be completely different from the meaning as a stand-alone statement, while retaining identical syntax. Regardless of the spelling chosen, this introduces a stark difference between comprehensions and the equivalent unrolled long-hand form of the loop. It is no longer possible to unwrap the loop into statement form without reworking any name bindings. The only keyword that can be repurposed to this task is ``with``, thus giving it sneakily different semantics in a comprehension than in a statement; alternatively, a new keyword is needed, with all the costs therein. Lowering operator precedence ---------------------------- There are two logical precedences for the ``:=`` operator. Either it should bind as loosely as possible, as does statement-assignment; or it should bind more tightly than comparison operators. Placing its precedence between the comparison and arithmetic operators (to be precise: just lower than bitwise OR) allows most uses inside ``while`` and ``if`` conditions to be spelled without parentheses, as it is most likely that you wish to capture the value of something, then perform a comparison on it:: pos = -1 while pos := buffer.find(search_term, pos + 1) >= 0: ... Once find() returns -1, the loop terminates. If ``:=`` binds as loosely as ``=`` does, this would capture the result of the comparison (generally either ``True`` or ``False``), which is less useful. While this behaviour would be convenient in many situations, it is also harder to explain than "the := operator behaves just like the assignment statement", and as such, the precedence for ``:=`` has been made as close as possible to that of ``=``. Migration path ============== The semantic changes to list/set/dict comprehensions, and more so to generator expressions, may potentially require migration of code. In many cases, the changes simply make legal what used to raise an exception, but there are some edge cases that were previously legal and now are not, and a few corner cases with altered semantics. Yield inside comprehensions --------------------------- As of Python 3.7, the outermost iterable in a comprehension is permitted to contain a 'yield' expression. If this is required, the iterable (or at least the yield) must be explicitly elevated from the comprehension:: # Python 3.7 def g(): return [x for x in [(yield 1)]] # With PEP 572 def g(): sent_item = (yield 1) return [x for x in [sent_item]] This more clearly shows that it is g(), not the comprehension, which is able to yield values (and is thus a generator function). The entire comprehension is consistently in a single scope. Name reuse inside comprehensions -------------------------------- If the same name is used in the outermost iterable and also as an iteration variable, this will now raise UnboundLocalError when previously it referred to the name in the surrounding scope. Example:: # Lib/typing.py tvars = [] for t in types: if isinstance(t, TypeVar) and t not in tvars: tvars.append(t) if isinstance(t, _GenericAlias) and not t._special: tvars.extend([ty for ty in t.__parameters__ if ty not in tvars]) If the list comprehension uses the name ``t`` rather than ``ty``, this will work in Python 3.7 but not with this proposal. As with other unwanted name shadowing, the solution is to use distinct names. Name lookups in class scope --------------------------- A comprehension inside a class previously was able to 'see' class members ONLY from the outermost iterable. Other name lookups would ignore the class and potentially locate a name at an outer scope:: pattern = "<%d>" class X: pattern = "[%d]" numbers = [pattern % n for n in range(5)] In Python 3.7, ``X.numbers`` would show angle brackets; with PEP 572, it would show square brackets. Maintaining the current behaviour here is best done by using distinct names for the different forms of ``pattern``, as would be the case with functions. Generator expression bugs can be caught later --------------------------------------------- Certain types of bugs in genexps were previously caught more quickly. Some are now detected only at first iteration:: gen = (x for x in rage(10)) # NameError gen = (x for x in 10) # TypeError (not iterable) gen = (x for x in range(1/0)) # Exception raised during evaluation This brings such generator expressions in line with a simple translation to function form:: def (): for x in rage(10): yield x gen = () # No exception yet tng = next(gen) # NameError Detecting these errors more quickly is nontrivial. It is, however, the exact same problem as generator functions currently suffer from, and this proposal brings the genexp in line with the most natural longhand form. Open questions ============== Can the outermost iterable still be evaluated early? ---------------------------------------------------- As of Python 3.7, the outermost iterable in a genexp is evaluated early, and the result passed to the implicit function as an argument. With PEP 572, this would no longer be the case. Can we still, somehow, evaluate it before moving on? One possible implementation would be:: gen = (x for x in rage(10)) # translates to def (): iterable = iter(rage(10)) yield None for x in iterable: yield x gen = () next(gen) This would pump the iterable up to just before the loop starts, evaluating exactly as much as is evaluated outside the generator function in Py3.7. This would result in it being possible to call ``gen.send()`` immediately, unlike with most generators, and may incur unnecessary overhead in the common case where the iterable is pumped immediately (perhaps as part of a larger expression). Importing names into comprehensions ----------------------------------- A list comprehension can use and update local names, and they will retain their values from one iteration to another. It would be convenient to use this feature to create rolling or self-effecting data streams:: progressive_sums = [total := total + value for value in data] This will fail with UnboundLocalError due to ``total`` not being initalized. Simply initializing it outside of the comprehension is insufficient - unless the comprehension is in class scope:: class X: total = 0 progressive_sums = [total := total + value for value in data] At other scopes, it may be beneficial to have a way to fetch a value from the surrounding scope. Should this be automatic? Should it be controlled with a keyword? Hypothetically (and using no new keywords), this could be written:: total = 0 progressive_sums = [total := total + value import nonlocal total for value in data] Translated into longhand, this would become:: total = 0 def (total=total): result = [] for value in data: result.append(total := total + value) return result progressive_sums = () ie utilizing the same early-binding technique that is used at class scope. Frequently Raised Objections ============================ Why not just turn existing assignment into an expression? --------------------------------------------------------- C and its derivatives define the ``=`` operator as an expression, rather than a statement as is Python's way. This allows assignments in more contexts, including contexts where comparisons are more common. The syntactic similarity between ``if (x == y)`` and ``if (x = y)`` belies their drastically different semantics. Thus this proposal uses ``:=`` to clarify the distinction. This could be used to create ugly code! --------------------------------------- So can anything else. This is a tool, and it is up to the programmer to use it where it makes sense, and not use it where superior constructs can be used. With assignment expressions, why bother with assignment statements? ------------------------------------------------------------------- The two forms have different flexibilities. The ``:=`` operator can be used inside a larger expression; the ``=`` statement can be augmented to ``+=`` and its friends. The assignment statement is a clear declaration of intent: this value is to be assigned to this target, and that's it. Why not use a sublocal scope and prevent namespace pollution? ------------------------------------------------------------- Previous revisions of this proposal involved sublocal scope (restricted to a single statement), preventing name leakage and namespace pollution. While a definite advantage in a number of situations, this increases complexity in many others, and the costs are not justified by the benefits. In the interests of language simplicity, the name bindings created here are exactly equivalent to any other name bindings, including that usage at class or module scope will create externally-visible names. This is no different from ``for`` loops or other constructs, and can be solved the same way: ``del`` the name once it is no longer needed, or prefix it with an underscore. Names bound within a comprehension are local to that comprehension, even in the outermost iterable, and can thus be used freely without polluting the surrounding namespace. Style guide recommendations =========================== As this adds another way to spell some of the same effects as can already be done, it is worth noting a few broad recommendations. These could be included in PEP 8 and/or other style guides. 1. If either assignment statements or assignment expressions can be used, prefer statements; they are a clear declaration of intent. 2. If using assignment expressions would lead to ambiguity about execution order, restructure it to use statements instead. 3. Chaining multiple assignment expressions should generally be avoided. More than one assignment per expression can detract from readability. Acknowledgements ================ The author wishes to thank Guido van Rossum and Nick Coghlan for their considerable contributions to this proposal, and to members of the core-mentorship mailing list for assistance with implementation. References ========== .. [1] Proof of concept / reference implementation (https://github.com/Rosuav/cpython/tree/assignment-expressions) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From chris.jerdonek at gmail.com Tue Apr 17 04:26:46 2018 From: chris.jerdonek at gmail.com (Chris Jerdonek) Date: Tue, 17 Apr 2018 01:26:46 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On Tue, Apr 17, 2018 at 12:46 AM, Chris Angelico wrote: > > Having survived four rounds in the boxing ring at python-ideas, PEP > 572 is now ready to enter the arena of python-dev. I'll let the > proposal speak for itself. Be aware that the reference implementation > currently has a few test failures, which I'm still working on, but to > my knowledge nothing will prevent the proposal itself from being > successfully implemented. Very interesting / exciting, thanks! > Augmented assignment is not supported in expression form:: > > >>> x +:= 1 > File "", line 1 > x +:= 1 > ^ > SyntaxError: invalid syntax Can you include in the PEP a brief rationale for not accepting this form? In particular, is the intent never to support it, or is the intent to expressly allow adding it at a later date (e.g. after getting experience with the simpler form, etc)? --Chris From rosuav at gmail.com Tue Apr 17 05:23:30 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 17 Apr 2018 19:23:30 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On Tue, Apr 17, 2018 at 6:26 PM, Chris Jerdonek wrote: > On Tue, Apr 17, 2018 at 12:46 AM, Chris Angelico wrote: >> >> Having survived four rounds in the boxing ring at python-ideas, PEP >> 572 is now ready to enter the arena of python-dev. I'll let the >> proposal speak for itself. Be aware that the reference implementation >> currently has a few test failures, which I'm still working on, but to >> my knowledge nothing will prevent the proposal itself from being >> successfully implemented. > > Very interesting / exciting, thanks! > >> Augmented assignment is not supported in expression form:: >> >> >>> x +:= 1 >> File "", line 1 >> x +:= 1 >> ^ >> SyntaxError: invalid syntax > > Can you include in the PEP a brief rationale for not accepting this > form? In particular, is the intent never to support it, or is the > intent to expressly allow adding it at a later date (e.g. after > getting experience with the simpler form, etc)? Sure. There are a few reasons; and Imaybe the best place to explain them is in the rejecteds section. Augmented assignment is currently all of these: augassign: ('+=' | '-=' | '*=' | '@=' | '/=' | '%=' | '&=' | '|=' | '^=' | '<<=' | '>>=' | '**=' | '//=') I'm actually not sure whether the augmented-assignment-expression operators should be "+:=" or ":+=", but either way, it'd be another thirteen tokens, some of which would be *four* character tokens. That's an awful lot of extra tokens. There's a lot less value in chaining on top of a "+=" than a simple assignment, and it doesn't seem right to have tons of code to support that. (Consider how much extra code there is to support async functions, and then imagine that you have a similarly large amount of effort for something that's of far lesser significance.) Another reason is that it can be confusing (to a human, not to the parser) that it would be the result of the augmented assignment, not the original expression, that would carry through. With regular assignment (whether it's to a simple name or to a subscript/attribute), removing the "target :=" part will leave you with the same value - the value of "x := 1" is 1. With augmented, the only logical way to do it would be to re-capture the left operand. Consider: x = 5 print(x +:= 2) Logically, this has to set x to 7, then print 7. But in more complicated situations, it can get confusing more easily than direct assignment does. Consider, for instance, augmented addition on a list, or similar. It could potentially be added later, but I'm not working hard to keep the option open or anything. Basically it's just "outside the scope of this document". ChrisA From ncoghlan at gmail.com Tue Apr 17 08:17:31 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 17 Apr 2018 22:17:31 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On 17 April 2018 at 17:46, Chris Angelico wrote: > Syntax and semantics > ==================== > > In any context where arbitrary Python expressions can be used, a **named > expression** can appear. This is of the form ``target := expr`` where > ``expr`` is any valid Python expression, and ``target`` is any valid > assignment target. The "assignment expressions should be restricted to names only" subthread from python-ideas finally crystallised for me (thanks in part to your own comment that 'With regular assignment (whether it's to a simple name or to a subscript/attribute), removing the "target :=" part will leave you with the same value - the value of "x := 1" is 1.'), and I now have a concrete argument for why I think we want to restrict the assignment targets to names only: all complex assignment targets create inherent ambiguity around the type of the expression result, and exactly which operations are performed as part of the assignment. Initially I thought the problem was specific to tuple unpacking syntax, but attempting to explain why subscript assignment and attribute assignments were OK made me realise that they're actually even worse off (since they can execute arbitrary code on both setting and retrieval, whereas tuple unpacking only iterates over iterables). Tackling those in order... Tuple unpacking: What's the result type for "a, b, c := range(3)"? Is it a range() object? Or is it a 3-tuple? If it's a 3-tuple, is that 3-tuple "(1, 2, 3)" or "(a, b, range(3))"? Once you have your answer, what about "a, b, c := iter(range(3))" or "a, b, *c := range(10)"? Whichever answers we chose would be surprising at least some of the time, so it seems simplest to disallow such ambiguous constructs, such that the only possible interpretation is as "(a, b, range(3))" Subscript assignment: What's the final value of "result" in "seq = list(); result = (seq[:] := range(3))"? Is it "range(3)"? Or is it "[1, 2, 3]"? As for tuple unpacking, does your preferred answer change for the case of "seq[:] := iter(range(3))"? More generally, if I write "container[k] := value", does only "type(container).__setitem__" get called, or does "type(container).__getitem__" get called as well? Again, this seems inherently ambiguous to me, and hence best avoided (at least for now), such that the result is always unambiguously "range(3)". Attribute assignment: If I write "obj.attr := value", does only "type(obj).__setattr__" get called, or does "type(obj).__getattribute__" get called as well? While I can't think of a simple obviously ambiguous example using builtins or the standard library, result ambiguity exists even for the attribute access case, since type or value coercion may occur either when setting the attribute, or when retrieving it, so it makes a difference as to whether a reference to the right hand side is passed through directly as the assignment expression result, or if the attribute is stored and then retrieved again. If all these constructs are prohibited, then a simple design principle serves to explain both their absence and the absence of the augmented assignment variants: "allowing the more complex forms of assignment as expressions makes the order of operations (as well as exactly which operations are executed) inherently ambiguous". That ambiguity generally doesn't exist with simple name bindings (I'm excluding execution namespaces with exotic binding behaviour from consideration here, as the consequences of trying to work with those are clearly on the folks defining and using them). > The value of such a named expression is the same as the incorporated > expression, with the additional side-effect that the target is assigned > that value:: > > # Handle a matched regex > if (match := pattern.search(data)) is not None: > ... > > # A more explicit alternative to the 2-arg form of iter() invocation > while (value := read_next_item()) is not None: > ... > > # Share a subexpression between a comprehension filter clause and its output > filtered_data = [y for x in data if (y := f(x)) is not None] [snip] > Style guide recommendations > =========================== > > As this adds another way to spell some of the same effects as can already be > done, it is worth noting a few broad recommendations. These could be included > in PEP 8 and/or other style guides. > > 1. If either assignment statements or assignment expressions can be > used, prefer statements; they are a clear declaration of intent. > > 2. If using assignment expressions would lead to ambiguity about > execution order, restructure it to use statements instead. > > 3. Chaining multiple assignment expressions should generally be avoided. > More than one assignment per expression can detract from readability. Given the many different uses for ":" identified on python-ideas, I'm inclined to suggest making these proposed style guidelines more prescriptive (at least initially) by either: 1. Listing out specific approved unambiguous use cases (i.e. if statement conditions, while loop conditions, list comprehensions, generation expressions) 2. Making the 3rd admonition more general by advising against using ":" for more than one purpose in the same expression (i.e. don't combine assignment expressions with slicing syntax, lambda expressions, function headers, variable annotations, dict or set displays, dict or set comprehensions) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From mertz at gnosis.cx Tue Apr 17 09:01:17 2018 From: mertz at gnosis.cx (David Mertz) Date: Tue, 17 Apr 2018 13:01:17 +0000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: Strongly agree with Nick that only simple name targets should be permitted (at least initially). NONE of the motivating cases use more complex targets, and allowing them encourages obscurity and code golf. On Tue, Apr 17, 2018, 8:20 AM Nick Coghlan wrote: > On 17 April 2018 at 17:46, Chris Angelico wrote: > > Syntax and semantics > > ==================== > > > > In any context where arbitrary Python expressions can be used, a **named > > expression** can appear. This is of the form ``target := expr`` where > > ``expr`` is any valid Python expression, and ``target`` is any valid > > assignment target. > > The "assignment expressions should be restricted to names only" > subthread from python-ideas finally crystallised for me (thanks in > part to your own comment that 'With regular assignment (whether it's > to a simple name or to a subscript/attribute), removing the "target > :=" part will leave you with the same value - the value of "x := 1" is > 1.'), and I now have a concrete argument for why I think we want to > restrict the assignment targets to names only: all complex assignment > targets create inherent ambiguity around the type of the expression > result, and exactly which operations are performed as part of the > assignment. > > Initially I thought the problem was specific to tuple unpacking > syntax, but attempting to explain why subscript assignment and > attribute assignments were OK made me realise that they're actually > even worse off (since they can execute arbitrary code on both setting > and retrieval, whereas tuple unpacking only iterates over iterables). > > Tackling those in order... > > Tuple unpacking: > > What's the result type for "a, b, c := range(3)"? Is it a range() > object? Or is it a 3-tuple? If it's a 3-tuple, is that 3-tuple "(1, 2, > 3)" or "(a, b, range(3))"? > Once you have your answer, what about "a, b, c := iter(range(3))" > or "a, b, *c := range(10)"? > > Whichever answers we chose would be surprising at least some of the > time, so it seems simplest to disallow such ambiguous constructs, such > that the only possible interpretation is as "(a, b, range(3))" > > Subscript assignment: > > What's the final value of "result" in "seq = list(); result = > (seq[:] := range(3))"? Is it "range(3)"? Or is it "[1, 2, 3]"? > As for tuple unpacking, does your preferred answer change for the > case of "seq[:] := iter(range(3))"? > > More generally, if I write "container[k] := value", does only > "type(container).__setitem__" get called, or does > "type(container).__getitem__" get called as well? > > Again, this seems inherently ambiguous to me, and hence best avoided > (at least for now), such that the result is always unambiguously > "range(3)". > > Attribute assignment: > > If I write "obj.attr := value", does only "type(obj).__setattr__" > get called, or does "type(obj).__getattribute__" get called as well? > > While I can't think of a simple obviously ambiguous example using > builtins or the standard library, result ambiguity exists even for the > attribute access case, since type or value coercion may occur either > when setting the attribute, or when retrieving it, so it makes a > difference as to whether a reference to the right hand side is passed > through directly as the assignment expression result, or if the > attribute is stored and then retrieved again. > > If all these constructs are prohibited, then a simple design principle > serves to explain both their absence and the absence of the augmented > assignment variants: "allowing the more complex forms of assignment as > expressions makes the order of operations (as well as exactly which > operations are executed) inherently ambiguous". > > That ambiguity generally doesn't exist with simple name bindings (I'm > excluding execution namespaces with exotic binding behaviour from > consideration here, as the consequences of trying to work with those > are clearly on the folks defining and using them). > > > The value of such a named expression is the same as the incorporated > > expression, with the additional side-effect that the target is assigned > > that value:: > > > > # Handle a matched regex > > if (match := pattern.search(data)) is not None: > > ... > > > > # A more explicit alternative to the 2-arg form of iter() invocation > > while (value := read_next_item()) is not None: > > ... > > > > # Share a subexpression between a comprehension filter clause and > its output > > filtered_data = [y for x in data if (y := f(x)) is not None] > > [snip] > > > Style guide recommendations > > =========================== > > > > As this adds another way to spell some of the same effects as can > already be > > done, it is worth noting a few broad recommendations. These could be > included > > in PEP 8 and/or other style guides. > > > > 1. If either assignment statements or assignment expressions can be > > used, prefer statements; they are a clear declaration of intent. > > > > 2. If using assignment expressions would lead to ambiguity about > > execution order, restructure it to use statements instead. > > > > 3. Chaining multiple assignment expressions should generally be avoided. > > More than one assignment per expression can detract from readability. > > Given the many different uses for ":" identified on python-ideas, I'm > inclined to suggest making these proposed style guidelines more > prescriptive (at least initially) by either: > > 1. Listing out specific approved unambiguous use cases (i.e. if > statement conditions, while loop conditions, list comprehensions, > generation expressions) > 2. Making the 3rd admonition more general by advising against using > ":" for more than one purpose in the same expression (i.e. don't > combine assignment expressions with slicing syntax, lambda > expressions, function headers, variable annotations, dict or set > displays, dict or set comprehensions) > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/mertz%40gnosis.cx > -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Tue Apr 17 09:07:28 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 17 Apr 2018 14:07:28 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On 17 April 2018 at 14:01, David Mertz wrote: > Strongly agree with Nick that only simple name targets should be permitted > (at least initially). NONE of the motivating cases use more complex targets, > and allowing them encourages obscurity and code golf. I also agree. Originally I would have said why not allow them, it's a potentially useful generalisation. But Nick's examples pretty clearly demonstrate that there are a lot of unclear edge cases involved, and even though "prevent people writing ugly code" is explicitly stated as a non-goal in the PEP, that doesn't mean it's OK to allow an obvious bug magnet with no clear use cases. Paul From p.f.moore at gmail.com Tue Apr 17 09:26:41 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 17 Apr 2018 14:26:41 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On 17 April 2018 at 14:07, Paul Moore wrote: > On 17 April 2018 at 14:01, David Mertz wrote: >> Strongly agree with Nick that only simple name targets should be permitted >> (at least initially). NONE of the motivating cases use more complex targets, >> and allowing them encourages obscurity and code golf. > > I also agree. Originally I would have said why not allow them, it's a > potentially useful generalisation. But Nick's examples pretty clearly > demonstrate that there are a lot of unclear edge cases involved, and > even though "prevent people writing ugly code" is explicitly stated as > a non-goal in the PEP, that doesn't mean it's OK to allow an obvious > bug magnet with no clear use cases. I should also point out that I remain -0 on this proposal (I'd already said this on python-ideas, but I should probably repeat it here). For me, the use cases are mostly marginal, and the major disadvantage is in having two forms of assignment. Explaining to a beginner why we use a := b in an expression, but a = b in a statement is going to be a challenge. The fact that the PEP needs a section covering all the style guide warnings we feel are needed seems like it's a warning bell, too. Paul From steve.dower at python.org Tue Apr 17 09:55:58 2018 From: steve.dower at python.org (Steve Dower) Date: Tue, 17 Apr 2018 06:55:58 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: Agree with Paul. The PEP is well thought out and well presented, but I really don?t think we need this in Python (and I say this as someone who uses it regularly in C/C#). -1 on the idea; no disrespect intended toward to people who did a lot of work on it. Top-posted from my Windows phone From: Paul Moore Sent: Tuesday, April 17, 2018 6:31 To: David Mertz Cc: Nick Coghlan; Python-Dev Subject: Re: [Python-Dev] PEP 572: Assignment Expressions On 17 April 2018 at 14:07, Paul Moore wrote: > On 17 April 2018 at 14:01, David Mertz wrote: >> Strongly agree with Nick that only simple name targets should be permitted >> (at least initially). NONE of the motivating cases use more complex targets, >> and allowing them encourages obscurity and code golf. > > I also agree. Originally I would have said why not allow them, it's a > potentially useful generalisation. But Nick's examples pretty clearly > demonstrate that there are a lot of unclear edge cases involved, and > even though "prevent people writing ugly code" is explicitly stated as > a non-goal in the PEP, that doesn't mean it's OK to allow an obvious > bug magnet with no clear use cases. I should also point out that I remain -0 on this proposal (I'd already said this on python-ideas, but I should probably repeat it here). For me, the use cases are mostly marginal, and the major disadvantage is in having two forms of assignment. Explaining to a beginner why we use a := b in an expression, but a = b in a statement is going to be a challenge. The fact that the PEP needs a section covering all the style guide warnings we feel are needed seems like it's a warning bell, too. Paul _______________________________________________ Python-Dev mailing list Python-Dev at python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Tue Apr 17 10:01:50 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 18 Apr 2018 00:01:50 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On Tue, Apr 17, 2018 at 10:17 PM, Nick Coghlan wrote: > Initially I thought the problem was specific to tuple unpacking > syntax, but attempting to explain why subscript assignment and > attribute assignments were OK made me realise that they're actually > even worse off (since they can execute arbitrary code on both setting > and retrieval, whereas tuple unpacking only iterates over iterables). > > Tackling those in order... > > Tuple unpacking: > > What's the result type for "a, b, c := range(3)"? Is it a range() > object? Or is it a 3-tuple? If it's a 3-tuple, is that 3-tuple "(1, 2, > 3)" or "(a, b, range(3))"? > Once you have your answer, what about "a, b, c := iter(range(3))" > or "a, b, *c := range(10)"? This is one that I didn't originally think about, and when I first tried it out a couple of weeks ago, I decided against mentioning it either way, because I've no idea what _should_ be done. But here's what my reference implementation does: >>> x = (a,b,c := range(3)) Traceback (most recent call last): File "", line 1, in NameError: name 'a' is not defined In other words, it's being parsed as: x = (a, b, (c := range(3))) Forcing the interpretation does work: >>> x = ((a,b,c) := range(3)) And then the value of the expression is the same object that just got assigned (or in this case, unpacked): >>> x range(0, 3) That's true even if it's an exhausted iterator: >>> x = ((a,b,c) := iter(range(3))) >>> x >>> next(x) Traceback (most recent call last): File "", line 1, in StopIteration The way it works is that the RHS gets evaluated, then that gets put on ice for a moment, and the assignment done with a copy of it. (In the CPython reference implementation, "put on ice" is done with DUP_TOP.) So the value that got assigned - even if that assignment involved unpacking a sequence - is still passed through the assignment and out the other side. >>> dis.dis("x = ((a,b,c) := range(3))") ### evaluate RHS 1 0 LOAD_NAME 0 (range) 2 LOAD_CONST 0 (3) 4 CALL_FUNCTION 1 ### grab another reference to the range 6 DUP_TOP ### do the assignment 8 UNPACK_SEQUENCE 3 10 STORE_NAME 1 (a) 12 STORE_NAME 2 (b) 14 STORE_NAME 3 (c) ### carry on with the rest of the expression 16 STORE_NAME 4 (x) 18 LOAD_CONST 1 (None) 20 RETURN_VALUE > Whichever answers we chose would be surprising at least some of the > time, so it seems simplest to disallow such ambiguous constructs, such > that the only possible interpretation is as "(a, b, range(3))" Yeah, that's what happens. Tuple display is defined as "test , test" and a 'test' can be 'target := value', so each element of a tuple can have an assignment expression in it. If you actually want to unpack inside an assignment expression, you need parentheses. > Subscript assignment: > > What's the final value of "result" in "seq = list(); result = > (seq[:] := range(3))"? Is it "range(3)"? Or is it "[1, 2, 3]"? > As for tuple unpacking, does your preferred answer change for the > case of "seq[:] := iter(range(3))"? It's range(3), and no, my preferred answer doesn't change. It'll probably never be useful to unpack an iterator in an assignment expression (since you'll always get an exhausted iterator at the end of it), but I'm sure there'll be uses for unpacking iterables. > More generally, if I write "container[k] := value", does only > "type(container).__setitem__" get called, or does > "type(container).__getitem__" get called as well? Only setitem. If you like, imagine the := operator as a tee junction: you "tap off" the pipe and snag it with assignment, and also keep using it just as if the assignment hadn't been there. > Again, this seems inherently ambiguous to me, and hence best avoided > (at least for now), such that the result is always unambiguously > "range(3)". > > Attribute assignment: > > If I write "obj.attr := value", does only "type(obj).__setattr__" > get called, or does "type(obj).__getattribute__" get called as well? I didn't change anything about how assignment actually works, so I would expect it to be exactly the same semantics as statement-assignment has. Let's test. >>> class X: ... @property ... def spam(self): return 42 ... @spam.setter ... def spam(self, val): print("Setting spam to", val) ... >>> x = X() >>> dis.dis("(x.spam := 7)") 1 0 LOAD_CONST 0 (7) 2 DUP_TOP 4 LOAD_NAME 0 (x) 6 STORE_ATTR 1 (spam) 8 RETURN_VALUE >>> (x.spam := 7) Setting spam to 7 7 Looks good to me. If I had to choose semantics, I don't think this would be a bad choice; and for something that derived naturally from a basic "let's just copy in the code for assignment", it's looking consistent and usable. > While I can't think of a simple obviously ambiguous example using > builtins or the standard library, result ambiguity exists even for the > attribute access case, since type or value coercion may occur either > when setting the attribute, or when retrieving it, so it makes a > difference as to whether a reference to the right hand side is passed > through directly as the assignment expression result, or if the > attribute is stored and then retrieved again. Agreed. > That ambiguity generally doesn't exist with simple name bindings (I'm > excluding execution namespaces with exotic binding behaviour from > consideration here, as the consequences of trying to work with those > are clearly on the folks defining and using them). The cool thing about the simple and naive code is that even those should work. I don't have an example ready for demo, but I fully expect that it would 'just work' the exact same way; the namespace would never be retrieved from, only set to. Hmm. I don't know what the consequences would be on class namespace with a non-vanilla dict. Probably functionally identical. But there might be some extremely weird cases if the namespace dict accepts setitem and then raises KeyError for that key. >> Style guide recommendations >> =========================== >> >> As this adds another way to spell some of the same effects as can already be >> done, it is worth noting a few broad recommendations. These could be included >> in PEP 8 and/or other style guides. >> >> 1. If either assignment statements or assignment expressions can be >> used, prefer statements; they are a clear declaration of intent. >> >> 2. If using assignment expressions would lead to ambiguity about >> execution order, restructure it to use statements instead. >> >> 3. Chaining multiple assignment expressions should generally be avoided. >> More than one assignment per expression can detract from readability. > > Given the many different uses for ":" identified on python-ideas, I'm > inclined to suggest making these proposed style guidelines more > prescriptive (at least initially) by either: > > 1. Listing out specific approved unambiguous use cases (i.e. if > statement conditions, while loop conditions, list comprehensions, > generation expressions) > 2. Making the 3rd admonition more general by advising against using > ":" for more than one purpose in the same expression (i.e. don't > combine assignment expressions with slicing syntax, lambda > expressions, function headers, variable annotations, dict or set > displays, dict or set comprehensions) I'm actually dubious about the third point as it stands. It's either too broad or too narrow, but I'm not sure which; there are plenty of legitimate uses for multiple colons in an expression without confusion, but there are also plenty of ways that even a single assignexp could be pretty bad for readability. So I'm hoping that we can get some people to test this out well before 3.8 lands, and refine the style recommendations before this feature hits release. ChrisA From ethan at stoneleaf.us Tue Apr 17 11:06:26 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 17 Apr 2018 08:06:26 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: <5AD60D72.8070602@stoneleaf.us> On 04/17/2018 07:01 AM, Chris Angelico wrote: > On Tue, Apr 17, 2018 at 10:17 PM, Nick Coghlan wrote: >> That ambiguity generally doesn't exist with simple name bindings (I'm >> excluding execution namespaces with exotic binding behaviour from >> consideration here, as the consequences of trying to work with those >> are clearly on the folks defining and using them). > > The cool thing about the simple and naive code is that even those > should work. I don't have an example ready for demo, but I fully > expect that it would 'just work' the exact same way; the namespace > would never be retrieved from, only set to. > > Hmm. I don't know what the consequences would be on class namespace > with a non-vanilla dict. Probably functionally identical. But there > might be some extremely weird cases if the namespace dict accepts > setitem and then raises KeyError for that key. If you want to play with a non-standard (okay, wierd) class namespaces, you can try using the assignment expression in an Enum class. -- ~Ethan~ From ethan at stoneleaf.us Tue Apr 17 11:08:34 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 17 Apr 2018 08:08:34 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: <5AD60DF2.4080307@stoneleaf.us> On 04/17/2018 12:46 AM, Chris Angelico wrote: > PEP: 572 > Title: Assignment Expressions > Author: Chris Angelico +1 Thanks for all the hard work, Chris! Hoping this PEP finally breaks your streak! ;) -- ~Ethan~ From ethan at stoneleaf.us Tue Apr 17 11:12:09 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 17 Apr 2018 08:12:09 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: <5AD60EC9.5010901@stoneleaf.us> On 04/17/2018 06:26 AM, Paul Moore wrote: > I should also point out that I remain -0 on this proposal (I'd already > said this on python-ideas, but I should probably repeat it here). For > me, the use cases are mostly marginal, and the major disadvantage is > in having two forms of assignment. Explaining to a beginner why we use > a := b in an expression, but a = b in a statement is going to be a > challenge. I don't see the challenge: They are different because '=' came first, but using '=' in an expression is a common source of bugs, so there we use ':=' instead. -- ~Ethan~ From mgainty at hotmail.com Mon Apr 16 18:00:49 2018 From: mgainty at hotmail.com (Martin Gainty) Date: Mon, 16 Apr 2018 22:00:49 +0000 Subject: [Python-Dev] Basic test to validate win10 install? Message-ID: Martin Gainty has shared a OneDrive file with you. To view it, click the link below. [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png] errors 3.lst i ran python test suite here and attaching output for your purview.. displaying output from stderr Python36>python -m test == CPython 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 17:00:18) [MSC v.1900 64 bit (AMD64)] == Windows-10-10.0.16299-SP0 little-endian == cwd: .\test_python_6296 == CPU count: 4 == encodings: locale=cp1252, FS=utf-8 Run tests sequentially 0:00:00 [ 1/406] test_grammar 0:00:00 [ 2/406] test_opcodes 0:00:00 [ 3/406] test_dict 0:00:00 [ 4/406] test_builtin 0:00:00 [ 5/406] test_exceptions 0:00:02 [ 6/406] test_types 0:00:02 [ 7/406] test_unittest 0:00:04 [ 8/406] test_doctest 0:05:41 [ 9/406] test_doctest2 -- test_doctest passed in 5 min 37 sec 0:05:41 [ 10/406] test_support 0:05:41 [ 11/406] test___all__ 0:05:57 [ 12/406] test___future__ 0:07:30 [ 13/406] test__locale -- test___future__ passed in 1 min 34 sec 0:07:30 [ 14/406] test__opcode 0:07:31 [ 15/406] test__osx_support 0:07:31 [ 16/406] test_abc 0:07:31 [ 17/406] test_abstract_numbers 0:07:31 [ 18/406] test_aifc 0:07:31 [ 19/406] test_argparse 0:07:33 [ 20/406] test_array 0:07:37 [ 21/406] test_asdl_parser test_asdl_parser skipped -- test irrelevant for an installed Python 0:07:37 [ 22/406] test_ast -- test_asdl_parser skipped 0:07:39 [ 23/406] test_asyncgen 0:08:01 [ 24/406] test_asynchat 0:08:03 [ 25/406] test_asyncio 0:08:53 [ 26/406] test_asyncore -- test_asyncio passed in 50 sec 0:11:38 [ 27/406] test_atexit -- test_asyncore passed in 2 min 45 sec 0:11:39 [ 28/406] test_audioop 0:11:39 [ 29/406] test_augassign 0:11:39 [ 30/406] test_base64 0:11:40 [ 31/406] test_baseexception 0:11:40 [ 32/406] test_bigaddrspace 0:11:40 [ 33/406] test_bigmem 0:11:40 [ 34/406] test_binascii 0:11:40 [ 35/406] test_binhex 0:11:40 [ 36/406] test_binop 0:11:40 [ 37/406] test_bisect 0:11:40 [ 38/406] test_bool 0:11:40 [ 39/406] test_buffer 0:11:47 [ 40/406] test_bufio 0:11:52 [ 41/406] test_bytes 0:11:54 [ 42/406] test_bz2 0:12:01 [ 43/406] test_calendar 0:12:05 [ 44/406] test_call 0:12:05 [ 45/406] test_capi 0:12:09 [ 46/406] test_cgi 0:12:09 [ 47/406] test_cgitb 0:12:10 [ 48/406] test_charmapcodec 0:12:10 [ 49/406] test_class 0:12:10 [ 50/406] test_cmath 0:12:10 [ 51/406] test_cmd 0:12:10 [ 52/406] test_cmd_line 0:12:14 [ 53/406] test_cmd_line_script 0:12:20 [ 54/406] test_code 0:12:20 [ 55/406] test_code_module 0:12:20 [ 56/406] test_codeccallbacks 0:12:20 [ 57/406] test_codecencodings_cn 0:12:21 [ 58/406] test_codecencodings_hk 0:12:21 [ 59/406] test_codecencodings_iso2022 0:12:21 [ 60/406] test_codecencodings_jp 0:12:22 [ 61/406] test_codecencodings_kr 0:12:22 [ 62/406] test_codecencodings_tw 0:12:22 [ 63/406] test_codecmaps_cn 0:12:24 [ 64/406] test_codecmaps_hk 0:12:25 [ 65/406] test_codecmaps_jp 0:12:28 [ 66/406] test_codecmaps_kr 0:12:30 [ 67/406] test_codecmaps_tw 0:12:31 [ 68/406] test_codecs 0:12:33 [ 69/406] test_codeop 0:12:33 [ 70/406] test_collections 0:12:34 [ 71/406] test_colorsys 0:12:34 [ 72/406] test_compare 0:12:34 [ 73/406] test_compile 0:12:34 [ 74/406] test_compileall 0:12:44 [ 75/406] test_complex 0:12:44 [ 76/406] test_concurrent_futures 0:14:00 [ 77/406] test_configparser -- test_concurrent_futures passed in 1 min 16 sec 0:14:01 [ 78/406] test_contains 0:14:01 [ 79/406] test_contextlib 0:14:01 [ 80/406] test_copy 0:14:02 [ 81/406] test_copyreg 0:14:02 [ 82/406] test_coroutines 0:14:02 [ 83/406] test_cprofile 0:14:03 [ 84/406] test_crashers 0:14:03 [ 85/406] test_crypt test_crypt skipped -- No module named '_crypt' 0:14:03 [ 86/406] test_csv -- test_crypt skipped 0:14:03 [ 87/406] test_ctypes 0:14:05 [ 88/406] test_curses test_curses skipped -- Use of the 'curses' resource not enabled 0:14:05 [ 89/406] test_datetime -- test_curses skipped (resource denied) 0:14:06 [ 90/406] test_dbm 0:14:06 [ 91/406] test_dbm_dumb 0:14:07 [ 92/406] test_dbm_gnu test_dbm_gnu skipped -- No module named '_gdbm' 0:14:07 [ 93/406] test_dbm_ndbm -- test_dbm_gnu skipped test_dbm_ndbm skipped -- No module named '_dbm' 0:14:07 [ 94/406] test_decimal -- test_dbm_ndbm skipped 0:14:14 [ 95/406] test_decorators 0:14:14 [ 96/406] test_defaultdict 0:14:14 [ 97/406] test_deque 0:14:17 [ 98/406] test_descr 0:14:19 [ 99/406] test_descrtut 0:14:19 [100/406] test_devpoll test_devpoll skipped -- test works only on Solaris OS family 0:14:19 [101/406] test_dict_version -- test_devpoll skipped 0:14:19 [102/406] test_dictcomps 0:14:19 [103/406] test_dictviews 0:14:20 [104/406] test_difflib 0:14:20 [105/406] test_dis 0:14:21 [106/406] test_distutils .\test_python_6296>exit 1 .\test_python_6296>exit 0 Warning -- files was modified by test_distutils Before: [] After: ['_configtest.c'] test test_distutils failed -- multiple errors occurred; run in verbose mode for details 0:14:26 [107/406/1] test_docxmlrpc -- test_distutils failed 0:14:53 [108/406/1] test_dtrace 0:14:54 [109/406/1] test_dummy_thread 0:14:54 [110/406/1] test_dummy_threading 0:14:54 [111/406/1] test_dynamic 0:14:54 [112/406/1] test_dynamicclassattribute 0:14:55 [113/406/1] test_eintr 0:14:55 [114/406/1] test_email 0:15:06 [115/406/1] test_ensurepip 0:15:07 [116/406/1] test_enum 0:15:07 [117/406/1] test_enumerate 0:15:08 [118/406/1] test_eof 0:15:08 [119/406/1] test_epoll test_epoll skipped -- test works only on Linux 2.6 0:15:08 [120/406/1] test_errno -- test_epoll skipped 0:15:08 [121/406/1] test_exception_hierarchy 0:15:08 [122/406/1] test_exception_variations 0:15:08 [123/406/1] test_extcall 0:15:09 [124/406/1] test_faulthandler 0:15:27 [125/406/1] test_fcntl test_fcntl skipped -- No module named 'fcntl' 0:15:27 [126/406/1] test_file -- test_fcntl skipped 0:15:27 [127/406/1] test_file_eintr 0:15:27 [128/406/1] test_filecmp 0:15:27 [129/406/1] test_fileinput 0:15:28 [130/406/1] test_fileio 0:15:28 [131/406/1] test_finalization 0:15:30 [132/406/1] test_float 0:15:30 [133/406/1] test_flufl 0:15:30 [134/406/1] test_fnmatch 0:15:31 [135/406/1] test_fork1 test_fork1 skipped -- object has no attribute 'fork' 0:15:31 [136/406/1] test_format -- test_fork1 skipped 0:15:31 [137/406/1] test_fractions 0:15:31 [138/406/1] test_frame 0:15:31 [139/406/1] test_fstring 0:15:32 [140/406/1] test_ftplib 0:15:40 [141/406/1] test_funcattrs 0:15:41 [142/406/1] test_functools 0:15:41 [143/406/1] test_future 0:15:41 [144/406/1] test_future3 0:15:41 [145/406/1] test_future4 0:15:41 [146/406/1] test_future5 0:15:42 [147/406/1] test_gc 0:15:47 [148/406/1] test_gdb test_gdb skipped -- Couldn't find gdb on the path 0:15:48 [149/406/1] test_generator_stop -- test_gdb skipped 0:15:48 [150/406/1] test_generators 0:15:49 [151/406/1] test_genericpath 0:15:49 [152/406/1] test_genexps 0:15:49 [153/406/1] test_getargs2 0:15:49 [154/406/1] test_getopt 0:15:49 [155/406/1] test_getpass 0:15:50 [156/406/1] test_gettext 0:15:50 [157/406/1] test_glob 0:15:51 [158/406/1] test_global 0:15:51 [159/406/1] test_grp test_grp skipped -- No module named 'grp' 0:15:51 [160/406/1] test_gzip -- test_grp skipped 0:15:52 [161/406/1] test_hash 0:15:54 [162/406/1] test_hashlib 0:15:58 [163/406/1] test_heapq 0:15:58 [164/406/1] test_hmac 0:15:59 [165/406/1] test_html 0:15:59 [166/406/1] test_htmlparser 0:15:59 [167/406/1] test_http_cookiejar 0:15:59 [168/406/1] test_http_cookies 0:15:59 [169/406/1] test_httplib 0:16:24 [170/406/1] test_httpservers 0:16:28 [171/406/1] test_idle 0:16:30 [172/406/1] test_imaplib 0:16:39 [173/406/1] test_imghdr 0:16:39 [174/406/1] test_imp 0:16:40 [175/406/1] test_import 0:16:41 [176/406/1] test_importlib 0:16:44 [177/406/1] test_index 0:16:45 [178/406/1] test_inspect 0:16:46 [179/406/1] test_int 0:16:47 [180/406/1] test_int_literal 0:16:47 [181/406/1] test_io 0:17:05 [182/406/1] test_ioctl test_ioctl skipped -- No module named 'fcntl' 0:17:05 [183/406/1] test_ipaddress -- test_ioctl skipped 0:17:05 [184/406/1] test_isinstance 0:17:06 [185/406/1] test_iter 0:17:06 [186/406/1] test_iterlen 0:17:06 [187/406/1] test_itertools 0:17:12 [188/406/1] test_json 0:17:15 [189/406/1] test_keyword 0:17:16 [190/406/1] test_keywordonlyarg 0:17:16 [191/406/1] test_kqueue test_kqueue skipped -- test works only on BSD 0:17:16 [192/406/1] test_largefile -- test_kqueue skipped 0:17:16 [193/406/1] test_lib2to3 0:17:32 [194/406/1] test_linecache 0:17:32 [195/406/1] test_list 0:17:33 [196/406/1] test_listcomps 0:17:33 [197/406/1] test_locale 0:17:34 [198/406/1] test_logging 0:17:59 [199/406/1] test_long 0:18:03 [200/406/1] test_longexp 0:18:03 [201/406/1] test_lzma 0:18:13 [202/406/1] test_macpath 0:18:13 [203/406/1] test_macurl2path 0:18:13 [204/406/1] test_mailbox 0:18:44 [205/406/1] test_mailcap -- test_mailbox passed in 31 sec 0:18:44 [206/406/1] test_marshal 0:18:46 [207/406/1] test_math 0:18:48 [208/406/1] test_memoryio 0:18:49 [209/406/1] test_memoryview 0:18:50 [210/406/1] test_metaclass 0:18:51 [211/406/1] test_mimetypes 0:18:51 [212/406/1] test_minidom 0:18:52 [213/406/1] test_mmap 0:18:52 [214/406/1] test_module 0:18:53 [215/406/1] test_modulefinder 0:18:54 [216/406/1] test_msilib 0:18:54 [217/406/1] test_multibytecodec 0:18:57 [218/406/1] test_multiprocessing_fork 0:18:57 [219/406/1] test_multiprocessing_forkserver 0:18:57 [220/406/1] test_multiprocessing_main_handling 0:19:06 [221/406/1] test_multiprocessing_spawn 0:20:52 [222/406/1] test_netrc -- test_multiprocessing_spawn passed in 1 min 45 sec 0:20:52 [223/406/1] test_nis test_nis skipped -- No module named 'nis' 0:20:52 [224/406/1] test_nntplib -- test_nis skipped 0:20:52 [225/406/1] test_normalization 0:20:55 [226/406/1] test_ntpath 0:20:55 [227/406/1] test_numeric_tower 0:20:56 [228/406/1] test_openpty test_openpty skipped -- os.openpty() not available. 0:20:56 [229/406/1] test_operator -- test_openpty skipped 0:20:56 [230/406/1] test_optparse 0:20:57 [231/406/1] test_ordered_dict 0:21:00 [232/406/1] test_os 0:21:04 [233/406/1] test_ossaudiodev test_ossaudiodev skipped -- Use of the 'audio' resource not enabled 0:21:04 [234/406/1] test_osx_env -- test_ossaudiodev skipped (resource denied) 0:21:04 [235/406/1] test_parser 0:21:04 [236/406/1] test_pathlib 0:21:06 [237/406/1] test_pdb 0:21:08 [238/406/1] test_peepholer 0:21:08 [239/406/1] test_pickle 0:21:14 [240/406/1] test_pickletools 0:21:16 [241/406/1] test_pipes test_pipes skipped -- pipes module only works on posix 0:21:16 [242/406/1] test_pkg -- test_pipes skipped 0:21:16 [243/406/1] test_pkgimport 0:21:16 [244/406/1] test_pkgutil 0:21:17 [245/406/1] test_platform 0:21:17 [246/406/1] test_plistlib 0:21:17 [247/406/1] test_poll test_poll skipped -- select.poll not defined 0:21:18 [248/406/1] test_popen -- test_poll skipped 0:21:18 [249/406/1] test_poplib 0:21:23 [250/406/1] test_posix test_posix skipped -- No module named 'posix' 0:21:24 [251/406/1] test_posixpath -- test_posix skipped 0:21:24 [252/406/1] test_pow 0:21:24 [253/406/1] test_pprint 0:21:25 [254/406/1] test_print 0:21:25 [255/406/1] test_profile 0:21:25 [256/406/1] test_property 0:21:25 [257/406/1] test_pstats 0:21:25 [258/406/1] test_pty test_pty skipped -- No module named 'termios' 0:21:26 [259/406/1] test_pulldom -- test_pty skipped 0:21:26 [260/406/1] test_pwd test_pwd skipped -- No module named 'pwd' 0:21:26 [261/406/1] test_py_compile -- test_pwd skipped 0:21:26 [262/406/1] test_pyclbr 0:21:29 [263/406/1] test_pydoc 0:21:36 [264/406/1] test_pyexpat 0:21:36 [265/406/1] test_queue 0:21:40 [266/406/1] test_quopri 0:21:40 [267/406/1] test_raise 0:21:40 [268/406/1] test_random 0:21:41 [269/406/1] test_range 0:21:42 [270/406/1] test_re 0:21:43 [271/406/1] test_readline test_readline skipped -- No module named 'readline' 0:21:44 [272/406/1] test_regrtest -- test_readline skipped test test_regrtest failed -- Traceback (most recent call last): File "C:\Python\Python36\lib\test\test_regrtest.py", line 601, in test_pcbuild_rt self.run_batch(script, *rt_args, *self.regrtest_args, *self.tests) File "C:\Python\Python36\lib\test\test_regrtest.py", line 576, in run_batch proc = self.run_command(args) File "C:\Python\Python36\lib\test\test_regrtest.py", line 474, in run_command **kw) File "C:\Python\Python36\lib\subprocess.py", line 403, in run with Popen(*popenargs, **kwargs) as process: File "C:\Python\Python36\lib\subprocess.py", line 709, in __init__ restore_signals, start_new_session) File "C:\Python\Python36\lib\subprocess.py", line 997, in _execute_child startupinfo) FileNotFoundError: [WinError 2] The system cannot find the file specified 0:21:56 [273/406/2] test_repl -- test_regrtest failed 0:21:56 [274/406/2] test_reprlib 0:21:57 [275/406/2] test_resource test_resource skipped -- No module named 'resource' 0:21:57 [276/406/2] test_richcmp -- test_resource skipped 0:21:57 [277/406/2] test_rlcompleter 0:21:57 [278/406/2] test_robotparser 0:21:58 [279/406/2] test_runpy 0:22:03 [280/406/2] test_sax 0:22:03 [281/406/2] test_sched 0:22:04 [282/406/2] test_scope 0:22:04 [283/406/2] test_script_helper 0:22:05 [284/406/2] test_secrets 0:22:05 [285/406/2] test_select 0:22:05 [286/406/2] test_selectors 0:22:07 [287/406/2] test_set 0:22:09 [288/406/2] test_setcomps 0:22:10 [289/406/2] test_shelve 0:22:11 [290/406/2] test_shlex 0:22:11 [291/406/2] test_shutil 0:22:12 [292/406/2] test_signal 0:22:13 [293/406/2] test_site 0:22:15 [294/406/2] test_slice 0:22:15 [295/406/2] test_smtpd 0:22:15 [296/406/2] test_smtplib 0:22:16 [297/406/2] test_smtpnet test_smtpnet skipped -- Use of the 'network' resource not enabled 0:22:16 [298/406/2] test_sndhdr -- test_smtpnet skipped (resource denied) 0:22:17 [299/406/2] test_socket 0:22:42 [300/406/2] test_socketserver test_socketserver skipped -- Use of the 'network' resource not enabled 0:22:43 [301/406/2] test_sort -- test_socketserver skipped (resource denied) 0:22:43 [302/406/2] test_source_encoding 0:22:44 [303/406/2] test_spwd test_spwd skipped -- No module named 'spwd' 0:22:44 [304/406/2] test_sqlite -- test_spwd skipped 0:22:47 [305/406/2] test_ssl 0:22:58 [306/406/2] test_startfile 0:22:59 [307/406/2] test_stat 0:22:59 [308/406/2] test_statistics 0:23:00 [309/406/2] test_strftime 0:23:00 [310/406/2] test_string 0:23:00 [311/406/2] test_string_literals 0:23:01 [312/406/2] test_stringprep 0:23:01 [313/406/2] test_strptime 0:23:02 [314/406/2] test_strtod 0:23:03 [315/406/2] test_struct 0:23:03 [316/406/2] test_structmembers 0:23:03 [317/406/2] test_structseq 0:23:04 [318/406/2] test_subclassinit 0:23:04 [319/406/2] test_subprocess a DOS box should flash briefly ... 0:23:25 [320/406/2] test_sunau 0:23:25 [321/406/2] test_sundry 0:23:26 [322/406/2] test_super 0:23:26 [323/406/2] test_symbol 0:23:26 [324/406/2] test_symtable 0:23:26 [325/406/2] test_syntax 0:23:27 [326/406/2] test_sys 0:23:29 [327/406/2] test_sys_setprofile 0:23:29 [328/406/2] test_sys_settrace 0:23:30 [329/406/2] test_sysconfig 0:23:30 [330/406/2] test_syslog test_syslog skipped -- No module named 'syslog' 0:23:30 [331/406/2] test_tarfile -- test_syslog skipped 0:23:43 [332/406/2] test_tcl 0:23:49 [333/406/2] test_telnetlib 0:23:49 [334/406/2] test_tempfile 0:23:51 [335/406/2] test_textwrap 0:23:52 [336/406/2] test_thread 0:23:52 [337/406/2] test_threaded_import 0:23:54 [338/406/2] test_threadedtempfile 0:23:54 [339/406/2] test_threading 0:24:05 [340/406/2] test_threading_local 0:24:06 [341/406/2] test_threadsignals test_threadsignals skipped -- Can't test signal on win32 0:24:06 [342/406/2] test_time -- test_threadsignals skipped 0:24:08 [343/406/2] test_timeit 0:24:09 [344/406/2] test_timeout test_timeout skipped -- Use of the 'network' resource not enabled 0:24:09 [345/406/2] test_tix -- test_timeout skipped (resource denied) test_tix skipped -- Use of the 'gui' resource not enabled 0:24:09 [346/406/2] test_tk -- test_tix skipped (resource denied) test_tk skipped -- Use of the 'gui' resource not enabled 0:24:09 [347/406/2] test_tokenize -- test_tk skipped (resource denied) 0:24:11 [348/406/2] test_tools At least one file or directory argument required. Use --help to show usage. test test_tools failed -- Traceback (most recent call last): File "C:\Python\Python36\lib\test\test_tools\test_sundry.py", line 36, in test_sundry import_tool(name) File "C:\Python\Python36\lib\test\test_tools\__init__.py", line 21, in import_tool return importlib.import_module(toolname) File "C:\Python\Python36\lib\importlib\__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 994, in _gcd_import File "", line 971, in _find_and_load File "", line 955, in _find_and_load_unlocked File "", line 665, in _load_unlocked File "", line 678, in exec_module File "", line 219, in _call_with_frames_removed File "C:\Python\Python36\Tools\scripts\2to3.py", line 5, in sys.exit(main("lib2to3.fixes")) SystemExit: 2 0:24:16 [349/406/3] test_trace -- test_tools failed 0:24:21 [350/406/3] test_traceback 0:24:23 [351/406/3] test_tracemalloc 0:24:24 [352/406/3] test_ttk_guionly test_ttk_guionly skipped -- Use of the 'gui' resource not enabled 0:24:25 [353/406/3] test_ttk_textonly -- test_ttk_guionly skipped (resource denied) 0:24:25 [354/406/3] test_tuple 0:24:33 [355/406/3] test_turtle 0:24:33 [356/406/3] test_typechecks 0:24:34 [357/406/3] test_typing 0:24:34 [358/406/3] test_ucn 0:24:35 [359/406/3] test_unary 0:24:35 [360/406/3] test_unicode 0:24:36 [361/406/3] test_unicode_file 0:24:37 [362/406/3] test_unicode_file_functions 0:24:38 [363/406/3] test_unicode_identifiers 0:24:38 [364/406/3] test_unicodedata 0:24:41 [365/406/3] test_univnewlines 0:24:41 [366/406/3] test_unpack 0:24:41 [367/406/3] test_unpack_ex 0:24:42 [368/406/3] test_urllib 0:24:45 [369/406/3] test_urllib2 0:24:48 [370/406/3] test_urllib2_localnet 0:25:54 [371/406/3] test_urllib2net -- test_urllib2_localnet passed in 1 min 5 sec test_urllib2net skipped -- Use of the 'network' resource not enabled 0:25:54 [372/406/3] test_urllib_response -- test_urllib2net skipped (resource denied) 0:25:54 [373/406/3] test_urllibnet test_urllibnet skipped -- Use of the 'network' resource not enabled 0:25:54 [374/406/3] test_urlparse -- test_urllibnet skipped (resource denied) 0:25:55 [375/406/3] test_userdict 0:25:55 [376/406/3] test_userlist 0:25:55 [377/406/3] test_userstring 0:25:57 [378/406/3] test_utf8source 0:25:57 [379/406/3] test_uu 0:25:57 [380/406/3] test_uuid 0:25:58 [381/406/3] test_venv 0:26:33 [382/406/3] test_wait3 -- test_venv passed in 35 sec test_wait3 skipped -- os.fork not defined 0:26:33 [383/406/3] test_wait4 -- test_wait3 skipped test_wait4 skipped -- object has no attribute 'fork' 0:26:34 [384/406/3] test_warnings -- test_wait4 skipped 0:26:36 [385/406/3] test_wave 0:26:37 [386/406/3] test_weakref 0:26:52 [387/406/3] test_weakset 0:26:52 [388/406/3] test_webbrowser 0:26:53 [389/406/3] test_winconsoleio ?^Z abc123 ?????? A?B ??AA? ????? ?????? ?????? ?????? ?????? ?????? ?????? ?????? ?????? ?????? ?????? ?????? ?????? ?????? ?????? ?????? ???? ???? ???? ???? ???? ???? ???? ???? ???? ???? ???? ???? ???? ???? ???? 0:26:53 [390/406/3] test_winreg 0:27:00 [391/406/3] test_winsound test_winsound skipped -- Use of the 'audio' resource not enabled 0:27:00 [392/406/3] test_with -- test_winsound skipped (resource denied) 0:27:00 [393/406/3] test_wsgiref 0:27:01 [394/406/3] test_xdrlib 0:27:01 [395/406/3] test_xml_dom_minicompat 0:27:01 [396/406/3] test_xml_etree 0:27:02 [397/406/3] test_xml_etree_c 0:27:04 [398/406/3] test_xmlrpc 0:27:15 [399/406/3] test_xmlrpc_net test_xmlrpc_net skipped -- Use of the 'network' resource not enabled 0:27:15 [400/406/3] test_yield_from -- test_xmlrpc_net skipped (resource denied) 0:27:16 [401/406/3] test_zipapp 0:27:16 [402/406/3] test_zipfile 0:27:40 [403/406/3] test_zipfile64 test_zipfile64 skipped -- test requires loads of disk-space bytes and a long time to run 0:27:40 [404/406/3] test_zipimport -- test_zipfile64 skipped (resource denied) 0:27:41 [405/406/3] test_zipimport_support 0:27:43 [406/406/3] test_zlib 364 tests OK. 3 tests failed: test_distutils test_regrtest test_tools 39 tests skipped: test_asdl_parser test_crypt test_curses test_dbm_gnu test_dbm_ndbm test_devpoll test_epoll test_fcntl test_fork1 test_gdb test_grp test_ioctl test_kqueue test_nis test_openpty test_ossaudiodev test_pipes test_poll test_posix test_pty test_pwd test_readline test_resource test_smtpnet test_socketserver test_spwd test_syslog test_threadsignals test_timeout test_tix test_tk test_ttk_guionly test_urllib2net test_urllibnet test_wait3 test_wait4 test_winsound test_xmlrpc_net test_zipfile64 Total duration: 27 min 45 sec Tests result: FAILURE I was told the python -m test are for devs only to test "auxiliary functions" do you have a "basic test" i can use to validate Python36 has installed properly on Win10? Thanks Martin ______________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From zachary.ware+pydev at gmail.com Tue Apr 17 11:51:48 2018 From: zachary.ware+pydev at gmail.com (Zachary Ware) Date: Tue, 17 Apr 2018 10:51:48 -0500 Subject: [Python-Dev] Basic test to validate win10 install? In-Reply-To: References: Message-ID: Hi Martin, On Mon, Apr 16, 2018 at 5:00 PM, Martin Gainty wrote: > I was told the python -m test are for devs only to test "auxiliary > functions" > do you have a "basic test" i can use to validate Python36 has installed > properly on Win10? The fact that the tests run at all show that everything installed properly. The three failures that you see don't look serious, and are probably only failing due to differences in the development and installation environments on Windows. One of these days I hope to have a bulidbot that automatically installs and tests Python on every commit on Windows, but we're not there yet. Until then, investigating these failures and contributing patches to fix them could be a great way to get involved in CPython development! Regards, -- Zach From ericfahlgren at gmail.com Tue Apr 17 12:15:55 2018 From: ericfahlgren at gmail.com (Eric Fahlgren) Date: Tue, 17 Apr 2018 09:15:55 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On Tue, Apr 17, 2018 at 2:23 AM, Chris Angelico wrote: > Augmented assignment is currently all of these: > > augassign: ('+=' | '-=' | '*=' | '@=' | '/=' | '%=' | '&=' | '|=' | '^=' | > '<<=' | '>>=' | '**=' | '//=') > > I'm actually not sure whether the augmented-assignment-expression > operators should be "+:=" or ":+=", but either way, it'd be another > thirteen tokens, some of which would be *four* character tokens. > ?Or simply rework the augmented assignment's semantics to become expression operators without any syntactic changes. Since there's no bug magnet arising in the usual context where '=' and '==' get confused: > if x += 1 < 2: -------------- next part -------------- An HTML attachment was scrubbed... URL: From v+python at g.nevcal.com Tue Apr 17 12:23:14 2018 From: v+python at g.nevcal.com (Glenn Linderman) Date: Tue, 17 Apr 2018 09:23:14 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On 4/17/2018 12:46 AM, Chris Angelico wrote: > gen = (x for x in rage(10)) # NameError > gen = (x for x in 10) # TypeError (not iterable) > gen = (x for x in range(1/0)) # Exception raised during evaluation > > This brings such generator expressions in line with a simple translation to > function form:: > > def (): > for x in rage(10): > yield x > gen = () # No exception yet > tng = next(gen) # NameError > > Detecting these errors more quickly is nontrivial. It is, however, the exact > same problem as generator functions currently suffer from, and this proposal > brings the genexp in line with the most natural longhand form. > > > Open questions > ============== > > Can the outermost iterable still be evaluated early? > ---------------------------------------------------- > > As of Python 3.7, the outermost iterable in a genexp is evaluated early, and > the result passed to the implicit function as an argument. With PEP 572, this > would no longer be the case. Can we still, somehow, evaluate it before moving > on? One possible implementation would be:: > > gen = (x for x in rage(10)) > # translates to > def (): > iterable = iter(rage(10)) > yield None > for x in iterable: > yield x > gen = () > next(gen) > I think "rage" is supposed to be "range" in four places in the quoted block. -------------- next part -------------- An HTML attachment was scrubbed... URL: From yselivanov.ml at gmail.com Tue Apr 17 13:03:39 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Tue, 17 Apr 2018 13:03:39 -0400 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: Hi Chris, Thank you for working on this PEP! Inline assignments is a long requested feature and this seems to be the first serious attempt at adding it. That said I'm very -1 on the idea. 1. I switch between Python / JavaScript / C frequently (although I code in Python 70% of my time.) C and JS have inline assignments but I don't find myself using them often. JavaScript has inline assignments and they are useful to get the match object after applying a regex. You use the same example in your PEP. But in my experience, this is the only common pattern in JavaScript. I don't see people using inline assignments for anything else, at least it's not a common pattern. C is low-level and has no exceptions. It uses function return values to signal if there was an error or not. It's a popular pattern to call a function from an 'if' statement like this: "if ((int ret = func()))" to save a line of code. If we ignore this particular pattern, we see that inline assignment isn't used that often. In your PEP you use comprehensions and regex match object to show how inline assignment can simplify the code. In my experience, comprehensions that are a little more complex than "(f(x) for x in something)" are always better being rewritten to an expanded form. I don't find "stuff = [[y := f(x), x/y] for x in range(5)]" very readable, and yes, I think that the simple expanded version of this comprehension is better. Using inline assignments in "while" statements is neat, but how often do we use "while" statements? 2. We all try to follow the Python zen when we are designing new language features. With the exception of string formatting Python follows the "There should be one-- and preferably only one -- obvious way to do it." line. Your PEP, in my personal opinion, goes agains this one, and also a few other lines. I simply don't see a very compelling use case to have two forms of assignment in Python. It does complicate the grammar by adding a new operator, it invites people to write more complex code, and it has only a couple good use cases. Yury From p.f.moore at gmail.com Tue Apr 17 13:14:00 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 17 Apr 2018 18:14:00 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <5AD60EC9.5010901@stoneleaf.us> References: <5AD60EC9.5010901@stoneleaf.us> Message-ID: On 17 April 2018 at 16:12, Ethan Furman wrote: > On 04/17/2018 06:26 AM, Paul Moore wrote: > >> I should also point out that I remain -0 on this proposal (I'd already >> said this on python-ideas, but I should probably repeat it here). For >> me, the use cases are mostly marginal, and the major disadvantage is >> in having two forms of assignment. Explaining to a beginner why we use >> a := b in an expression, but a = b in a statement is going to be a >> challenge. > > > I don't see the challenge: They are different because '=' came first, but > using '=' in an expression is a common source of bugs, so there we use ':=' > instead. I fully expect the question "so if I want to assign to a variable, why shouldn't I just use x := 12 (as a statement)?" I don't have a good answer for that. And if I can't come up with a convincing reply to that, the next question will likely be "so why does = exist at all?" If we want to change Python so that assignments are expressions, and assignment statements only remain for backward compatibility, then fine - we should propose that (I'd probably be against it, BTW, but I'd reserve judgement until I saw a proper proposal) but this PEP would need a lot of changes if it were to go down that route. This half-way house of having both seems like it will just confuse people. Paul From larry at hastings.org Tue Apr 17 13:14:10 2018 From: larry at hastings.org (Larry Hastings) Date: Tue, 17 Apr 2018 10:14:10 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <40QRdF48vPzFr4F@mail.python.org> References: <40QRdF48vPzFr4F@mail.python.org> Message-ID: On 04/17/2018 06:55 AM, Steve Dower wrote: > > Agree with Paul. The PEP is well thought out and well presented, but I > really don?t think we need this in Python (and I say this as someone > who uses it regularly in C/C#). > > -1 on the idea; no disrespect intended toward to people who did a lot > of work on it. > Well put!? I feel the same, and yes that includes being -1 on the proposal. Best wishes, //arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From fromwheretowhere.service at gmail.com Tue Apr 17 14:32:25 2018 From: fromwheretowhere.service at gmail.com (Xuan Wu) Date: Tue, 17 Apr 2018 11:32:25 -0700 Subject: [Python-Dev] Translating sample programs in documentation In-Reply-To: References: Message-ID: Hi Erik, thanks a lot for the support. As mentioned in the thread in Doc-Sig list, IMO it can be great showcase to use identifiers in native languages, which PEP 3131 was designed for initially, while even now many beginners don't even know it's possible. Besides, thanks to the references from that thread, it seems some change in Sphinx config is needed before the programs can be translated. Help or pointers are greatly appreciated, as I'm brand new to the documentation system. I've setup translation system locally and got a testing Chinese version up and running, but haven't got far on Sphinx config. Last but not least, please let me know if it is OK to continue this topic here, especially the technical details. Best wishes, Xuan. On 4/16/18 5:35 AM, Erik Bray wrote: > On Mon, Apr 16, 2018 at 4:49 AM, Shell Xu wrote: >> Well, I'm not sure weather or not this is what you're looking for, but pep-8 >> (https://www.python.org/dev/peps/pep-0008/) suggest like this: >> >> For Python 3.0 and beyond, the following policy is prescribed for the >> standard library (see PEP 3131): All identifiers in the Python standard >> library MUST use ASCII-only identifiers, and SHOULD use English words >> wherever feasible (in many cases, abbreviations and technical terms are used >> which aren't English). In addition, string literals and comments must also >> be in ASCII. The only exceptions are (a) test cases testing the non-ASCII >> features, and (b) names of authors. Authors whose names are not based on the >> Latin alphabet (latin-1, ISO/IEC 8859-1 character set) MUST provide a >> transliteration of their names in this character set. >> >> So, I guess translate symbols to Chinese are not gonna help reader to figure >> out what kind of code should they writing... > > That only applies to the Python stdlib itself. It's a feature that > Python allows unicode identifiers, and there's nothing about that > against PEP-8 for sure. > > I think it's a great idea; I'm not sure how it works out technically > in terms of providing .po files for .rst documentation or if there's > some better mechanism for that... > > Best, > E > > >> On Mon, Apr 16, 2018 at 12:41 AM, Xuan Wu >> wrote: >>> Excuse me if this was discussed before, but in French and Japanese >>> translations, all the sample programs seem to have identifiers in English >>> still. According to "PEP 545 -- Python Documentation Translations", as I >>> understand .po files are used for translations. May I ask if there's >>> technical restrictions causing translations being only applied to the text >>> parts? >>> >>> For example, here's the first sample program in 4.2: >>> >>>>>> # Measure some strings: >>> ... words = ['cat', 'window', 'defenestrate'] >>>>>> for w in words: >>> ... print(w, len(w)) >>> ... >>> cat 3 >>> window 6 >>> defenestrate 12 >>> >>> Here's a possible translation in Chinese: >>> >>>>>> # ??????? >>> ... ?? = ['?', '??', '????'] >>>>>> for ? in ??: >>> ... print(?, len(?)) >>> ... >>> ? 1 >>> ?? 2 >>> ???? 4 >>> >>> As you may notice the strings differ in size if they are translated >>> directly. Obviously that does add extra burden to review the new sample >>> programs to assure effectiveness and readability. >>> Any suggestion or comments are welcome. >>> >>> >>> Thanks, >>> Xuan. >>> >>> _______________________________________________ >>> Python-Dev mailing list >>> Python-Dev at python.org >>> https://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: >>> https://mail.python.org/mailman/options/python-dev/shell909090%40gmail.com >>> >> >> >> -- >> ????????????????????????????????? >> blog: http://shell909090.org/ >> twitter: @shell909090 >> about.me: http://about.me/shell909090 >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/erik.m.bray%40gmail.com >> From tim.peters at gmail.com Tue Apr 17 15:28:06 2018 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 17 Apr 2018 14:28:06 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: I'll channel that Guido would be happiest if this rule were followed: Given an assignment statement using "=", the meaning is the same if "=" is replaced with ":=". In particular, the expression at the far right is evaluated once, and - in case of chained assignments - is applied in turn to each target left-to-right. Otherwise the semantics of "=" and ":=" can be very different indeed. So, then, e.g., and assuming the rule above always applies: [Nick] > Tuple unpacking: > > What's the result type for "a, b, c := range(3)"? Is it a range() > object? Or is it a 3-tuple? If it's a 3-tuple, is that 3-tuple "(1, 2, > 3)" or "(a, b, range(3))"? It's the range object range(3). Same as in: x = a, b, c = range(3) `x` is bound to the range object range(3). > Once you have your answer, what about "a, b, c := iter(range(3))" A range_iterator object, same as what `x` is bound to in: x = a, b, c = iter(range(3)) However, `list(x)` then returns an empty list, because iter(range(3)) was evaluated only once, and the iterator was run to exhaustion when unpacking it for the `a, b, c` target. > or "a, b, *c := range(10)"? The range object range(10). > Whichever answers we chose would be surprising at least some of the > time, so it seems simplest to disallow such ambiguous constructs, such > that the only possible interpretation is as "(a, b, range(3))" That's why Guido would be happiest with the rule at the top. "The answers" can already be surprising at times with current assignment statements, but they are well defined. It would be mondo bonkers to make up entirely different subtle answers ;-) > Subscript assignment: > > What's the final value of "result" in "seq = list(); result = > (seq[:] := range(3))"? Is it "range(3)"? Or is it "[1, 2, 3]"? As above, it's range(3). > As for tuple unpacking, does your preferred answer change for the > case of "seq[:] := iter(range(3))"? As above, a range_iterator object, but one that's already been run to exhaustion. > More generally, if I write "container[k] := value", does only > "type(container).__setitem__" get called, or does > "type(container).__getitem__" get called as well? The rule at the top implies __setitem_ is called once, and __getitem__ not at all. The value of the assignment is the object `value` was bound to at the start, regardless of how tricky __setitem__ may be. And in k := container[k] := value `k` is bound to `value` before `container[k]` is evaluated. Why? Because that's how assignment _statements_ have always worked. > Attribute assignment: > > If I write "obj.attr := value", does only "type(obj).__setattr__" > get called, or does "type(obj).__getattribute__" get called as well? As above, only __setattr__. > While I can't think of a simple obviously ambiguous example using > builtins or the standard library, result ambiguity exists even for the > attribute access case, since type or value coercion may occur either > when setting the attribute, or when retrieving it, so it makes a > difference as to whether a reference to the right hand side is passed > through directly as the assignment expression result, or if the > attribute is stored and then retrieved again. This is already defined for assignment statements. While the PEP doesn't say "and the same for assignment expressions", my guess is that it won't be accepted unless it does. Or, indeed, the target is limited to a name. But Guido wasn't keen on that. In short, I think the PEP's chance of acceptance increases the _more_ assignment expressions act like assignment statements, not the less, and is highest if they act exactly the same (except for returning a value; e.g., while >>> a = 3 at a shell displays nothing, >>> a := 3 should display 3). From mgainty at hotmail.com Tue Apr 17 14:49:19 2018 From: mgainty at hotmail.com (Martin Gainty) Date: Tue, 17 Apr 2018 18:49:19 +0000 Subject: [Python-Dev] Basic test to validate win10 install? In-Reply-To: References: , Message-ID: Please feel free to reply to python-dev at python.org rather than me personally to take this back to the list. On Tue, Apr 17, 2018 at 1:20 PM, Martin Gainty wrote: > I'll need a few specifics before i wandering into "dll hell" > > .NetFramework version? > RT Library version? ....in the old days we used to call this msvcrt.dll > > msvc or mingw compiler? > if msvc which version msvc? > if mingw which version msvc ? Have a look at https://devguide.python.org/ to get started Python Developer?s Guide ? Python Developer's Guide devguide.python.org Contributing?. We encourage everyone to contribute to Python and that?s why we have put up this developer?s guide. If you still have questions after reviewing the material in this guide, then the Core Python Mentorship group is available to help guide new contributors through the process. contributing to CPython, and in particular https://devguide.python.org/setup/#windows for getting set up on Windows. To answer your questions in brief, though: we use the latest MSVC compiler available with Visual Studio 2017, which links to the "universal CRT" and avoids most of the old headaches of incompatibilities between the runtimes used by different compiler versions. You may also be interested in the core-mentorship mailing list, details about which can be found at https://mail.python.org/mm3/mailman3/lists/core-mentorship.python.org/ Regards, -- Zach -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Tue Apr 17 15:31:31 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 17 Apr 2018 15:31:31 -0400 Subject: [Python-Dev] Basic test to validate win10 install? In-Reply-To: References: Message-ID: On 4/16/2018 6:00 PM, Martin Gainty wrote: > I was told the python -m test are for devs only to test "auxiliary > functions" People say all sorts of nonsense. I don't know what 'auxiliary function' would mean in this context. The Windows installer optionally installs the tests so that users can run them. > do you have a "basic test" i can use to validate Python36 has?installed > properly on Win10? If clicking the Python 3.6 Python 3.6 Start Menu entry or entering 'python' or in particular 'py -3.6' at a command prompt brings up the 3.6 interactive interpreter, the installation is 'proper' unless a particular file has an error. For a recent StackOverflow question, a strange error in lib.idlelib.rpc was fixed by re-installing (whereas the IDLE editor worked), so I suspect a disk error. I wonder if the installer could do a better job of verifying the download and then the installation of each file. -- Terry Jan Reedy From rosuav at gmail.com Tue Apr 17 15:44:56 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 18 Apr 2018 05:44:56 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On Wed, Apr 18, 2018 at 5:28 AM, Tim Peters wrote: > I'll channel that Guido would be happiest if this rule were followed: > > Given an assignment statement using "=", the meaning is the same if > "=" is replaced with ":=". That's broadly the intention. At the moment, there are two exceptions: 1) Augmented assignment isn't a thing 2) Chained assignment isn't a thing, which means that the assignments operate right-to-left 2a) Assignment > In particular, the expression at the far right is evaluated once, and > - in case of chained assignments - is applied in turn to each target > left-to-right. I'll toy with this and see if I can implement it sanely. If so, that'll eliminate one more distinction. > Otherwise the semantics of "=" and ":=" can be very different indeed. TBH, the common cases won't actually be much affected. You give this example: k := container[k] := value but that's not going to be more common. What I'm more likely to see is something like this: k, container[k] = new_key(), new_value() which can instead be written: container[k := new_key()] = new_value() and is, IMO, clearer that way. > So, then, e.g., and assuming the rule above always applies: > > [Nick] >> Tuple unpacking: >> >> What's the result type for "a, b, c := range(3)"? Is it a range() >> object? Or is it a 3-tuple? If it's a 3-tuple, is that 3-tuple "(1, 2, >> 3)" or "(a, b, range(3))"? > > It's the range object range(3). Same as in: > > x = a, b, c = range(3) > > `x` is bound to the range object range(3). At the moment, "x = a, b, c := range(3)" will set c to range(3), then build a tuple of that with the existing values of a and b. You can, however, parenthesize the (a, b, c) part, and then it'll behave as you say. >> Whichever answers we chose would be surprising at least some of the >> time, so it seems simplest to disallow such ambiguous constructs, such >> that the only possible interpretation is as "(a, b, range(3))" > > That's why Guido would be happiest with the rule at the top. "The > answers" can already be surprising at times with current assignment > statements, but they are well defined. It would be mondo bonkers to > make up entirely different subtle answers ;-) Wholeheartedly agreed. ChrisA From olegs at traiana.com Tue Apr 17 05:46:58 2018 From: olegs at traiana.com (Oleg Sivokon) Date: Tue, 17 Apr 2018 09:46:58 +0000 Subject: [Python-Dev] https://bugs.python.org/issue33127 breaks pip / easy_install / pipenv etc in corporate networks on MS Windows using self-signed certificates Message-ID: It is common practice in corporate networks that connect MS Windows machines to redirect all (encrypted included) traffic through company's router. For this purpose routers are usually configured to act as a CA. However, the certificate issued by such "CA" will of course not be found in the certificates distributed with LibreSSL (how would they even know?). MS Windows networking, however, has a way to configure these policies. Prior to this issue, Python relied on the OS libraries to implement TLS protocol, so the overall setup worked transparently for users. Since 3.6.5, however, this is no longer possible (requires alteration of certificates distributed with Python). I'm asking that this be made configurable / possible to disable using simple means, perhaps an environment variable / registry key or similar. PS. I still cannot register to the bug tracker (never received a confirmation email), this is why you are reading this email. - Best. Oleg This communication and all information contained in or attached to it is confidential, intended solely for the addressee, may be legally privileged and is the intellectual property of one of the companies of NEX Group plc ("NEX") or third parties. If you are not the intended addressee or receive this message in error, please immediately delete all copies of it and notify the sender. We have taken precautions to minimise the risk of transmitting software viruses, but we advise you to carry out your own virus checks on any attachments. We do not accept liability for any loss or damage caused by software viruses. NEX reserves the right to monitor all communications. We do not accept any legal responsibility for the content of communications, and no communication shall be considered legally binding. Furthermore, if the content of this communication is personal or unconnected with our business, we accept no liability or responsibility for it. NEX Group plc is a public limited company registered in England and Wales under number 10013770 and certain of its affiliates are authorised and regulated by regulatory authorities. For further regulatory information please see www.NEX.com. From steve.dower at python.org Tue Apr 17 16:58:54 2018 From: steve.dower at python.org (Steve Dower) Date: Tue, 17 Apr 2018 13:58:54 -0700 Subject: [Python-Dev] https://bugs.python.org/issue33127 breaks pip / easy_install / pipenv etc in corporate networks on MS Windows using self-signed certificates In-Reply-To: References: Message-ID: <15c254a5-23c6-9d13-00a7-4602bef4dc93@python.org> On 17Apr2018 0246, Oleg Sivokon wrote: > It is common practice in corporate networks that connect MS Windows machines to redirect all (encrypted included) traffic through company's router. For this purpose routers are usually configured to act as a CA. However, the certificate issued by such "CA" will of course not be found in the certificates distributed with LibreSSL (how would they even know?). MS Windows networking, however, has a way to configure these policies. > > Prior to this issue, Python relied on the OS libraries to implement TLS protocol, so the overall setup worked transparently for users. Since 3.6.5, however, this is no longer possible (requires alteration of certificates distributed with Python). If you are referring to Python on Windows, this was never true. We've always relied on OpenSSL and at best will read locally installed certificates (and by default, most certificates are not locally installed). This should not have changed recently, and certainly not with the bug you reference. > I'm asking that this be made configurable / possible to disable using simple means, perhaps an environment variable / registry key or similar. I'm not clear on what you're asking for. The only thing we can disable is reading OS certificates into OpenSSL, and that would be the opposite of what you are having trouble with. Perhaps this is an issue with pip more specifically than Python? > PS. I still cannot register to the bug tracker (never received a confirmation email), this is why you are reading this email. I would guess it ended up in a junk mail folder, though that may be controlled by your organization rather than anywhere you can get to it. Perhaps using an alternate email address will be easiest? Cheers, Steve From ericsnowcurrently at gmail.com Tue Apr 17 17:51:49 2018 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Tue, 17 Apr 2018 15:51:49 -0600 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <40QRfq2w5LzFqsw@mail.python.org> References: <40QRfq2w5LzFqsw@mail.python.org> Message-ID: On Tue, Apr 17, 2018 at 7:55 AM, Steve Dower wrote: > Agree with Paul. The PEP is well thought out and well presented, but I > really don?t think we need this in Python (and I say this as someone who > uses it regularly in C/C#). > > -1 on the idea; no disrespect intended toward to people who did a lot of > work on it. Same here. I'm more interested in having anonymous blocks (i.e. scopes), along the lines of PEP 3150, though it's currently #2 on my wish list. :) -eric From tjreedy at udel.edu Tue Apr 17 17:53:27 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 17 Apr 2018 17:53:27 -0400 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On 4/17/2018 3:46 AM, Chris Angelico wrote: > Abstract > ======== > > This is a proposal for creating a way to assign to names within an expression. I started at -something as this is nice but not necessary. I migrated to +something for the specific, limited proposal you wrote above: expressions of the form "name := expression". > Additionally, the precise scope of comprehensions is adjusted, to maintain > consistency and follow expectations. We fiddled with comprehension scopes, and broke some code, in 3.0. I oppose doing so again. People expect their 3.x code to continue working in future versions. Breaking that expectation should require deprecation for at least 2 versions. > Rationale > ========= > > Naming the result of an expression is an important part of programming, > allowing a descriptive name to be used in place of a longer expression, > and permitting reuse. Right. In other words, "name := expression". > Merely introducing a way to assign as an expression > would create bizarre edge cases around comprehensions, though, and to avoid > the worst of the confusions, we change the definition of comprehensions, > causing some edge cases to be interpreted differently, but maintaining the > existing behaviour in the majority of situations. If it is really true that introducing 'name := expression' requires such a side-effect, then I might oppose it. > > Syntax and semantics > ==================== > > In any context where arbitrary Python expressions can be used, a **named > expression** can appear. This is of the form ``target := expr`` where > ``expr`` is any valid Python expression, and ``target`` is any valid > assignment target. This generalization is different from what you said in the abstract and rationale. No rationale is given. After reading Nick's examination of the generalization, and your response, -1. > The value of such a named expression is the same as the incorporated > expression, with the additional side-effect that the target is assigned > that value:: As someone else noted, you only use names as targets, thus providing no rationale for anything else. > # Handle a matched regex > if (match := pattern.search(data)) is not None: > ... > > # A more explicit alternative to the 2-arg form of iter() invocation > while (value := read_next_item()) is not None: > ... To me, being able to name and test expressions fits with Python names not being typed. To me, usage such as the above is the justification for the limited proposal. > # Share a subexpression between a comprehension filter clause and its output > filtered_data = [y for x in data if (y := f(x)) is not None] And this is secondary. > Differences from regular assignment statements > ---------------------------------------------- > > Most importantly, since ``:=`` is an expression, it can be used in contexts > where statements are illegal, including lambda functions and comprehensions. > > An assignment statement can assign to multiple targets, left-to-right:: > > x = y = z = 0 This is a bad example as there is very seldom a reason to assign multiple names, as opposed to multiple targets. Here is a typical real example. self.x = x = expression # Use local x in the rest of the method. In "x = y = 0", x and y likely represent two *different* concepts (variables) that happen to be initialized with the same value. One could instead write "x,y = 0,0". > The equivalent assignment expression should be a syntax error. > is parsed as separate binary operators, ':=' is not a binary operator, any more than '=' is, as names, and targets in general, are not objects. Neither fetch and operate on the current value, if any, of the name or target. Therefore neither has an 'associativity'. > and is therefore processed right-to-left, as if it were spelled thus:: > > assert 0 == (x := (y := (z := 0))) Parentheses should be required, to maintain the syntax "name := expression". > Augmented assignment is not supported in expression form:: > > >>> x +:= 1 > File "", line 1 > x +:= 1 > ^ > SyntaxError: invalid syntax I would have expected :+=, but agree with the omission. > Otherwise, the semantics of assignment are identical in statement and > expression forms. Mostly replacing '=' with ':=' is a different proposal and a different goal than naming expressions within an expression for reuse (primarily) within the expression (including compound expressions). Proposing an un-augmentable, un-chainable, name_only := expression expression would not be duplicating assignment statements. > Alterations to comprehensions > ----------------------------- > > The current behaviour of list/set/dict comprehensions and generator > expressions has some edge cases that would behave strangely if an assignment > expression were to be used. You have not shown this. Your examples do not involve assignment expressions, and adding them should make no difference. Changing the scoping of comprehensions should be a separate PEP. Therefore the proposed semantics are changed, > removing the current edge cases, and instead altering their behaviour *only* > in a class scope. > > As of Python 3.7, the outermost iterable of any comprehension is evaluated > in the surrounding context, and then passed as an argument to the implicit > function that evaluates the comprehension. > > Under this proposal, the entire body of the comprehension is evaluated in > its implicit function. Names not assigned to within the comprehension are > located in the surrounding scopes, as with normal lookups. As one special > case, a comprehension at class scope will **eagerly bind** any name which > is already defined in the class scope. > > A list comprehension can be unrolled into an equivalent function. With > Python 3.7 semantics:: > > numbers = [x + y for x in range(3) for y in range(4)] > # Is approximately equivalent to > def (iterator): > result = [] > for x in iterator: > for y in range(4): > result.append(x + y) > return result > numbers = (iter(range(3))) > > Under the new semantics, this would instead be equivalent to:: > > def (): > result = [] > for x in range(3): > for y in range(4): > result.append(x + y) > return result > numbers = () Why make the change? > When a class scope is involved, a naive transformation into a function would > prevent name lookups (as the function would behave like a method):: > > class X: > names = ["Fred", "Barney", "Joe"] > prefix = "> " > prefixed_names = [prefix + name for name in names] > > With Python 3.7 semantics, I believe in all of 3.x .. > this will evaluate the outermost iterable at class > scope, which will succeed; but it will evaluate everything else in a function:: > > class X: > names = ["Fred", "Barney", "Joe"] > prefix = "> " > def (iterator): > result = [] > for name in iterator: > result.append(prefix + name) > return result > prefixed_names = (iter(names)) > > The name ``prefix`` is thus searched for at global scope, ignoring the class > name. And today it fails. This has nothing to do with adding name assignment expressions. > Under the proposed semantics, this name will be eagerly bound; and the > same early binding then handles the outermost iterable as well. The list > comprehension is thus approximately equivalent to:: > > class X: > names = ["Fred", "Barney", "Joe"] > prefix = "> " > def (names=names, prefix=prefix): > result = [] > for name in names: > result.append(prefix + name) > return result > prefixed_names = () > > With list comprehensions, this is unlikely to cause any confusion. With > generator expressions, this has the potential to affect behaviour, as the > eager binding means that the name could be rebound between the creation of > the genexp and the first call to ``next()``. It is, however, more closely > aligned to normal expectations. The effect is ONLY seen with names that > are looked up from class scope; global names (eg ``range()``) will still > be late-bound as usual. > > One consequence of this change is that certain bugs in genexps will not > be detected until the first call to ``next()``, where today they would be > caught upon creation of the generator. See 'open questions' below. > > > Recommended use-cases > ===================== > > Simplifying list comprehensions > ------------------------------- I consider this secondary and would put it second. > Capturing condition values > -------------------------- I would put this first, as you did above. > Assignment expressions can be used to good effect in the header of > an ``if`` or ``while`` statement:: > > # Proposed syntax > while (command := input("> ")) != "quit": > print("You entered:", command) > > # Capturing regular expression match objects > # See, for instance, Lib/pydoc.py, which uses a multiline spelling > # of this effect > if match := re.search(pat, text): > print("Found:", match.group(0)) > > # Reading socket data until an empty string is returned > while data := sock.read(): > print("Received data:", data) > > # Equivalent in current Python, not caring about function return value > while input("> ") != "quit": > print("You entered a command.") > > # To capture the return value in current Python demands a four-line > # loop header. > while True: > command = input("> "); > if command == "quit": > break > print("You entered:", command) This idiom is not obvious to beginners and is awkward at best, so I consider eliminating this the biggest gain. Beginners commonly write little games and entry loops and get tripped up trying to do so. > Particularly with the ``while`` loop, this can remove the need to have an > infinite loop, an assignment, and a condition. It also creates a smooth > parallel between a loop which simply uses a function call as its condition, > and one which uses that as its condition but also uses the actual value. ... Bottom line: I suggest rewriting again, as indicated, changing title to 'Name Assignment Expressions'. -- Terry Jan Reedy From tjreedy at udel.edu Tue Apr 17 18:05:41 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 17 Apr 2018 18:05:41 -0400 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On 4/17/2018 1:03 PM, Yury Selivanov wrote: > Using inline assignments in "while" statements is neat, but how > often do we use "while" statements? Beginners commonly write game and entry loops and commonly stumble over the need to write an infinite loop and a half. The evidence is on both python-list and StackOverflow. Reducing 4 lines to 1 here reduces complexity and, I think, increases clarity. > 2. We all try to follow the Python zen when we are designing new > language features. With the exception of string formatting Python > follows the "There should be one-- and preferably only one -- > obvious way to do it." line. There is currently no way, not one way, to bind a name to an expression for use within the same statement. I recommended to Chris that he remove from the PEP everything but this. No other targets. No chaining. (And no redefining class-scope comprehensions.) This would remove all but the unavoidable overlap and reduce the added complexity of what people could write. > I simply don't see a very compelling use case to have two forms > of assignment in Python. We already have more than two ;-). -- Terry Jan Reedy From tim.peters at gmail.com Tue Apr 17 18:23:13 2018 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 17 Apr 2018 17:23:13 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: [Tim] >> I'll channel that Guido would be happiest if this rule were followed: >> >> Given an assignment statement using "=", the meaning is the same if >> "=" is replaced with ":=". [Chris] > That's broadly the intention. At the moment, there are two exceptions: > > 1) Augmented assignment isn't a thing Doesn't have to be :-) "Augmented assignment statement" is already a different thing than "assignment statement" (for example, in an augmented assignment statement, there is no chaining, and the sole target can' t be, e.g., a slice or any form of unpacking syntax). > 2) Chained assignment isn't a thing, which means that the assignments > operate right-to-left >> In particular, the expression at the far right is evaluated once, and >> - in case of chained assignments - is applied in turn to each target >> left-to-right. > I'll toy with this and see if I can implement it sanely. If so, > that'll eliminate one more distinction. >> Otherwise the semantics of "=" and ":=" can be very different indeed. > TBH, the common cases won't actually be much affected. Or at all! That's not the point here, though: if making assignment expressions work as exactly like assignment statements as possible is what's needed for the PEP to pass, it's the _annoying_ cases that have to be looked at. Personally, after considerable staring at my own code, I would be perfectly happy to settle for assignment expressions no fancier than identifier ":=" expression That alone covers over 99% of the cases I'd be tempted to use the new feature at all, and then gobs of general-case assignment-statement difficulties go away, including the "right-to-left or left-to-right?" distinction (there's no way to tell which order bindings happen in `x := y := z := 3` short of staring at the generated code). But so far I haven't gotten the impression that Guido is fond of that. He should be, though ;-) > You give this example: > > k := container[k] := value > > but that's not going to be more common. What I'm more likely to see is > something like this: Not about what's common, but about the full range of what's possible to express. ... [Nick] >>> Tuple unpacking: >>> >>> What's the result type for "a, b, c := range(3)"? Is it a range() >>> object? Or is it a 3-tuple? If it's a 3-tuple, is that 3-tuple "(1, 2, >>> 3)" or "(a, b, range(3))"? >> It's the range object range(3). Same as in: >> >> x = a, b, c = range(3) >> >> `x` is bound to the range object range(3). > At the moment, "x = a, b, c := range(3)" will set c to range(3), then > build a tuple of that with the existing values of a and b. You can, > however, parenthesize the (a, b, c) part, and then it'll behave as you > say. Which would be really annoying to "repair". >>> Whichever answers we chose would be surprising at least some of the >>> time, so it seems simplest to disallow such ambiguous constructs, such >>> that the only possible interpretation is as "(a, b, range(3))" >> That's why Guido would be happiest with the rule at the top. "The >> answers" can already be surprising at times with current assignment >> statements, but they are well defined. It would be mondo bonkers to >> make up entirely different subtle answers ;-) > Wholeheartedly agreed. I'd like Guido to chime in again, because I'm pretty sure he won't accept what's currently on the table. There are two plausible ways to repair that: 1. Continue down the road of making assignment expressions "exactly like" assignment statements in their full generality. 2. Back off and limit assignment expressions to what appears to be the overwhelmingly most common case motivated by looking at real code (as opposed to constructing examples to illustrate pitfalls & obscurities): identifier ":=" expression From rob.cliffe at btinternet.com Tue Apr 17 19:34:30 2018 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Wed, 18 Apr 2018 00:34:30 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: <0668be4c-abdf-9913-f76e-a9af6de75152@btinternet.com> On 17/04/2018 15:01, Chris Angelico wrote: > On Tue, Apr 17, 2018 at 10:17 PM, Nick Coghlan wrote: > >>> Style guide recommendations >>> =========================== >>> >>> As this adds another way to spell some of the same effects as can already be >>> done, it is worth noting a few broad recommendations. These could be included >>> in PEP 8 and/or other style guides. >>> >>> 1. If either assignment statements or assignment expressions can be >>> used, prefer statements; they are a clear declaration of intent. >>> >>> 2. If using assignment expressions would lead to ambiguity about >>> execution order, restructure it to use statements instead. >>> >>> 3. Chaining multiple assignment expressions should generally be avoided. >>> More than one assignment per expression can detract from readability. >> Given the many different uses for ":" identified on python-ideas, I'm >> inclined to suggest making these proposed style guidelines more >> prescriptive (at least initially) by either: >> >> 1. Listing out specific approved unambiguous use cases (i.e. if >> statement conditions, while loop conditions, list comprehensions, >> generation expressions) >> 2. Making the 3rd admonition more general by advising against using >> ":" for more than one purpose in the same expression (i.e. don't >> combine assignment expressions with slicing syntax, lambda >> expressions, function headers, variable annotations, dict or set >> displays, dict or set comprehensions) > I'm actually dubious about the third point as it stands. I'm more than dubious - I disagree with Nick on this point.? It is already possible to have multiple uses of ":" in an expression; surely we wouldn't advise that such existing code should be changed, in cases where it is arises naturally and is genuinely useful. Best wishes Rob Cliffe From python at mrabarnett.plus.com Tue Apr 17 19:51:46 2018 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 18 Apr 2018 00:51:46 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On 2018-04-17 22:53, Terry Reedy wrote: > On 4/17/2018 3:46 AM, Chris Angelico wrote: [snip] > >> Augmented assignment is not supported in expression form:: >> >> >>> x +:= 1 >> File "", line 1 >> x +:= 1 >> ^ >> SyntaxError: invalid syntax > > I would have expected :+=, but agree with the omission. > x = x op 1 is abbreviated to x op= 1, so x := x op 1 would be abbreviated to x op:= 1. That's what's used in the Icon language. [snip] From rosuav at gmail.com Tue Apr 17 20:13:58 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 18 Apr 2018 10:13:58 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On Wed, Apr 18, 2018 at 7:53 AM, Terry Reedy wrote: > On 4/17/2018 3:46 AM, Chris Angelico wrote: > >> Abstract >> ======== >> >> This is a proposal for creating a way to assign to names within an >> expression. > > > I started at -something as this is nice but not necessary. I migrated to > +something for the specific, limited proposal you wrote above: expressions > of the form "name := expression". > >> Additionally, the precise scope of comprehensions is adjusted, to maintain >> consistency and follow expectations. > > > We fiddled with comprehension scopes, and broke some code, in 3.0. I oppose > doing so again. People expect their 3.x code to continue working in future > versions. Breaking that expectation should require deprecation for at least > 2 versions. The changes here are only to edge and corner cases, other than as they specifically relate to assignment expressions. The current behaviour is intended to "do the right thing" according to people's expectations, and it largely does so; those cases are not changing. For list comprehensions at global or function scope, the ONLY case that can change (to my knowledge) is where you reuse a variable name: [t for t in t.__parameters__ if t not in tvars] This works in 3.7 but will fail easily and noisily (UnboundLocalError) with PEP 572. IMO this is a poor way to write a loop, and the fact that it "happened to work" is on par with code that depended on dict iteration order in Python 3.2 and earlier. Yes, the implementation is well defined, but since you can achieve exactly the same thing by picking a different variable name, it's better to be clear. Note that the second of the open questions would actually return this to current behaviour, by importing the name 't' into the local scope. The biggest semantic change is to the way names are looked up at class scope. Currently, the behaviour is somewhat bizarre unless you think in terms of unrolling a loop *as a function*; there is no way to reference names from the current scope, and you will instead ignore the surrounding class and "reach out" into the next scope outwards (probably global scope). Out of all the code in the stdlib, the *only* one that needed changing was in Lib/typing.py, where the above comprehension was found. (Not counting a couple of unit tests whose specific job is to verify this behaviour.) The potential for breakage is extremely low. Non-zero, but far lower than the cost of introducing a new keyword, for instance, which is done without deprecation cycles. >> Merely introducing a way to assign as an expression >> would create bizarre edge cases around comprehensions, though, and to >> avoid >> the worst of the confusions, we change the definition of comprehensions, >> causing some edge cases to be interpreted differently, but maintaining the >> existing behaviour in the majority of situations. > > > If it is really true that introducing 'name := expression' requires such a > side-effect, then I might oppose it. It's that comprehensions/genexps are currently bizarre, only people don't usually notice it because there aren't many ways to recognize the situation. Introducing assignment expressions will make the existing weirdnesses more visible. >> Syntax and semantics >> ==================== >> >> In any context where arbitrary Python expressions can be used, a **named >> expression** can appear. This is of the form ``target := expr`` where >> ``expr`` is any valid Python expression, and ``target`` is any valid >> assignment target. > > > This generalization is different from what you said in the abstract and > rationale. No rationale is given. After reading Nick's examination of the > generalization, and your response, -1. Without trying it or looking up any reference documentation, can you tell me whether these statements are legal? with open(f) as self.file: pass try: pass except Exception as self.exc: pass The rationale for assignment to arbitrary targets is the same as for assigning to names: it's useful to be able to assign as an expression. >> Differences from regular assignment statements >> ---------------------------------------------- >> >> Most importantly, since ``:=`` is an expression, it can be used in >> contexts >> where statements are illegal, including lambda functions and >> comprehensions. >> >> An assignment statement can assign to multiple targets, left-to-right:: >> >> x = y = z = 0 > > > This is a bad example as there is very seldom a reason to assign multiple > names, as opposed to multiple targets. Here is a typical real example. > > self.x = x = expression > # Use local x in the rest of the method. > > In "x = y = 0", x and y likely represent two *different* concepts > (variables) that happen to be initialized with the same value. One could > instead write "x,y = 0,0". Personally, if I need to quickly set a bunch of things to zero or None, I'll use chained assignment. But sure. If you want to, you can repeat the zero. Don't forget that adding or removing a target then also requires that you update the tuple, and that it's not a syntax error to fail to do so. >> The equivalent assignment expression > > should be a syntax error. > >> is parsed as separate binary operators, > > ':=' is not a binary operator, any more than '=' is, as names, and targets > in general, are not objects. Neither fetch and operate on the current > value, if any, of the name or target. Therefore neither has an > 'associativity'. What would you call it then? I need some sort of word to use. >> When a class scope is involved, a naive transformation into a function >> would >> prevent name lookups (as the function would behave like a method):: >> >> class X: >> names = ["Fred", "Barney", "Joe"] >> prefix = "> " >> prefixed_names = [prefix + name for name in names] >> >> With Python 3.7 semantics, > > I believe in all of 3.x .. Probably, but that isn't my point. >> this will evaluate the outermost iterable at class >> scope, which will succeed; but it will evaluate everything else in a >> function:: >> >> class X: >> names = ["Fred", "Barney", "Joe"] >> prefix = "> " >> def (iterator): >> result = [] >> for name in iterator: >> result.append(prefix + name) >> return result >> prefixed_names = (iter(names)) >> >> The name ``prefix`` is thus searched for at global scope, ignoring the >> class >> name. > > > And today it fails. This has nothing to do with adding name assignment > expressions. Fails in what way? > Bottom line: I suggest rewriting again, as indicated, changing title to > 'Name Assignment Expressions'. You're welcome to write a competing proposal :) I'm much happier promoting a full-featured assignment expression than something that can only be used in a limited set of situations. Is there reason to believe that extensions to the := operator might take it in a different direction? If not, there's very little to lose by permitting any assignment target, and then letting style guides frown on it if they like. ChrisA From guido at python.org Tue Apr 17 20:20:29 2018 From: guido at python.org (Guido van Rossum) Date: Tue, 17 Apr 2018 17:20:29 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On Tue, Apr 17, 2018 at 3:23 PM, Tim Peters wrote: > [Tim] > >> I'll channel that Guido would be happiest if this rule were followed: > >> > >> Given an assignment statement using "=", the meaning is the same if > >> "=" is replaced with ":=". > Thanks for channeling me. :=) I'd like Guido to chime in again, because I'm pretty sure he won't > accept what's currently on the table. There are two plausible ways to > repair that: > > 1. Continue down the road of making assignment expressions "exactly > like" assignment statements in their full generality. > > 2. Back off and limit assignment expressions to what appears to be the > overwhelmingly most common case motivated by looking at real code (as > opposed to constructing examples to illustrate pitfalls & > obscurities): > > identifier ":=" expression > I haven't had the time to follow this thread in detail; fortunately I don't have to because of Tim's excellent channeling. I am fine with this, it certainly seems the easiest to implement, with the fewest corner cases, and the easiest restriction to explain. (I was thinking there would be a use case for basic tuple unpacking, like seen a lot in for-loop, but the only examples I tried to come up with were pretty sub-optimal, so I don't worry about that any more.) -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue Apr 17 21:20:48 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 18 Apr 2018 11:20:48 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: <20180418012048.GQ11616@ando.pearwood.info> On Wed, Apr 18, 2018 at 10:13:58AM +1000, Chris Angelico wrote: [regarding comprehensions] > The changes here are only to edge and corner cases, other than as they > specifically relate to assignment expressions. The current behaviour > is intended to "do the right thing" according to people's > expectations, and it largely does so; those cases are not changing. > For list comprehensions at global or function scope, the ONLY case > that can change (to my knowledge) is where you reuse a variable name: > > [t for t in t.__parameters__ if t not in tvars] > > This works in 3.7 but will fail easily and noisily (UnboundLocalError) > with PEP 572. That's a major semantic change, and the code you show is no better or worse than: t = ... result = [] for t in t.parameters: if t not in tvars: result.append(t) which is allowed. I think you need a better justification for breaking it than merely the opinion: > IMO this is a poor way to write a loop, Reusing names is permitted in Python. If you're going to break code, that surely needs a future import or deprecation period. As you say yourself, you've already found one example in the standard library that will break. > and the fact > that it "happened to work" is on par with code that depended on dict > iteration order in Python 3.2 and earlier. I don't think that's justified. As far as I can tell, the fact that it works is not a mere accident of implementation but a consequence of the promised semantics of comprehensions and Python's scoping rules. If that's not the case, I think you need to justify exactly why it isn't guaranteed. > Yes, the implementation is > well defined, but since you can achieve exactly the same thing by > picking a different variable name, it's better to be clear. Ah, but the aim of the PEP is not to prohibit ugly or unclear code. > Note that the second of the open questions would actually return this > to current behaviour, by importing the name 't' into the local scope. Indeed. Maybe this needs to stop being an open question and become a settled question. > The biggest semantic change is to the way names are looked up at class > scope. Currently, the behaviour is somewhat bizarre unless you think > in terms of unrolling a loop *as a function*; there is no way to > reference names from the current scope, and you will instead ignore > the surrounding class and "reach out" into the next scope outwards > (probably global scope). > > Out of all the code in the stdlib, the *only* one that needed changing > was in Lib/typing.py, where the above comprehension was found. (Not > counting a couple of unit tests whose specific job is to verify this > behaviour.) If there are tests which intentionally verify this behaviour, that really hurts your position that the behaviour is an accident of implementation. It sounds like the behaviour is intended and required. > The potential for breakage is extremely low. Non-zero, but > far lower than the cost of introducing a new keyword, for instance, > which is done without deprecation cycles. Which new keywords are you thinking of? The most recent new keywords I can think of were "True/False", "as" and "with". - True, False became keywords in 3.x during the "breaking code is allowed" 2 -> 3 transition; - "as" became a keyword in 2.6 following a deprecation period in 2.5: py> as = 1 :1: Warning: 'as' will become a reserved keyword in Python 2.6 - and "with" needed a __future__ import. Have I missed any? -- Steve From tim.peters at gmail.com Tue Apr 17 21:35:35 2018 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 17 Apr 2018 20:35:35 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: [Guido, makes peace with `identifier := expression`] > ... > I am fine with this, it certainly seems the easiest to implement, with the > fewest corner cases, and the easiest restriction to explain. > > (I was thinking there would be a use case for basic tuple unpacking, like > seen a lot in for-loop, but the only examples I tried to come up with were > pretty sub-optimal, so I don't worry about that any more.) Chris's pain threshold appears to be higher than ours ;-) So I would really like to see if anyone has plausibly realistic uses for fancier forms of assignment expression. I have plenty of code that does stuff like this: while True: x, y = func_returning_tuple() if y is None: break ... Maybe it's just that I'm used to it, but I find that very easy to understand now. If we had fancy assignment expressions, my first thought was I could write it like so instead: while ((x, y) := func_returning_tuple()) and y is not None: ... and pray that I put in enough parens to get the intended meaning. And maybe it's just that I'm _not_ used to that, but I do find it harder to understand. Contributing factor: I don't really want "and" there - what the context requires is really more like C's comma operator (take only the last value from a sequence of expressions). As is, I'm relying on that a 2-tuple is truthy regardless of its content (so that `and` always goes on to evaluate its RHS). And, for some reason, I find this even worse: while ((x, y) := func_returning_tuple())[1] is not None: ... The rub there: I gave `y` a name but can't use it in the test?! And those are the same kinds of headaches I saw over & over in my own "fancier" code: stuff that's already perfectly clear would become more obscure instead. Tuple unpacking works great in for-loops because the only effect there is to give names to the tuple components, none of which are needed _in_ the `for` statement itself. But in a `while" or `if` statement, I would typically _also_ want to use the names _in_ the `while` or `if` tests. But, as in C, that's what the comma operator is for, not the assignment operator. while (s = function_returning_struct_with_x_and_y_members(), s.y != NULL) { ... } In contrast, ,many plausible uses I saw for `identifier := expression` in a `while` or `if` statement would have been improvements, and most of the rest neutral: I'm still wondering whether this one is better or worse ;-): def newton(f, fprime, x): import math while not math.isclose((next_x := x - f(x) / fprime(x)), x): x = next_x return next_x From rosuav at gmail.com Tue Apr 17 22:04:04 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 18 Apr 2018 12:04:04 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <20180418012048.GQ11616@ando.pearwood.info> References: <20180418012048.GQ11616@ando.pearwood.info> Message-ID: On Wed, Apr 18, 2018 at 11:20 AM, Steven D'Aprano wrote: > On Wed, Apr 18, 2018 at 10:13:58AM +1000, Chris Angelico wrote: > > [regarding comprehensions] > >> The changes here are only to edge and corner cases, other than as they >> specifically relate to assignment expressions. The current behaviour >> is intended to "do the right thing" according to people's >> expectations, and it largely does so; those cases are not changing. >> For list comprehensions at global or function scope, the ONLY case >> that can change (to my knowledge) is where you reuse a variable name: >> >> [t for t in t.__parameters__ if t not in tvars] >> >> This works in 3.7 but will fail easily and noisily (UnboundLocalError) >> with PEP 572. > > That's a major semantic change, and the code you show is no better or > worse than: > > t = ... > result = [] > for t in t.parameters: > if t not in tvars: > result.append(t) > > > which is allowed. I think you need a better justification for breaking > it than merely the opinion: > >> IMO this is a poor way to write a loop, Ah but that isn't what the list comp is equivalent to. If you want to claim that "for t in t.parameters" is legal, you first have to assume that you're overwriting t, not shadowing it. In the list comp as it is today, the "for t in" part is inside an implicit nested function, but the "t.parameters" part is outside that function. Try this instead: t = ... def listcomp(): result = [] for t in t.parameters: if t not in tvars: result.append(t) return result listcomp() Except that it isn't that either, because the scope isn't quite that clean. It actually involves a function parameter, and the iterator is fetched before it's passed as a parameter, and then NOT fetched inside the loop. So you actually can't write perfectly equivalent longhand. PEP 572 will *reduce* the edge cases and complexity. >> Note that the second of the open questions would actually return this >> to current behaviour, by importing the name 't' into the local scope. > > Indeed. Maybe this needs to stop being an open question and become a > settled question. Okay. The question is open if you wish to answer it. Are you happy with the extra complexity that this would entail? Discuss. >> The biggest semantic change is to the way names are looked up at class >> scope. Currently, the behaviour is somewhat bizarre unless you think >> in terms of unrolling a loop *as a function*; there is no way to >> reference names from the current scope, and you will instead ignore >> the surrounding class and "reach out" into the next scope outwards >> (probably global scope). >> >> Out of all the code in the stdlib, the *only* one that needed changing >> was in Lib/typing.py, where the above comprehension was found. (Not >> counting a couple of unit tests whose specific job is to verify this >> behaviour.) > > If there are tests which intentionally verify this behaviour, that > really hurts your position that the behaviour is an accident of > implementation. It sounds like the behaviour is intended and required. These changes also broke some tests of disassembly, which quote the exact bytecode created for specific pieces of code. Does that mean that we can't change anything? They're specifically verifying (asserting) the behaviour that currently exists. I've never tried to claim that this is going to have no impact. Of course it will change things. Otherwise why have a PEP? >> The potential for breakage is extremely low. Non-zero, but >> far lower than the cost of introducing a new keyword, for instance, >> which is done without deprecation cycles. > > Which new keywords are you thinking of? The most recent new keywords I > can think of were "True/False", "as" and "with". async, await? They became "soft keywords" and then full keywords, but that's not exactly a deprecation period. rosuav at sikorsky:~$ python3.5 -c "await = 1; print(await)" 1 rosuav at sikorsky:~$ python3.6 -c "await = 1; print(await)" 1 rosuav at sikorsky:~$ python3.7 -c "await = 1; print(await)" File "", line 1 await = 1; print(await) ^ SyntaxError: invalid syntax The shift from 3.6 to 3.7 breaks any code that uses 'async' or 'await' as an identifier. And that kind of breakage CAN happen in a minor release. Otherwise, it would be virtually impossible to improve anything in the language. PEP 572 will make changes. The result of these changes will be fewer complex or unintuitive interactions between different features in Python, most notably comprehensions/genexps and class scope. It also makes the transformation from list comp to external function more accurate and easier to understand. For normal usage, the net result will be the same, but the differences are visible if you actually probe for them. ChrisA From ericfahlgren at gmail.com Tue Apr 17 23:24:33 2018 From: ericfahlgren at gmail.com (Eric Fahlgren) Date: Tue, 17 Apr 2018 20:24:33 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <20180418012048.GQ11616@ando.pearwood.info> References: <20180418012048.GQ11616@ando.pearwood.info> Message-ID: On Tue, Apr 17, 2018 at 6:20 PM, Steven D'Aprano wrote: > If there are tests which intentionally verify this behaviour, that > really hurts your position that the behaviour is an accident of > implementation. It sounds like the behaviour is intended and required. > ?It is nonetheless bizarre and unexpected behavior. >>> prefix = 'global' >>> [prefix+c for c in 'abc'] ['globala', 'globalb', 'globalc'] >>> def func(): ... prefix = 'local' ... print([prefix+c for c in 'abc']) >>> func() ['locala', 'localb', 'localc'] >>> class klass: ... prefix = 'classy' ... items = [prefix+c for c in 'abc'] >>> print(klass.items) ['globala', 'globalb', 'globalc']? In Python 2, that last one would produce 'classya' and friends, due to the "broken" comprehension scope. -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Tue Apr 17 17:44:26 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 18 Apr 2018 09:44:26 +1200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <5AD60EC9.5010901@stoneleaf.us> Message-ID: <5AD66ABA.5090908@canterbury.ac.nz> Paul Moore wrote: > the next question will likely be "so why does = exist at all?" And if we decide to make ':=' the official assigment operator and deprectate '=', the next question will be "Why do we have '==' instead of '='?" -- Greg From tim.peters at gmail.com Tue Apr 17 23:57:15 2018 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 17 Apr 2018 22:57:15 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <5AD66ABA.5090908@canterbury.ac.nz> References: <5AD60EC9.5010901@stoneleaf.us> <5AD66ABA.5090908@canterbury.ac.nz> Message-ID: [Paul Moore] >> the next question will likely be "so why does = exist at all?" [Greg Ewing ] > And if we decide to make ':=' the official assigment operator and > deprectate '=', the next question will be "Why do we have '==' > instead of '='?" Which would be a fine question! In Python's very early days, it didn't have "==" at all: plain "=" was used for both assignment and equality testing. >From the HISTORY file: """ New features in 0.9.6: ... - '==' is now the only equality operator; "../demo/scripts/eqfix.py" is a script that fixes old Python modules """ That script crawled a source tree and replaced instances of "=" used for equality testing with the new-fangled "==". We can obviously do something similar to replace instances of "=" used for assignment when that's removed, and I'm sure nobody will complain about that either ;-) From J.Demeyer at UGent.be Wed Apr 18 01:49:47 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Wed, 18 Apr 2018 07:49:47 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <95cbdb99447b4f96a9423e47cd04c0a6@xmail101.UGent.be> References: <95cbdb99447b4f96a9423e47cd04c0a6@xmail101.UGent.be> Message-ID: <5AD6DC7B.1090201@UGent.be> On 2018-04-18 02:13, Chris Angelico wrote: > I'm much happier promoting a full-featured assignment expression than > something that can only be used in a limited set of situations. Is > there reason to believe that extensions to the := operator might take > it in a different direction? If not, there's very little to lose by > permitting any assignment target, and then letting style guides frown > on it if they like. This is a very good argument: why artificially restrict the operator? This reminds me of the artificial restriction of decorator syntax (why is @foo()() not legal?). There was never a rationale given for that and now we are stuck with it. From ncoghlan at gmail.com Wed Apr 18 06:18:58 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 18 Apr 2018 20:18:58 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <5AD6DC7B.1090201@UGent.be> References: <95cbdb99447b4f96a9423e47cd04c0a6@xmail101.UGent.be> <5AD6DC7B.1090201@UGent.be> Message-ID: On 18 April 2018 at 15:49, Jeroen Demeyer wrote: > On 2018-04-18 02:13, Chris Angelico wrote: >> >> I'm much happier promoting a full-featured assignment expression than >> something that can only be used in a limited set of situations. Is >> there reason to believe that extensions to the := operator might take >> it in a different direction? If not, there's very little to lose by >> permitting any assignment target, and then letting style guides frown >> on it if they like. > > > This is a very good argument: why artificially restrict the operator? > > This reminds me of the artificial restriction of decorator syntax (why is > @foo()() not legal?). There was never a rationale given for that and now we > are stuck with it. That's not the case - Guido's given in-principle approval for the decorator restriction to be lifted (I believe primarily due to the validity of the "@ns[lookup]" case), but it's never irritated anyone enough for them to do the work to propose the necessary grammar and reference implementation changes (vs using a pass-through function like "@decorator(ns[lookup])" to bypass the restriction). So what these kinds of initial restrictions do is require that any extra complexity be justified *in its own right*, rather than coming along for the ride as part of proposals with motivating use cases that only require much simpler capabilities (names and calls in the case of decorators, names in the case of assignment expressions). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Wed Apr 18 06:23:39 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 18 Apr 2018 20:23:39 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On 18 April 2018 at 11:35, Tim Peters wrote: > And, for some reason, I find this even worse: > > while ((x, y) := func_returning_tuple())[1] is not None: > ... > > The rub there: I gave `y` a name but can't use it in the test?! > > And those are the same kinds of headaches I saw over & over in my own > "fancier" code: stuff that's already perfectly clear would become > more obscure instead. Whereas I think: while (s := func_returning_tuple())[1] is not None: s = x, y ... compares favourably with the loop-and-a-half version. It does make the guarantee that "y is not None" harder to spot than it is in the loop-and-a-half version, though. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Wed Apr 18 06:25:03 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 18 Apr 2018 20:25:03 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On 18 April 2018 at 10:20, Guido van Rossum wrote: [Tim Peters wrote] >> 2. Back off and limit assignment expressions to what appears to be the >> overwhelmingly most common case motivated by looking at real code (as >> opposed to constructing examples to illustrate pitfalls & >> obscurities): >> >> identifier ":=" expression > > > I haven't had the time to follow this thread in detail; fortunately I don't > have to because of Tim's excellent channeling. > > I am fine with this, it certainly seems the easiest to implement, with the > fewest corner cases, and the easiest restriction to explain. > > (I was thinking there would be a use case for basic tuple unpacking, like > seen a lot in for-loop, but the only examples I tried to come up with were > pretty sub-optimal, so I don't worry about that any more.) In the other direction I was thinking about the question "Then why do I think tuple unpacking is OK in comprehensions?", and realised that it's because in that situation there are keywords as delimiters on both sides (i.e. "... for name [, name]* in ..."), so it's harder for the unpacking operation to get confused with other uses of commas as separators. Similarly, in regular assignments, the unpacking target is always either between two "=" or else from the start of the line to the first "=". By contrast, for assignment expressions, the only potential explicit opening delimiter is "(", and that's also the case for tuple literals. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From guido at python.org Wed Apr 18 09:58:37 2018 From: guido at python.org (Guido van Rossum) Date: Wed, 18 Apr 2018 06:58:37 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180418012048.GQ11616@ando.pearwood.info> Message-ID: On Tue, Apr 17, 2018 at 7:04 PM, Chris Angelico wrote: > On Wed, Apr 18, 2018 at 11:20 AM, Steven D'Aprano > wrote: > > On Wed, Apr 18, 2018 at 10:13:58AM +1000, Chris Angelico wrote: > > > > [regarding comprehensions] > > > >> The changes here are only to edge and corner cases, other than as they > >> specifically relate to assignment expressions. The current behaviour > >> is intended to "do the right thing" according to people's > >> expectations, and it largely does so; those cases are not changing. > >> For list comprehensions at global or function scope, the ONLY case > >> that can change (to my knowledge) is where you reuse a variable name: > >> > >> [t for t in t.__parameters__ if t not in tvars] > >> > >> This works in 3.7 but will fail easily and noisily (UnboundLocalError) > >> with PEP 572. > > > > That's a major semantic change, and the code you show is no better or > > worse than: > > > > t = ... > > result = [] > > for t in t.parameters: > > if t not in tvars: > > result.append(t) > > > > > > which is allowed. I think you need a better justification for breaking > > it than merely the opinion: > > > >> IMO this is a poor way to write a loop, > > Ah but that isn't what the list comp is equivalent to. If you want to > claim that "for t in t.parameters" is legal, you first have to assume > that you're overwriting t, not shadowing it. In the list comp as it is > today, the "for t in" part is inside an implicit nested function, but > the "t.parameters" part is outside that function. > > Try this instead: > > t = ... > def listcomp(): > result = [] > for t in t.parameters: > if t not in tvars: > result.append(t) > return result > listcomp() > > Except that it isn't that either, because the scope isn't quite that > clean. It actually involves a function parameter, and the iterator is > fetched before it's passed as a parameter, and then NOT fetched inside > the loop. So you actually can't write perfectly equivalent longhand. > > PEP 572 will *reduce* the edge cases and complexity. I can't tell from this what the PEP actually says should happen in that example. When I first saw it I thought "Gaah! What a horrible piece of code." But it works today, and people's code *will* break if we change its meaning. However we won't have to break that. Suppose the code is (perversely) t = range(3) a = [t for t in t if t] If we translate this to t = range(3) def listcomp(t=t): a = [] for t in t: if t: a.append(t) return a a = listcomp() Then it will still work. The trick will be to recognize "imported" names that are also assigned and capture those (as well as other captures as already described in the PEP). -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Wed Apr 18 10:35:05 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 19 Apr 2018 00:35:05 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180418012048.GQ11616@ando.pearwood.info> Message-ID: On Wed, Apr 18, 2018 at 11:58 PM, Guido van Rossum wrote: > I can't tell from this what the PEP actually says should happen in that > example. When I first saw it I thought "Gaah! What a horrible piece of > code." But it works today, and people's code *will* break if we change its > meaning. > > However we won't have to break that. Suppose the code is (perversely) > > t = range(3) > a = [t for t in t if t] > > If we translate this to > > t = range(3) > def listcomp(t=t): > a = [] > for t in t: > if t: > a.append(t) > return a > a = listcomp() > > Then it will still work. The trick will be to recognize "imported" names > that are also assigned and capture those (as well as other captures as > already described in the PEP). That can be done. However, this form of importing will have one of two consequences: 1) Referencing an unbound name will scan to outer scopes at run time, changing the semantics of Python name lookups 2) Genexps will eagerly evaluate a lookup if it happens to be the same name as an internal iteration variable. Of the two, #2 is definitely my preference, but it does mean more eager binding. While this won't make a difference in the outermost iterable (since that's *already* eagerly bound), it might make a difference with others: t = range(3) gen = (t for _ in range(1) for t in t if t) t = [4, 5, 6] print(next(gen)) print(next(gen)) Current semantics: UnboundLocalError on first next() call. PEP 572 semantics: Either UnboundLocalError (with current reference implementation) or it yields 1 and 2 (with eager lookups). So either we change things for the outermost iterable, or we change things for everything BUT the outermost iterable. Either way, I'm happy to eliminate the special-casing of the outermost iterable. Yes, it's a change in semantics, but a change that removes special cases is generally better than one that creates them. ChrisA From guido at python.org Wed Apr 18 12:18:22 2018 From: guido at python.org (Guido van Rossum) Date: Wed, 18 Apr 2018 09:18:22 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180418012048.GQ11616@ando.pearwood.info> Message-ID: On Wed, Apr 18, 2018 at 7:35 AM, Chris Angelico wrote: > On Wed, Apr 18, 2018 at 11:58 PM, Guido van Rossum > wrote: > > I can't tell from this what the PEP actually says should happen in that > > example. When I first saw it I thought "Gaah! What a horrible piece of > > code." But it works today, and people's code *will* break if we change > its > > meaning. > > > > However we won't have to break that. Suppose the code is (perversely) > > > > t = range(3) > > a = [t for t in t if t] > > > > If we translate this to > > > > t = range(3) > > def listcomp(t=t): > > a = [] > > for t in t: > > if t: > > a.append(t) > > return a > > a = listcomp() > > > > Then it will still work. The trick will be to recognize "imported" names > > that are also assigned and capture those (as well as other captures as > > already described in the PEP). > > That can be done. However, this form of importing will have one of two > consequences: > > 1) Referencing an unbound name will scan to outer scopes at run time, > changing the semantics of Python name lookups > I'm not even sure what this would do. > 2) Genexps will eagerly evaluate a lookup if it happens to be the same > name as an internal iteration variable. > I think we would have to specify this more precisely. Let's say by "eagerly evaluate a lookup" you mean "include it in the function parameters with a default value being the lookup (i.e. starting in the outer scope), IOW "t=t" as I showed above. The question is *when* we would do this. IIUC the PEP already does this if the "outer scope" is a class scope for any names that a simple static analysis shows are references to variables in the class scope. (I don't know exactly what this static analysis should do but it could be as simple as gathering all names that are assigned to in the class, or alternatively all names assigned to before the point where the comprehension occurs. We shouldn't be distracted by dynamic definitions like `exec()` although we should perhaps be aware of `del`.) My proposal is to extend this static analysis for certain loop control variables (any simple name assigned to in a for-clause in the comprehension), regardless of what kind of scope the outer scope is. If the outer scope is a function we already know how to do this. If it's a class we use the analysis referred to above. If the outer scope is the global scope we have to do something new. I propose to use the same simple static analysis we use for class scopes. Furthermore I propose to *only* do this for the loop control variable(s) of the outermost for-clause, since that's the only place where without all this rigmarole we would have a clear difference in behavior with Python 3.7 in cases like [t for t in t]. Oh, and probably we only need to do this if that loop control variable is also used as an expression in the iterable (so we don't waste time doing any of this for e.g. [t for t in q]). (But what about [t for _ in t for t in t]? That's currently an UnboundLocalError and we shouldn't try to "fix" that case.) Since we now have once again introduced an exception for the outermost loop control variable and the outermost iterable, we can consider doing this only as a temporary measure. We could have a goal to eventually make [t for t in t] fail, and in the meantime we would deprecate it -- e.g. in 3.8 a silent deprecation, in 3.9 a noisy one, in 3.10 break it. Yes, that's a lot of new static analysis for deprecating an edge case, but it seems reasonable to want to preserve backward compatibility when breaking this edge case since it's likely not all that uncommon. Even if most occurrences are bad style written by lazy programmers, we should not break working code, if it is reasonable to expect that it's relied upon in real code. > Of the two, #2 is definitely my preference, but it does mean more > eager binding.While this won't make a difference in the outermost > iterable (since that's *already* eagerly bound), it might make a > difference with others: > > t = range(3) > gen = (t for _ in range(1) for t in t if t) > t = [4, 5, 6] > print(next(gen)) > print(next(gen)) > I don't like this particular example, because it uses an obscure bit of semantics of generator expressions. It's fine to demonstrate the finer details of how those work, but it's unlikely to see real code relying on this. (As I argued before, generator expressions are typically either fed into other code that eagerly evaluates them before reaching the next line, or returned from a function, and in the latter case intentional modification of some variable in that function's scope to affect the meaning of the generator expression would seem a remote possibility at best, and an accident waiting to happen at worst.) > Current semantics: UnboundLocalError on first next() call. > > PEP 572 semantics: Either UnboundLocalError (with current reference > implementation) or it yields 1 and 2 (with eager lookups). > > So either we change things for the outermost iterable, or we change > things for everything BUT the outermost iterable. Either way, I'm > happy to eliminate the special-casing of the outermost iterable. Yes, > it's a change in semantics, but a change that removes special cases is > generally better than one that creates them. > Hopefully my proposal above satisfies you. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Wed Apr 18 12:38:02 2018 From: brett at python.org (Brett Cannon) Date: Wed, 18 Apr 2018 16:38:02 +0000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <40QRfq2w5LzFqsw@mail.python.org> Message-ID: On Tue, 17 Apr 2018 at 14:53 Eric Snow wrote: > On Tue, Apr 17, 2018 at 7:55 AM, Steve Dower > wrote: > > Agree with Paul. The PEP is well thought out and well presented, but I > > really don?t think we need this in Python (and I say this as someone who > > uses it regularly in C/C#). > > > > -1 on the idea; no disrespect intended toward to people who did a lot of > > work on it. > > Same here. I'm more interested in having anonymous blocks (i.e. > scopes), along the lines of PEP 3150, though it's currently #2 on my > wish list. :) > I'm also -1. I understand the usefulness of the construct in languages where block scopes make having this kind of expression assignment in e.g. an 'if' guard useful. But for Python and it's LGB scoping -- although I think we need to add an "N" for "non-local" :) -- this is syntactic sugar and I don't see enough wide benefit on top of the potential ugliness/misuse of it to warrant the cognitive overhead of adding it. Although, as usual, Chris, great PEP! :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.peters at gmail.com Wed Apr 18 12:38:26 2018 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 18 Apr 2018 11:38:26 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: [Tim] >> And, for some reason, I find this even worse: >> >> while ((x, y) := func_returning_tuple())[1] is not None: >> ... >> >> The rub there: I gave `y` a name but can't use it in the test?! >> >> And those are the same kinds of headaches I saw over & over in my own >> "fancier" code: stuff that's already perfectly clear would become >> more obscure instead. [Nick] > Whereas I think: > > while (s := func_returning_tuple())[1] is not None: > s = x, y > ... > > compares favourably with the loop-and-a-half version. Obviously not, since it really needs to be x, y = s instead ;-) In context, I was looking for realistic cases in which assignment expressions _fancier than_ identifier ":=" expression is a real improvement. You found an improvement instead by _replacing_ a "fancier than" instance with a plain-single-name target. I already have lots of examples from real code where plain-single-name target reads better to me. I don't have any yet from real code where something fancier does. In this specific case, I find your rewriting about as readable as the loop-and-a-half, except for the obvious drawback of the former: > It does make the guarantee that "y is not None" harder to spot than it > is in the loop-and-a-half version, though. Over time, the functions in the real codes from which the example was synthesized change, sometimes carrying more or less state in tuples. When that happens, the original x, y = s will helpfully blow up (size mismatch in unpacking), But, if the tuple length increased, is it still the case that I want to test the 1'th component? The test is now divorced from the unpacking. I do know that I'll still want to test the component I think of as being "the 'y' component", and the loop-and-a-half version accommodates that naturally. Then again, I could switch to new-fanged namedtuples instead, and do while (s := func_returning_tuple()).y is not None: to get the best of all worlds. From python at mrabarnett.plus.com Wed Apr 18 13:43:51 2018 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 18 Apr 2018 18:43:51 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <5AD66ABA.5090908@canterbury.ac.nz> References: <5AD60EC9.5010901@stoneleaf.us> <5AD66ABA.5090908@canterbury.ac.nz> Message-ID: <182f3235-7046-6ae0-6a95-61a0bcbbddb5@mrabarnett.plus.com> On 2018-04-17 22:44, Greg Ewing wrote: > Paul Moore wrote: >> the next question will likely be "so why does = exist at all?" > > And if we decide to make ':=' the official assigment operator and > deprectate '=', the next question will be "Why do we have '==' > instead of '='?" > Some languages use '=' for assignment, others for equality, but do you know of a language that uses ':=' for equality' or '==' for assignment? If Python saw '=' it could ask "Do you mean assignment ':=' or equality '=='?". From barry at python.org Wed Apr 18 14:04:01 2018 From: barry at python.org (Barry Warsaw) Date: Wed, 18 Apr 2018 11:04:01 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <182f3235-7046-6ae0-6a95-61a0bcbbddb5@mrabarnett.plus.com> References: <5AD60EC9.5010901@stoneleaf.us> <5AD66ABA.5090908@canterbury.ac.nz> <182f3235-7046-6ae0-6a95-61a0bcbbddb5@mrabarnett.plus.com> Message-ID: <254D07A5-6C04-4C96-98A2-11F5D776D0AA@python.org> On Apr 18, 2018, at 10:43, MRAB wrote: > Some languages use '=' for assignment, others for equality, but do you know of a language that uses ':=' for equality' or '==' for assignment? Clearly we should take a page from the ternary operator and make the assignment expression operator just ugly enough that people won?t overuse it. Since I can?t have ?>>? or ?<>? back, I propose ?=======?. go-ahead-count-?em-every-time-ly y?rs, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From rosuav at gmail.com Wed Apr 18 14:17:40 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 19 Apr 2018 04:17:40 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180418012048.GQ11616@ando.pearwood.info> Message-ID: On Thu, Apr 19, 2018 at 2:18 AM, Guido van Rossum wrote: > On Wed, Apr 18, 2018 at 7:35 AM, Chris Angelico wrote: >> >> On Wed, Apr 18, 2018 at 11:58 PM, Guido van Rossum >> wrote: >> > I can't tell from this what the PEP actually says should happen in that >> > example. When I first saw it I thought "Gaah! What a horrible piece of >> > code." But it works today, and people's code *will* break if we change >> > its >> > meaning. >> > >> > However we won't have to break that. Suppose the code is (perversely) >> > >> > t = range(3) >> > a = [t for t in t if t] >> > >> > If we translate this to >> > >> > t = range(3) >> > def listcomp(t=t): >> > a = [] >> > for t in t: >> > if t: >> > a.append(t) >> > return a >> > a = listcomp() >> > >> > Then it will still work. The trick will be to recognize "imported" names >> > that are also assigned and capture those (as well as other captures as >> > already described in the PEP). >> >> That can be done. However, this form of importing will have one of two >> consequences: >> >> 1) Referencing an unbound name will scan to outer scopes at run time, >> changing the semantics of Python name lookups > > > I'm not even sure what this would do. The implicit function of the listcomp would attempt to LOAD_FAST 't', and upon finding that it doesn't have a value for it, would go and look for the name 't' in a surrounding scope. (Probably LOAD_CLOSURE.) >> 2) Genexps will eagerly evaluate a lookup if it happens to be the same >> name as an internal iteration variable. > > > I think we would have to specify this more precisely. > > Let's say by "eagerly evaluate a lookup" you mean "include it in the > function parameters with a default value being the lookup (i.e. starting in > the outer scope), IOW "t=t" as I showed above. Yes. To be technically precise, there's no default argument involved, and the call to the implicit function explicitly passes all the arguments. > The question is *when* we > would do this. IIUC the PEP already does this if the "outer scope" is a > class scope for any names that a simple static analysis shows are references > to variables in the class scope. Correct. > (I don't know exactly what this static > analysis should do but it could be as simple as gathering all names that are > assigned to in the class, or alternatively all names assigned to before the > point where the comprehension occurs. We shouldn't be distracted by dynamic > definitions like `exec()` although we should perhaps be aware of `del`.) At the moment, it isn't aware of 'del'. The analysis is simple and 100% static: If a name is in the table of names the class uses AND it's in the table of names the comprehension uses, it gets passed as a parameter. I don't want to try to be aware of del, because of this: class X: x = 1 if y: del x print(x) z = (q for q in x if q) If y is true, this will eagerly look up x using the same semantics in both the print and the genexp (on construction, not when you iterate over the genexp). If y is false, it'll still eagerly look up x, and it'll still use the same semantics for print and the genexp (and it'll find an 'x' in a surrounding scope). (The current implementation actually is a bit different from that. I'm not sure whether it's possible to do it as simply as given without an extra compilation pass. But it's close enough.) > My proposal is to extend this static analysis for certain loop control > variables (any simple name assigned to in a for-clause in the > comprehension), regardless of what kind of scope the outer scope is. If the > outer scope is a function we already know how to do this. If it's a class we > use the analysis referred to above. If the outer scope is the global scope > we have to do something new. I propose to use the same simple static > analysis we use for class scopes. > > Furthermore I propose to *only* do this for the loop control variable(s) of > the outermost for-clause, since that's the only place where without all this > rigmarole we would have a clear difference in behavior with Python 3.7 in > cases like [t for t in t]. Oh, and probably we only need to do this if that > loop control variable is also used as an expression in the iterable (so we > don't waste time doing any of this for e.g. [t for t in q]). Okay. Here's something that would be doable: If the name is written to within the comprehension, AND it is read from in the outermost iterable, it is flagged early-bind. I'll have to try implementing that to be sure, but it should be possible I think. It would cover a lot of cases, keeping them the same as we currently have. > Since we now have once again introduced an exception for the outermost loop > control variable and the outermost iterable, we can consider doing this only > as a temporary measure. We could have a goal to eventually make [t for t in > t] fail, and in the meantime we would deprecate it -- e.g. in 3.8 a silent > deprecation, in 3.9 a noisy one, in 3.10 break it. Yes, that's a lot of new > static analysis for deprecating an edge case, but it seems reasonable to > want to preserve backward compatibility when breaking this edge case since > it's likely not all that uncommon. Even if most occurrences are bad style > written by lazy programmers, we should not break working code, if it is > reasonable to expect that it's relied upon in real code. Fair enough. So the outermost iterable remains special for a short while, with deprecation. I'll get onto the coding side of it during my Copious Free Time, hopefully this week some time. Here's hoping! ChrisA From storchaka at gmail.com Wed Apr 18 15:16:23 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Wed, 18 Apr 2018 22:16:23 +0300 Subject: [Python-Dev] Timing for removing legacy Unicode APIs deprecated by PEP 393 In-Reply-To: References: Message-ID: 13.04.18 16:27, INADA Naoki ????: > Then, I want to reschedule the removal of these APIs. > Can we remove them in 3.8? 3.9? or 3.10? > I prefer sooner as possible. I suppose that many users will start porting to Python 3 only in 2020, after 2.7 EOL. After that time we shouldn't support compatibility with 2.7 and can start emitting deprecation warnings at runtime. After 1 or 2 releases after that we can make corresponding public API always failing and remove private API and data fields. > Slightly off topic, there are 4bytes alignment gap in the unicode object, > on 64bit platform. > > typedef struct { > ..... > struct { > unsigned int interned:2; > unsigned int kind:3; > unsigned int compact:1; > unsigned int ascii:1; > unsigned int ready:1; > unsigned int :24; > } state; // 4 bytes > > // implicit 4 bytes gap here. > > wchar_t *wstr; // 8 bytes > } PyASCIIObject; > > So, I think we can reduce 12 bytes instead of 8 bytes when removing wstr. > Or we can reduce 4 bytes soon by moving `wstr` before `state`. > > Off course, it needs siphash support 4byte aligned data instead of 8byte. There are other functions which expect that data is aligned to sizeof(long) or 8 bytes. Siphash hashing is special because it is called not just for strings and bytes, but for memoryview, which doesn't guarantee any alignment. Note that after removing the wchar_t* field the gap will not gone, because the size of the structure should be a multiple of the alignment of the first field (which is a pointer). From barry at python.org Wed Apr 18 16:26:04 2018 From: barry at python.org (Barry Warsaw) Date: Wed, 18 Apr 2018 13:26:04 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180418012048.GQ11616@ando.pearwood.info> Message-ID: <80492676-1529-4E95-B380-A7DBD4FECCFA@python.org> On Apr 18, 2018, at 11:17, Chris Angelico wrote: > At the moment, it isn't aware of 'del?. I don?t know if it?s relevant to the current discussion, but don?t forget about implicit dels: def foo(): x = 1 try: 1/0 except ZeroDivisionError as x: pass print(x) This is one of my favorite Python oddities because it always makes me look like a genius when I diagnose it. :) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From rosuav at gmail.com Wed Apr 18 17:03:13 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 19 Apr 2018 07:03:13 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <80492676-1529-4E95-B380-A7DBD4FECCFA@python.org> References: <20180418012048.GQ11616@ando.pearwood.info> <80492676-1529-4E95-B380-A7DBD4FECCFA@python.org> Message-ID: On Thu, Apr 19, 2018 at 6:26 AM, Barry Warsaw wrote: > On Apr 18, 2018, at 11:17, Chris Angelico wrote: > >> At the moment, it isn't aware of 'del?. > > I don?t know if it?s relevant to the current discussion, but don?t forget about implicit dels: > > def foo(): > x = 1 > try: > 1/0 > except ZeroDivisionError as x: > pass > print(x) > > > This is one of my favorite Python oddities because it always makes me look like a genius when I diagnose it. :) Heh, yeah. My intention is to ignore that altogether. The general policy in Python is "if ever it MIGHT be assigned to, it belongs to that scope" (so even "if 0: x = 1" will mark x as local), so sticking to that would mean treating x as local regardless of the try/except. (On an unrelated subject, I'm keeping the "sublocal scope" concept from the original PEP on ice. It might be worth implementing exception name binding with a sublocal name. But that's for a completely separate PEP.) ChrisA From greg at krypto.org Wed Apr 18 17:26:17 2018 From: greg at krypto.org (Gregory P. Smith) Date: Wed, 18 Apr 2018 21:26:17 +0000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <254D07A5-6C04-4C96-98A2-11F5D776D0AA@python.org> References: <5AD60EC9.5010901@stoneleaf.us> <5AD66ABA.5090908@canterbury.ac.nz> <182f3235-7046-6ae0-6a95-61a0bcbbddb5@mrabarnett.plus.com> <254D07A5-6C04-4C96-98A2-11F5D776D0AA@python.org> Message-ID: On Wed, Apr 18, 2018 at 11:04 AM Barry Warsaw wrote: > On Apr 18, 2018, at 10:43, MRAB wrote: > > > Some languages use '=' for assignment, others for equality, but do you > know of a language that uses ':=' for equality' or '==' for assignment? > > Clearly we should take a page from the ternary operator and make the > assignment expression operator just ugly enough that people won?t overuse > it. Since I can?t have ?>>? or ?<>? back, I propose ?=======?. > go-ahead-count-?em-every-time-ly y?rs, > 8 of course. to "match" what merge conflict markers look like. ;) php already uses === for something, we should just use =========== so we can say "it goes to eleven", ending the operator war once and for all. :P -gps -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Wed Apr 18 17:49:55 2018 From: guido at python.org (Guido van Rossum) Date: Wed, 18 Apr 2018 14:49:55 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180418012048.GQ11616@ando.pearwood.info> Message-ID: On Wed, Apr 18, 2018 at 11:17 AM, Chris Angelico wrote: > On Thu, Apr 19, 2018 at 2:18 AM, Guido van Rossum > wrote: > > On Wed, Apr 18, 2018 at 7:35 AM, Chris Angelico > wrote: > >> > >> On Wed, Apr 18, 2018 at 11:58 PM, Guido van Rossum > >> wrote: > >> > I can't tell from this what the PEP actually says should happen in > that > >> > example. When I first saw it I thought "Gaah! What a horrible piece of > >> > code." But it works today, and people's code *will* break if we change > >> > its > >> > meaning. > >> > > >> > However we won't have to break that. Suppose the code is (perversely) > >> > > >> > t = range(3) > >> > a = [t for t in t if t] > >> > > >> > If we translate this to > >> > > >> > t = range(3) > >> > def listcomp(t=t): > >> > a = [] > >> > for t in t: > >> > if t: > >> > a.append(t) > >> > return a > >> > a = listcomp() > >> > > >> > Then it will still work. The trick will be to recognize "imported" > names > >> > that are also assigned and capture those (as well as other captures as > >> > already described in the PEP). > >> > >> That can be done. However, this form of importing will have one of two > >> consequences: > >> > >> 1) Referencing an unbound name will scan to outer scopes at run time, > >> changing the semantics of Python name lookups > > > > > > I'm not even sure what this would do. > > The implicit function of the listcomp would attempt to LOAD_FAST 't', > and upon finding that it doesn't have a value for it, would go and > look for the name 't' in a surrounding scope. (Probably LOAD_CLOSURE.) > We agree that that's too dynamic to be explainable. > >> 2) Genexps will eagerly evaluate a lookup if it happens to be the same > >> name as an internal iteration variable. > > > > > > I think we would have to specify this more precisely. > > > > Let's say by "eagerly evaluate a lookup" you mean "include it in the > > function parameters with a default value being the lookup (i.e. starting > in > > the outer scope), IOW "t=t" as I showed above. > > Yes. To be technically precise, there's no default argument involved, > and the call to the implicit function explicitly passes all the > arguments. > OK, and the idea is the same -- it's explicitly evaluated in the outer scope either way. > > The question is *when* we > > would do this. IIUC the PEP already does this if the "outer scope" is a > > class scope for any names that a simple static analysis shows are > references > > to variables in the class scope. > > Correct. > > > (I don't know exactly what this static > > analysis should do but it could be as simple as gathering all names that > are > > assigned to in the class, or alternatively all names assigned to before > the > > point where the comprehension occurs. We shouldn't be distracted by > dynamic > > definitions like `exec()` although we should perhaps be aware of `del`.) > > At the moment, it isn't aware of 'del'. The analysis is simple and > 100% static: If a name is in the table of names the class uses AND > it's in the table of names the comprehension uses, it gets passed as a > parameter. I don't want to try to be aware of del, because of this: > > class X: > x = 1 > if y: del x > print(x) > z = (q for q in x if q) > > If y is true, this will eagerly look up x using the same semantics in > both the print and the genexp (on construction, not when you iterate > over the genexp). If y is false, it'll still eagerly look up x, and > it'll still use the same semantics for print and the genexp (and it'll > find an 'x' in a surrounding scope). > > (The current implementation actually is a bit different from that. I'm > not sure whether it's possible to do it as simply as given without an > extra compilation pass. But it's close enough.) > Yeah, I threw 'del' in there mostly so we wouldn't get *too* confident. I see a fair amount of this: d = {} for x, y in blah(): d[x] = y del x, y > > My proposal is to extend this static analysis for certain loop control > > variables (any simple name assigned to in a for-clause in the > > comprehension), regardless of what kind of scope the outer scope is. If > the > > outer scope is a function we already know how to do this. If it's a > class we > > use the analysis referred to above. If the outer scope is the global > scope > > we have to do something new. I propose to use the same simple static > > analysis we use for class scopes. > > > > Furthermore I propose to *only* do this for the loop control variable(s) > of > > the outermost for-clause, since that's the only place where without all > this > > rigmarole we would have a clear difference in behavior with Python 3.7 in > > cases like [t for t in t]. Oh, and probably we only need to do this if > that > > loop control variable is also used as an expression in the iterable (so > we > > don't waste time doing any of this for e.g. [t for t in q]). > > Okay. Here's something that would be doable: > > If the name is written to within the comprehension, AND it is read > from in the outermost iterable, it is flagged early-bind. > OK, that's close enough to what I am looking for that I don't think it matters. > I'll have to try implementing that to be sure, but it should be > possible I think. It would cover a lot of cases, keeping them the same > as we currently have. > > > Since we now have once again introduced an exception for the outermost > loop > > control variable and the outermost iterable, we can consider doing this > only > > as a temporary measure. We could have a goal to eventually make [t for t > in > > t] fail, and in the meantime we would deprecate it -- e.g. in 3.8 a > silent > > deprecation, in 3.9 a noisy one, in 3.10 break it. Yes, that's a lot of > new > > static analysis for deprecating an edge case, but it seems reasonable to > > want to preserve backward compatibility when breaking this edge case > since > > it's likely not all that uncommon. Even if most occurrences are bad style > > written by lazy programmers, we should not break working code, if it is > > reasonable to expect that it's relied upon in real code. > > Fair enough. So the outermost iterable remains special for a short > while, with deprecation. > > I'll get onto the coding side of it during my Copious Free Time, > hopefully this week some time. > > Here's hoping! > Don't get your hopes up too high. A lot of respectable core devs have expressed a -1. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From phd at phdru.name Wed Apr 18 18:18:06 2018 From: phd at phdru.name (Oleg Broytman) Date: Thu, 19 Apr 2018 00:18:06 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <5AD60EC9.5010901@stoneleaf.us> <5AD66ABA.5090908@canterbury.ac.nz> <182f3235-7046-6ae0-6a95-61a0bcbbddb5@mrabarnett.plus.com> <254D07A5-6C04-4C96-98A2-11F5D776D0AA@python.org> Message-ID: <20180418221806.4nlmtsurwyu7osh3@phdru.name> On Wed, Apr 18, 2018 at 09:26:17PM +0000, "Gregory P. Smith" wrote: > On Wed, Apr 18, 2018 at 11:04 AM Barry Warsaw wrote: > > Since I can???t have ???>>??? or ???<>??? back, I propose ???=======???. > > 8 of course. to "match" what merge conflict markers look like. ;) Sorry for being pedantic, but git conflict markers are 7 in length. > -gps Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From solipsis at pitrou.net Wed Apr 18 18:46:41 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 19 Apr 2018 00:46:41 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions References: <40QRfQ2ytkzFqxb@mail.python.org> Message-ID: <20180419004641.16df43d7@fsol> Agreed with Paul and Steve. A lot of work seems to have gone into the PEP (congratulations for that), but still the feature brings little to no additional power to the language while making it more complex. -1 from me. Regards Antoine. On Tue, 17 Apr 2018 06:55:58 -0700 Steve Dower wrote: > Agree with Paul. The PEP is well thought out and well presented, but I really don?t think we need this in Python (and I say this as someone who uses it regularly in C/C#). > > -1 on the idea; no disrespect intended toward to people who did a lot of work on it. > > Top-posted from my Windows phone > > From: Paul Moore > Sent: Tuesday, April 17, 2018 6:31 > To: David Mertz > Cc: Nick Coghlan; Python-Dev > Subject: Re: [Python-Dev] PEP 572: Assignment Expressions > > On 17 April 2018 at 14:07, Paul Moore wrote: > > On 17 April 2018 at 14:01, David Mertz wrote: > >> Strongly agree with Nick that only simple name targets should be permitted > >> (at least initially). NONE of the motivating cases use more complex targets, > >> and allowing them encourages obscurity and code golf. > > > > I also agree. Originally I would have said why not allow them, it's a > > potentially useful generalisation. But Nick's examples pretty clearly > > demonstrate that there are a lot of unclear edge cases involved, and > > even though "prevent people writing ugly code" is explicitly stated as > > a non-goal in the PEP, that doesn't mean it's OK to allow an obvious > > bug magnet with no clear use cases. > > I should also point out that I remain -0 on this proposal (I'd already > said this on python-ideas, but I should probably repeat it here). For > me, the use cases are mostly marginal, and the major disadvantage is > in having two forms of assignment. Explaining to a beginner why we use > a := b in an expression, but a = b in a statement is going to be a > challenge. > > The fact that the PEP needs a section covering all the style guide > warnings we feel are needed seems like it's a warning bell, too. > > Paul > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org > > From songofacandy at gmail.com Thu Apr 19 02:39:21 2018 From: songofacandy at gmail.com (INADA Naoki) Date: Thu, 19 Apr 2018 15:39:21 +0900 Subject: [Python-Dev] Timing for removing legacy Unicode APIs deprecated by PEP 393 In-Reply-To: References: Message-ID: > > I suppose that many users will start porting to Python 3 only in 2020, after > 2.7 EOL. After that time we shouldn't support compatibility with 2.7 and can > start emitting deprecation warnings at runtime. After 1 or 2 releases after > that we can make corresponding public API always failing and remove private > API and data fields. > Python 3.8 is planned to be released at 2019-10-20. It's just before 2.7 EOL. My current thought is: * In 3.8, we make sure deprecated API emits warning (compile time if possible, runtime for others). * If the deprecation is adopted smoothly, drop them in 3.9 (Mid 2021). Otherwise, removal is postponed to 3.10 (Late 2023). > > There are other functions which expect that data is aligned to sizeof(long) > or 8 bytes. > > Siphash hashing is special because it is called not just for strings and > bytes, but for memoryview, which doesn't guarantee any alignment. > Oh, I'm sad about hear that... > Note that after removing the wchar_t* field the gap will not gone, because > the size of the structure should be a multiple of the alignment of the first > field (which is a pointer). Of course, we need hack for packing. -- INADA Naoki From greg.ewing at canterbury.ac.nz Thu Apr 19 02:42:11 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 19 Apr 2018 18:42:11 +1200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <182f3235-7046-6ae0-6a95-61a0bcbbddb5@mrabarnett.plus.com> References: <5AD60EC9.5010901@stoneleaf.us> <5AD66ABA.5090908@canterbury.ac.nz> <182f3235-7046-6ae0-6a95-61a0bcbbddb5@mrabarnett.plus.com> Message-ID: <5AD83A43.3030005@canterbury.ac.nz> MRAB wrote: > Some languages use '=' for assignment, others for equality, but do you > know of a language that uses ':=' for equality' or '==' for assignment? No, but the only sane reason to use "==" for equality testing seems to be if you're already using "=" for something else. So maybe we should just implement "from __future__ import pascal" and be done with. :-) -- Greg From ncoghlan at gmail.com Thu Apr 19 08:21:51 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 19 Apr 2018 22:21:51 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180418012048.GQ11616@ando.pearwood.info> Message-ID: On 19 April 2018 at 02:18, Guido van Rossum wrote: > On Wed, Apr 18, 2018 at 7:35 AM, Chris Angelico wrote: >> >> On Wed, Apr 18, 2018 at 11:58 PM, Guido van Rossum >> wrote: >> 2) Genexps will eagerly evaluate a lookup if it happens to be the same >> name as an internal iteration variable. > > I think we would have to specify this more precisely. > > Let's say by "eagerly evaluate a lookup" you mean "include it in the > function parameters with a default value being the lookup (i.e. starting in > the outer scope), IOW "t=t" as I showed above. The question is *when* we > would do this. IIUC the PEP already does this if the "outer scope" is a > class scope for any names that a simple static analysis shows are references > to variables in the class scope. (I don't know exactly what this static > analysis should do but it could be as simple as gathering all names that are > assigned to in the class, or alternatively all names assigned to before the > point where the comprehension occurs. We shouldn't be distracted by dynamic > definitions like `exec()` although we should perhaps be aware of `del`.) > > My proposal is to extend this static analysis for certain loop control > variables (any simple name assigned to in a for-clause in the > comprehension), regardless of what kind of scope the outer scope is. If the > outer scope is a function we already know how to do this. If it's a class we > use the analysis referred to above. If the outer scope is the global scope > we have to do something new. I propose to use the same simple static > analysis we use for class scopes. > > Furthermore I propose to *only* do this for the loop control variable(s) of > the outermost for-clause, since that's the only place where without all this > rigmarole we would have a clear difference in behavior with Python 3.7 in > cases like [t for t in t]. Oh, and probably we only need to do this if that > loop control variable is also used as an expression in the iterable (so we > don't waste time doing any of this for e.g. [t for t in q]). I'm not sure the symtable pass is currently clever enough to make these kinds of distinctions - it's pretty thoroughly block scope oriented. (Although I guess it *does* know enough now to treat the outermost comprehension as being part of the surrounding scope in terms of where names are referenced, so it might be feasible to adapt that logic to enable the eager binding you're describing). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Thu Apr 19 08:45:01 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 19 Apr 2018 22:45:01 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <40QRfq2w5LzFqsw@mail.python.org> Message-ID: On 19 April 2018 at 02:38, Brett Cannon wrote: > I'm also -1. > > I understand the usefulness of the construct in languages where block scopes > make having this kind of expression assignment in e.g. an 'if' guard useful. > But for Python and it's LGB scoping -- although I think we need to add an > "N" for "non-local" :) -- this is syntactic sugar and I don't see enough > wide benefit on top of the potential ugliness/misuse of it to warrant the > cognitive overhead of adding it. > > Although, as usual, Chris, great PEP! :) Aye, tremendous work on the PEP Chris - assignment expressions are an idea that has come up many, many times on python-ideas, and it's great to finally have a consolidated proposal that attempts to make the best case it can for the idea. Unfortunately, I still have to admit that even I'm -0 on the idea of actually adding it to the language (despite helping it to reach a plausibly acceptable state), primarily on "more than one way to do it" grounds: * assignment statements and top-level assignment expressions would be syntactically distinct, but semantically identical * for the if statement use case, pre-assignment of the re-used part of the condition expression already works fine * for the while loop use case, I think my discussion with Tim about tuple unpacking shows that the loop-and-a-half construct won't be going anywhere, so users would still end up needing to learn both forms (even for new code) The comprehension/genexp use case still seems like the most promising candidate for a reasonable justification for the extra complexity, but there's also a solid argument that if a comprehension is complex enough that the lack of assignment expressions is causing problems, then it's also complex enough that it's reasonable to require extracting the algorithm into a named generator function in order to access assignment statements. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From chris.barker at noaa.gov Thu Apr 19 18:59:59 2018 From: chris.barker at noaa.gov (Chris Barker) Date: Thu, 19 Apr 2018 15:59:59 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <40QRfq2w5LzFqsw@mail.python.org> Message-ID: On Thu, Apr 19, 2018 at 5:45 AM, Nick Coghlan wrote: > * for the while loop use case, I think my discussion with Tim about > tuple unpacking shows that the loop-and-a-half construct won't be > going anywhere, so users would still end up needing to learn both > forms (even for new code) > well, yes, but I've always found while awkward in the common case -- and wanted a better way (maybe an until?). using: while True: ... for a loop with a clear testable termination criteria always felt clunky. Anyone recall the "canonical" way to loop through the lines of a file before we had the file iterator? while True: line = fp.readline() if not line: break do_stuff_with(line) This is in fact, the only compeling use case for this to me -- maybe we could make a way to do an assign and test in a while loop enhancement just for that? And no, I have no idea what that would look like -- I'm mostly joking. So -1 on this -- though I agree -- great PEP Chris! And -1, not -0 because I think this really would add one more complication to a language that used to be so simple, and still is (and should be) used a first programming language. I teach a lot of newbies, and I'm realy not l;ooking forward to one more way to do assignement. It starts simpel enough: x = something binds the name, "x" the the object, 'something' is bound to (or a literal) Then we get into: - multiple assignment - tuple unpacking (oops, sequence unpacking), - augmented assignment - oh, that works different depending on whether the object is mutable We don't need any more. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph at grothesque.org Thu Apr 19 19:27:29 2018 From: christoph at grothesque.org (Christoph Groth) Date: Fri, 20 Apr 2018 01:27:29 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: Message-ID: <874lk6bwv2.fsf@grothesque.org> I'd like to break a lance for PEP 572. I read that in the bad old days Python used to have a "=" operator in expressions that had the meaning of today's "==". Perhaps there were other reasons, but this choice alone meant that assignment (that was using the same token) could not be made an expression. When the equality comparison operator was changed to "==", nothing changed for assignments which remained statements. Let's imagine, just for a moment, that Python is redesigned from scratch. Would it make sense to make assignment an expression? (For clarity an operator that is different from "=" would be used, say ":=".) I would say that the answer to the above question depends on whether assignment expressions can (a) fully replace assignment statements (so that both don't need to coexist) and (b) are useful and not too prone to abuse. I will try to answer both questions separately. ---------------- It seems to me that assignment expressions can be defined in a way that is fully analogous with the assignment statements of today's Python (and that consequently is equivalent in confusion potential): The "value" of any assignment statement from today's Python can be defined as the value that will be captured when "__value__ =" is prepended to that statement. So what should be the value of 'result' after the following snippet? a := list() b := list() result := (a[:] := b[:] := iter(range(3))) It seems to me that it should be the same as with today's result = a[:] = b[:] = iter(range(3)) Accepting this convention would mean that parens in assignment expressions would behave differently from C. For example result := (a[:] := (b[:] := iter(range(3)))) would produce a different 'result'. But I think that's OK, just like it's OK that the Python expression -1 < 0 < 1 is not equivalent to (-1 < 0) < 1 ---------------- The question remains whether assignment expressions are actually useful and not too prone to abuse? I think that the situation is similar to the "or" and "and" operators when used for their values. The snippet value = dic.get(key) or default could be replaced by an if clause, so there's no "one obvious way to do it" here. Still, many will agree that the above one-liner is nicer and more expressive. It wouldn't be possible if there were no boolean expressions and the use of "or" was limited to within if and while statements. In a similar way assignment expressions have many uses that are not absolutely needed but make code more expressive. The danger of abuse is also similar to "and" and "or" where it's also possible to write expressions that are difficult to understand. Here's an example from real life where the existence of assignment expressions would crucially simplify an API. I'm working on a library that has "learners" that are driven by "runners". The simplest possible synchronous runner looks like this: def sync_runner(learner, f, static_hint): while True: points = learner.get(static_hint) if not points: break learner.feed(f(points)) (Note that more advanced runners supply a "hint" that dynamically depends on currently available resources for example.) With assignment expressions the body of the above function could be simplified to while points := learner.get(static_hint): learner.feed(f(points)) making it crucially simpler. Using the learner API becomes so natural that a 'sync_runner()' function seems not even necessary. This API could be even adopted by other learner-providing libraries that want to support being driven by advanced asynchronous runners but would like to remain easily usable directly. Surely there are other uses of similar idioms. ---------------- Perhaps you agree with me that assignment should be an expression if Python was to be redesigned from scratch, but that is not going to happen. But couldn't Python simply introduce ":=" as described above and keep "=" for backwards compatibility? New users of Python could by taught to use ":=" everywhere. Old users could either convert their code base, or ignore the addition. There's no problem in mixing both old and new style code. Like with other expressive features, there's potential for confusion, but I think that it's limited. Wouldn't it be a pity not to liberate assignments from their boring statement existence? Cheers, Christoph From rosuav at gmail.com Thu Apr 19 23:10:00 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 20 Apr 2018 13:10:00 +1000 Subject: [Python-Dev] PEP 572: Now with 25% less reference implementation! Message-ID: Working on the reference implementation for PEP 572 is turning out to be a massive time sink, both on my personal schedule and on the PEP's discussion. I can't just hold off all discussion on a topic until I figure out whether something is possible or not, because that could take me several days, even a week or more. And considering the massive backlash against the proposal, it seems like a poor use of my time to try to prove that something's impossible, find that I don't know enough about grammar editing to be able to say anything more than "well, I couldn't do it, but someone else might be able to", and then try to resume the discussion with no more certainty than we had before. So here's the PEP again, simplified. I'm fairly sure it's just going to be another on a growing list of rejected PEPs to my name, and I'm done with trying to argue some of these points. Either the rules get simplified, or they don't. Trying to simplify the rules and maintain perfect backward compatibility is just making the rules even more complicated. PEP 572, if accepted, *will* change the behaviour of certain constructs inside comprehensions, mainly due to interactions with class scope that make no sense unless you know how they're implemented internally. The official tutorial pretends that comprehensions are "equivalent to" longhand: https://docs.python.org/3/tutorial/datastructures.html?highlight=equivalent#list-comprehensions https://docs.python.org/3/howto/functional.html?highlight=equivalent#generator-expressions-and-list-comprehensions and this is an inaccuracy for the sake of simplicity. PEP 572 will make this far more accurate; the only difference is that the comprehension is inside a function. Current semantics are far more bizarre than that. Do you want absolutely 100% backward compatibility? Then reject this PEP. Or better still, keep using Python 3.7, and don't upgrade to 3.8, in case something breaks. Do you want list comprehensions that make better sense? Then accept that some code will need to change, if it tried to use the same name in multiple scopes, or tried to use ancient Python 2 semantics with a yield expression in the outermost iterable. I'm pretty much ready for pronouncement. https://www.python.org/dev/peps/pep-0572/ ChrisA PEP: 572 Title: Assignment Expressions Author: Chris Angelico Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 28-Feb-2018 Python-Version: 3.8 Post-History: 28-Feb-2018, 02-Mar-2018, 23-Mar-2018, 04-Apr-2018, 17-Apr-2018 Abstract ======== This is a proposal for creating a way to assign to variables within an expression. Additionally, the precise scope of comprehensions is adjusted, to maintain consistency and follow expectations. Rationale ========= Naming the result of an expression is an important part of programming, allowing a descriptive name to be used in place of a longer expression, and permitting reuse. Currently, this feature is available only in statement form, making it unavailable in list comprehensions and other expression contexts. Merely introducing a way to assign as an expression would create bizarre edge cases around comprehensions, though, and to avoid the worst of the confusions, we change the definition of comprehensions, causing some edge cases to be interpreted differently, but maintaining the existing behaviour in the majority of situations. Syntax and semantics ==================== In any context where arbitrary Python expressions can be used, a **named expression** can appear. This is of the form ``target := expr`` where ``expr`` is any valid Python expression, and ``target`` is any valid assignment target. The value of such a named expression is the same as the incorporated expression, with the additional side-effect that the target is assigned that value:: # Handle a matched regex if (match := pattern.search(data)) is not None: ... # A more explicit alternative to the 2-arg form of iter() invocation while (value := read_next_item()) is not None: ... # Share a subexpression between a comprehension filter clause and its output filtered_data = [y for x in data if (y := f(x)) is not None] Differences from regular assignment statements ---------------------------------------------- Most importantly, since ``:=`` is an expression, it can be used in contexts where statements are illegal, including lambda functions and comprehensions. An assignment statement can assign to multiple targets, left-to-right:: x = y = z = 0 The equivalent assignment expression is parsed as separate binary operators, and is therefore processed right-to-left, as if it were spelled thus:: assert 0 == (x := (y := (z := 0))) Augmented assignment is not supported in expression form:: >>> x +:= 1 File "", line 1 x +:= 1 ^ SyntaxError: invalid syntax Otherwise, the semantics of assignment are identical in statement and expression forms. Alterations to comprehensions ----------------------------- The current behaviour of list/set/dict comprehensions and generator expressions has some edge cases that would behave strangely if an assignment expression were to be used. Therefore the proposed semantics are changed, removing the current edge cases, and instead altering their behaviour *only* in a class scope. As of Python 3.7, the outermost iterable of any comprehension is evaluated in the surrounding context, and then passed as an argument to the implicit function that evaluates the comprehension. Under this proposal, the entire body of the comprehension is evaluated in its implicit function. Names not assigned to within the comprehension are located in the surrounding scopes, as with normal lookups. As one special case, a comprehension at class scope will **eagerly bind** any name which is already defined in the class scope. A list comprehension can be unrolled into an equivalent function. With Python 3.7 semantics:: numbers = [x + y for x in range(3) for y in range(4)] # Is approximately equivalent to def (iterator): result = [] for x in iterator: for y in range(4): result.append(x + y) return result numbers = (iter(range(3))) Under the new semantics, this would instead be equivalent to:: def (): result = [] for x in range(3): for y in range(4): result.append(x + y) return result numbers = () When a class scope is involved, a naive transformation into a function would prevent name lookups (as the function would behave like a method):: class X: names = ["Fred", "Barney", "Joe"] prefix = "> " prefixed_names = [prefix + name for name in names] With Python 3.7 semantics, this will evaluate the outermost iterable at class scope, which will succeed; but it will evaluate everything else in a function:: class X: names = ["Fred", "Barney", "Joe"] prefix = "> " def (iterator): result = [] for name in iterator: result.append(prefix + name) return result prefixed_names = (iter(names)) The name ``prefix`` is thus searched for at global scope, ignoring the class name. Under the proposed semantics, this name will be eagerly bound; and the same early binding then handles the outermost iterable as well. The list comprehension is thus approximately equivalent to:: class X: names = ["Fred", "Barney", "Joe"] prefix = "> " def (names=names, prefix=prefix): result = [] for name in names: result.append(prefix + name) return result prefixed_names = () With list comprehensions, this is unlikely to cause any confusion. With generator expressions, this has the potential to affect behaviour, as the eager binding means that the name could be rebound between the creation of the genexp and the first call to ``next()``. It is, however, more closely aligned to normal expectations. The effect is ONLY seen with names that are looked up from class scope; global names (eg ``range()``) will still be late-bound as usual. One consequence of this change is that certain bugs in genexps will not be detected until the first call to ``next()``, where today they would be caught upon creation of the generator. Recommended use-cases ===================== Simplifying list comprehensions ------------------------------- A list comprehension can map and filter efficiently by capturing the condition:: results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0] Similarly, a subexpression can be reused within the main expression, by giving it a name on first use:: stuff = [[y := f(x), x/y] for x in range(5)] # There are a number of less obvious ways to spell this in current # versions of Python, such as: # Inline helper function stuff = [(lambda y: [y,x/y])(f(x)) for x in range(5)] # Extra 'for' loop - potentially could be optimized internally stuff = [[y, x/y] for x in range(5) for y in [f(x)]] # Using a mutable cache object (various forms possible) c = {} stuff = [[c.update(y=f(x)) or c['y'], x/c['y']] for x in range(5)] In all cases, the name is local to the comprehension; like iteration variables, it cannot leak out into the surrounding context. Capturing condition values -------------------------- Assignment expressions can be used to good effect in the header of an ``if`` or ``while`` statement:: # Proposed syntax while (command := input("> ")) != "quit": print("You entered:", command) # Capturing regular expression match objects # See, for instance, Lib/pydoc.py, which uses a multiline spelling # of this effect if match := re.search(pat, text): print("Found:", match.group(0)) # Reading socket data until an empty string is returned while data := sock.read(): print("Received data:", data) # Equivalent in current Python, not caring about function return value while input("> ") != "quit": print("You entered a command.") # To capture the return value in current Python demands a four-line # loop header. while True: command = input("> "); if command == "quit": break print("You entered:", command) Particularly with the ``while`` loop, this can remove the need to have an infinite loop, an assignment, and a condition. It also creates a smooth parallel between a loop which simply uses a function call as its condition, and one which uses that as its condition but also uses the actual value. Rejected alternative proposals ============================== Proposals broadly similar to this one have come up frequently on python-ideas. Below are a number of alternative syntaxes, some of them specific to comprehensions, which have been rejected in favour of the one given above. Alternative spellings --------------------- Broadly the same semantics as the current proposal, but spelled differently. 1. ``EXPR as NAME``:: stuff = [[f(x) as y, x/y] for x in range(5)] Since ``EXPR as NAME`` already has meaning in ``except`` and ``with`` statements (with different semantics), this would create unnecessary confusion or require special-casing (eg to forbid assignment within the headers of these statements). 2. ``EXPR -> NAME``:: stuff = [[f(x) -> y, x/y] for x in range(5)] This syntax is inspired by languages such as R and Haskell, and some programmable calculators. (Note that a left-facing arrow ``y <- f(x)`` is not possible in Python, as it would be interpreted as less-than and unary minus.) This syntax has a slight advantage over 'as' in that it does not conflict with ``with`` and ``except`` statements, but otherwise is equivalent. 3. Adorning statement-local names with a leading dot:: stuff = [[(f(x) as .y), x/.y] for x in range(5)] # with "as" stuff = [[(.y := f(x)), x/.y] for x in range(5)] # with ":=" This has the advantage that leaked usage can be readily detected, removing some forms of syntactic ambiguity. However, this would be the only place in Python where a variable's scope is encoded into its name, making refactoring harder. 4. Adding a ``where:`` to any statement to create local name bindings:: value = x**2 + 2*x where: x = spam(1, 4, 7, q) Execution order is inverted (the indented body is performed first, followed by the "header"). This requires a new keyword, unless an existing keyword is repurposed (most likely ``with:``). See PEP 3150 for prior discussion on this subject (with the proposed keyword being ``given:``). 5. ``TARGET from EXPR``:: stuff = [[y from f(x), x/y] for x in range(5)] This syntax has fewer conflicts than ``as`` does (conflicting only with the ``raise Exc from Exc`` notation), but is otherwise comparable to it. Instead of paralleling ``with expr as target:`` (which can be useful but can also be confusing), this has no parallels, but is evocative. Special-casing conditional statements ------------------------------------- One of the most popular use-cases is ``if`` and ``while`` statements. Instead of a more general solution, this proposal enhances the syntax of these two statements to add a means of capturing the compared value:: if re.search(pat, text) as match: print("Found:", match.group(0)) This works beautifully if and ONLY if the desired condition is based on the truthiness of the captured value. It is thus effective for specific use-cases (regex matches, socket reads that return `''` when done), and completely useless in more complicated cases (eg where the condition is ``f(x) < 0`` and you want to capture the value of ``f(x)``). It also has no benefit to list comprehensions. Advantages: No syntactic ambiguities. Disadvantages: Answers only a fraction of possible use-cases, even in ``if``/``while`` statements. Special-casing comprehensions ----------------------------- Another common use-case is comprehensions (list/set/dict, and genexps). As above, proposals have been made for comprehension-specific solutions. 1. ``where``, ``let``, or ``given``:: stuff = [(y, x/y) where y = f(x) for x in range(5)] stuff = [(y, x/y) let y = f(x) for x in range(5)] stuff = [(y, x/y) given y = f(x) for x in range(5)] This brings the subexpression to a location in between the 'for' loop and the expression. It introduces an additional language keyword, which creates conflicts. Of the three, ``where`` reads the most cleanly, but also has the greatest potential for conflict (eg SQLAlchemy and numpy have ``where`` methods, as does ``tkinter.dnd.Icon`` in the standard library). 2. ``with NAME = EXPR``:: stuff = [(y, x/y) with y = f(x) for x in range(5)] As above, but reusing the `with` keyword. Doesn't read too badly, and needs no additional language keyword. Is restricted to comprehensions, though, and cannot as easily be transformed into "longhand" for-loop syntax. Has the C problem that an equals sign in an expression can now create a name binding, rather than performing a comparison. Would raise the question of why "with NAME = EXPR:" cannot be used as a statement on its own. 3. ``with EXPR as NAME``:: stuff = [(y, x/y) with f(x) as y for x in range(5)] As per option 2, but using ``as`` rather than an equals sign. Aligns syntactically with other uses of ``as`` for name binding, but a simple transformation to for-loop longhand would create drastically different semantics; the meaning of ``with`` inside a comprehension would be completely different from the meaning as a stand-alone statement, while retaining identical syntax. Regardless of the spelling chosen, this introduces a stark difference between comprehensions and the equivalent unrolled long-hand form of the loop. It is no longer possible to unwrap the loop into statement form without reworking any name bindings. The only keyword that can be repurposed to this task is ``with``, thus giving it sneakily different semantics in a comprehension than in a statement; alternatively, a new keyword is needed, with all the costs therein. Lowering operator precedence ---------------------------- There are two logical precedences for the ``:=`` operator. Either it should bind as loosely as possible, as does statement-assignment; or it should bind more tightly than comparison operators. Placing its precedence between the comparison and arithmetic operators (to be precise: just lower than bitwise OR) allows most uses inside ``while`` and ``if`` conditions to be spelled without parentheses, as it is most likely that you wish to capture the value of something, then perform a comparison on it:: pos = -1 while pos := buffer.find(search_term, pos + 1) >= 0: ... Once find() returns -1, the loop terminates. If ``:=`` binds as loosely as ``=`` does, this would capture the result of the comparison (generally either ``True`` or ``False``), which is less useful. While this behaviour would be convenient in many situations, it is also harder to explain than "the := operator behaves just like the assignment statement", and as such, the precedence for ``:=`` has been made as close as possible to that of ``=``. Migration path ============== The semantic changes to list/set/dict comprehensions, and more so to generator expressions, may potentially require migration of code. In many cases, the changes simply make legal what used to raise an exception, but there are some edge cases that were previously legal and now are not, and a few corner cases with altered semantics. The Outermost Iterable ---------------------- As of Python 3.7, the outermost iterable in a comprehension is special: it is evaluated in the surrounding context, instead of inside the comprehension. Thus it is permitted to contain a ``yield`` expression, to use a name also used elsewhere, and to reference names from class scope. Also, in a genexp, the outermost iterable is pre-evaluated, but the rest of the code is not touched until the genexp is first iterated over. Class scope is now handled more generally (see above), but if other changes require the old behaviour, the iterable must be explicitly elevated from the comprehension:: # Python 3.7 def f(x): return [x for x in x if x] def g(): return [x for x in [(yield 1)]] # With PEP 572 def f(x): return [y for y in x if y] def g(): sent_item = (yield 1) return [x for x in [sent_item]] This more clearly shows that it is g(), not the comprehension, which is able to yield values (and is thus a generator function). The entire comprehension is consistently in a single scope. The following expressions would, in Python 3.7, raise exceptions immediately. With the removal of the outermost iterable's special casing, they are now equivalent to the most obvious longhand form:: gen = (x for x in rage(10)) # NameError gen = (x for x in 10) # TypeError (not iterable) gen = (x for x in range(1/0)) # ZeroDivisionError def (): for x in rage(10): yield x gen = () # No exception yet tng = next(gen) # NameError Open questions ============== Importing names into comprehensions ----------------------------------- A list comprehension can use and update local names, and they will retain their values from one iteration to another. It would be convenient to use this feature to create rolling or self-effecting data streams:: progressive_sums = [total := total + value for value in data] This will fail with UnboundLocalError due to ``total`` not being initalized. Simply initializing it outside of the comprehension is insufficient - unless the comprehension is in class scope:: class X: total = 0 progressive_sums = [total := total + value for value in data] At other scopes, it may be beneficial to have a way to fetch a value from the surrounding scope. Should this be automatic? Should it be controlled with a keyword? Hypothetically (and using no new keywords), this could be written:: total = 0 progressive_sums = [total := total + value import nonlocal total for value in data] Translated into longhand, this would become:: total = 0 def (total=total): result = [] for value in data: result.append(total := total + value) return result progressive_sums = () ie utilizing the same early-binding technique that is used at class scope. Frequently Raised Objections ============================ Why not just turn existing assignment into an expression? --------------------------------------------------------- C and its derivatives define the ``=`` operator as an expression, rather than a statement as is Python's way. This allows assignments in more contexts, including contexts where comparisons are more common. The syntactic similarity between ``if (x == y)`` and ``if (x = y)`` belies their drastically different semantics. Thus this proposal uses ``:=`` to clarify the distinction. This could be used to create ugly code! --------------------------------------- So can anything else. This is a tool, and it is up to the programmer to use it where it makes sense, and not use it where superior constructs can be used. With assignment expressions, why bother with assignment statements? ------------------------------------------------------------------- The two forms have different flexibilities. The ``:=`` operator can be used inside a larger expression; the ``=`` statement can be augmented to ``+=`` and its friends. The assignment statement is a clear declaration of intent: this value is to be assigned to this target, and that's it. Why not use a sublocal scope and prevent namespace pollution? ------------------------------------------------------------- Previous revisions of this proposal involved sublocal scope (restricted to a single statement), preventing name leakage and namespace pollution. While a definite advantage in a number of situations, this increases complexity in many others, and the costs are not justified by the benefits. In the interests of language simplicity, the name bindings created here are exactly equivalent to any other name bindings, including that usage at class or module scope will create externally-visible names. This is no different from ``for`` loops or other constructs, and can be solved the same way: ``del`` the name once it is no longer needed, or prefix it with an underscore. Names bound within a comprehension are local to that comprehension, even in the outermost iterable, and can thus be used freely without polluting the surrounding namespace. (The author wishes to thank Guido van Rossum and Christoph Groth for their suggestions to move the proposal in this direction. [2]_) Style guide recommendations =========================== As this adds another way to spell some of the same effects as can already be done, it is worth noting a few broad recommendations. These could be included in PEP 8 and/or other style guides. 1. If either assignment statements or assignment expressions can be used, prefer statements; they are a clear declaration of intent. 2. If using assignment expressions would lead to ambiguity about execution order, restructure it to use statements instead. Acknowledgements ================ The author wishes to thank Guido van Rossum and Nick Coghlan for their considerable contributions to this proposal, and to members of the core-mentorship mailing list for assistance with implementation. References ========== .. [1] Proof of concept / reference implementation (https://github.com/Rosuav/cpython/tree/assignment-expressions) .. [2] Pivotal post regarding inline assignment semantics (https://mail.python.org/pipermail/python-ideas/2018-March/049409.html) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From turnbull.stephen.fw at u.tsukuba.ac.jp Thu Apr 19 23:30:20 2018 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Fri, 20 Apr 2018 12:30:20 +0900 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <874lk6bwv2.fsf@grothesque.org> References: <874lk6bwv2.fsf@grothesque.org> Message-ID: <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> Christoph Groth writes: > Wouldn't it be a pity not to liberate assignments from their boring > statement existence? Maybe not. While it would be nice to solve the loop-and-a-half "problem" and the loop variable initialization "problem" (not everyone agrees these are even problems, especially now that we have comprehensions and generator expressions), as a matter of taste I like the fact that this particular class of side effects is given weighty statement syntax rather than more lightweight expression syntax. That is, I find statement syntax more readable. Steve -- Associate Professor Division of Policy and Planning Science http://turnbull/sk.tsukuba.ac.jp/ Faculty of Systems and Information Email: turnbull at sk.tsukuba.ac.jp University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN From damalinov at gmail.com Fri Apr 20 00:45:37 2018 From: damalinov at gmail.com (Dmitry Malinovsky) Date: Fri, 20 Apr 2018 10:45:37 +0600 Subject: [Python-Dev] PEP 572: Now with 25% less reference implementation! In-Reply-To: References: Message-ID: Hello Chris, and thank you for working on this PEP! What do you think about using variable type hints with this syntax? I tried to search through python-dev and couldn't find a single post discussing that question. If I missed it somehow, could you please include its conclusions into the PEP? For instance, as I understand now the parser will fail on this snippet: while data: bytes := stream.read(): print("Received data:", data) Do brackets help? while (data: bytes := stream.read()): print("Received data:", data) IIUC, in 3.7 It is invalid syntax to specify a type hint for a for loop item; should brackets help? Currently they don't: Python 3.7.0b3+ (heads/3.7:7dcfd6c, Mar 30 2018, 21:30:34) [Clang 9.0.0 (clang-900.0.39.2)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> for (x: int) in [1,2,3]: File "", line 1 for (x: int) in [1,2,3]: ^ SyntaxError: invalid syntax Thanks, > On 20 Apr 2018, at 09:10, Chris Angelico wrote: > > Working on the reference implementation for PEP 572 is turning out to > be a massive time sink, both on my personal schedule and on the PEP's > discussion. I can't just hold off all discussion on a topic until I > figure out whether something is possible or not, because that could > take me several days, even a week or more. And considering the massive > backlash against the proposal, it seems like a poor use of my time to > try to prove that something's impossible, find that I don't know > enough about grammar editing to be able to say anything more than > "well, I couldn't do it, but someone else might be able to", and then > try to resume the discussion with no more certainty than we had > before. > > So here's the PEP again, simplified. I'm fairly sure it's just going > to be another on a growing list of rejected PEPs to my name, and I'm > done with trying to argue some of these points. Either the rules get > simplified, or they don't. Trying to simplify the rules and maintain > perfect backward compatibility is just making the rules even more > complicated. > > PEP 572, if accepted, *will* change the behaviour of certain > constructs inside comprehensions, mainly due to interactions with > class scope that make no sense unless you know how they're implemented > internally. The official tutorial pretends that comprehensions are > "equivalent to" longhand: > > https://docs.python.org/3/tutorial/datastructures.html?highlight=equivalent#list-comprehensions > https://docs.python.org/3/howto/functional.html?highlight=equivalent#generator-expressions-and-list-comprehensions > > and this is an inaccuracy for the sake of simplicity. PEP 572 will > make this far more accurate; the only difference is that the > comprehension is inside a function. Current semantics are far more > bizarre than that. > > Do you want absolutely 100% backward compatibility? Then reject this > PEP. Or better still, keep using Python 3.7, and don't upgrade to 3.8, > in case something breaks. Do you want list comprehensions that make > better sense? Then accept that some code will need to change, if it > tried to use the same name in multiple scopes, or tried to use ancient > Python 2 semantics with a yield expression in the outermost iterable. > > I'm pretty much ready for pronouncement. > > https://www.python.org/dev/peps/pep-0572/ > > ChrisA > > PEP: 572 > Title: Assignment Expressions > Author: Chris Angelico > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 28-Feb-2018 > Python-Version: 3.8 > Post-History: 28-Feb-2018, 02-Mar-2018, 23-Mar-2018, 04-Apr-2018, 17-Apr-2018 > > > Abstract > ======== > > This is a proposal for creating a way to assign to variables within an > expression. Additionally, the precise scope of comprehensions is adjusted, to > maintain consistency and follow expectations. > > > Rationale > ========= > > Naming the result of an expression is an important part of programming, > allowing a descriptive name to be used in place of a longer expression, > and permitting reuse. Currently, this feature is available only in > statement form, making it unavailable in list comprehensions and other > expression contexts. Merely introducing a way to assign as an expression > would create bizarre edge cases around comprehensions, though, and to avoid > the worst of the confusions, we change the definition of comprehensions, > causing some edge cases to be interpreted differently, but maintaining the > existing behaviour in the majority of situations. > > > Syntax and semantics > ==================== > > In any context where arbitrary Python expressions can be used, a **named > expression** can appear. This is of the form ``target := expr`` where > ``expr`` is any valid Python expression, and ``target`` is any valid > assignment target. > > The value of such a named expression is the same as the incorporated > expression, with the additional side-effect that the target is assigned > that value:: > > # Handle a matched regex > if (match := pattern.search(data)) is not None: > ... > > # A more explicit alternative to the 2-arg form of iter() invocation > while (value := read_next_item()) is not None: > ... > > # Share a subexpression between a comprehension filter clause and its output > filtered_data = [y for x in data if (y := f(x)) is not None] > > > Differences from regular assignment statements > ---------------------------------------------- > > Most importantly, since ``:=`` is an expression, it can be used in contexts > where statements are illegal, including lambda functions and comprehensions. > > An assignment statement can assign to multiple targets, left-to-right:: > > x = y = z = 0 > > The equivalent assignment expression is parsed as separate binary operators, > and is therefore processed right-to-left, as if it were spelled thus:: > > assert 0 == (x := (y := (z := 0))) > > Augmented assignment is not supported in expression form:: > >>>> x +:= 1 > File "", line 1 > x +:= 1 > ^ > SyntaxError: invalid syntax > > Otherwise, the semantics of assignment are identical in statement and > expression forms. > > > Alterations to comprehensions > ----------------------------- > > The current behaviour of list/set/dict comprehensions and generator > expressions has some edge cases that would behave strangely if an assignment > expression were to be used. Therefore the proposed semantics are changed, > removing the current edge cases, and instead altering their behaviour *only* > in a class scope. > > As of Python 3.7, the outermost iterable of any comprehension is evaluated > in the surrounding context, and then passed as an argument to the implicit > function that evaluates the comprehension. > > Under this proposal, the entire body of the comprehension is evaluated in > its implicit function. Names not assigned to within the comprehension are > located in the surrounding scopes, as with normal lookups. As one special > case, a comprehension at class scope will **eagerly bind** any name which > is already defined in the class scope. > > A list comprehension can be unrolled into an equivalent function. With > Python 3.7 semantics:: > > numbers = [x + y for x in range(3) for y in range(4)] > # Is approximately equivalent to > def (iterator): > result = [] > for x in iterator: > for y in range(4): > result.append(x + y) > return result > numbers = (iter(range(3))) > > Under the new semantics, this would instead be equivalent to:: > > def (): > result = [] > for x in range(3): > for y in range(4): > result.append(x + y) > return result > numbers = () > > When a class scope is involved, a naive transformation into a function would > prevent name lookups (as the function would behave like a method):: > > class X: > names = ["Fred", "Barney", "Joe"] > prefix = "> " > prefixed_names = [prefix + name for name in names] > > With Python 3.7 semantics, this will evaluate the outermost iterable at class > scope, which will succeed; but it will evaluate everything else in a function:: > > class X: > names = ["Fred", "Barney", "Joe"] > prefix = "> " > def (iterator): > result = [] > for name in iterator: > result.append(prefix + name) > return result > prefixed_names = (iter(names)) > > The name ``prefix`` is thus searched for at global scope, ignoring the class > name. Under the proposed semantics, this name will be eagerly bound; and the > same early binding then handles the outermost iterable as well. The list > comprehension is thus approximately equivalent to:: > > class X: > names = ["Fred", "Barney", "Joe"] > prefix = "> " > def (names=names, prefix=prefix): > result = [] > for name in names: > result.append(prefix + name) > return result > prefixed_names = () > > With list comprehensions, this is unlikely to cause any confusion. With > generator expressions, this has the potential to affect behaviour, as the > eager binding means that the name could be rebound between the creation of > the genexp and the first call to ``next()``. It is, however, more closely > aligned to normal expectations. The effect is ONLY seen with names that > are looked up from class scope; global names (eg ``range()``) will still > be late-bound as usual. > > One consequence of this change is that certain bugs in genexps will not > be detected until the first call to ``next()``, where today they would be > caught upon creation of the generator. > > > Recommended use-cases > ===================== > > Simplifying list comprehensions > ------------------------------- > > A list comprehension can map and filter efficiently by capturing > the condition:: > > results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0] > > Similarly, a subexpression can be reused within the main expression, by > giving it a name on first use:: > > stuff = [[y := f(x), x/y] for x in range(5)] > > # There are a number of less obvious ways to spell this in current > # versions of Python, such as: > > # Inline helper function > stuff = [(lambda y: [y,x/y])(f(x)) for x in range(5)] > > # Extra 'for' loop - potentially could be optimized internally > stuff = [[y, x/y] for x in range(5) for y in [f(x)]] > > # Using a mutable cache object (various forms possible) > c = {} > stuff = [[c.update(y=f(x)) or c['y'], x/c['y']] for x in range(5)] > > In all cases, the name is local to the comprehension; like iteration variables, > it cannot leak out into the surrounding context. > > > Capturing condition values > -------------------------- > > Assignment expressions can be used to good effect in the header of > an ``if`` or ``while`` statement:: > > # Proposed syntax > while (command := input("> ")) != "quit": > print("You entered:", command) > > # Capturing regular expression match objects > # See, for instance, Lib/pydoc.py, which uses a multiline spelling > # of this effect > if match := re.search(pat, text): > print("Found:", match.group(0)) > > # Reading socket data until an empty string is returned > while data := sock.read(): > print("Received data:", data) > > # Equivalent in current Python, not caring about function return value > while input("> ") != "quit": > print("You entered a command.") > > # To capture the return value in current Python demands a four-line > # loop header. > while True: > command = input("> "); > if command == "quit": > break > print("You entered:", command) > > Particularly with the ``while`` loop, this can remove the need to have an > infinite loop, an assignment, and a condition. It also creates a smooth > parallel between a loop which simply uses a function call as its condition, > and one which uses that as its condition but also uses the actual value. > > > Rejected alternative proposals > ============================== > > Proposals broadly similar to this one have come up frequently on python-ideas. > Below are a number of alternative syntaxes, some of them specific to > comprehensions, which have been rejected in favour of the one given above. > > > Alternative spellings > --------------------- > > Broadly the same semantics as the current proposal, but spelled differently. > > 1. ``EXPR as NAME``:: > > stuff = [[f(x) as y, x/y] for x in range(5)] > > Since ``EXPR as NAME`` already has meaning in ``except`` and ``with`` > statements (with different semantics), this would create unnecessary > confusion or require special-casing (eg to forbid assignment within the > headers of these statements). > > 2. ``EXPR -> NAME``:: > > stuff = [[f(x) -> y, x/y] for x in range(5)] > > This syntax is inspired by languages such as R and Haskell, and some > programmable calculators. (Note that a left-facing arrow ``y <- f(x)`` is > not possible in Python, as it would be interpreted as less-than and unary > minus.) This syntax has a slight advantage over 'as' in that it does not > conflict with ``with`` and ``except`` statements, but otherwise is > equivalent. > > 3. Adorning statement-local names with a leading dot:: > > stuff = [[(f(x) as .y), x/.y] for x in range(5)] # with "as" > stuff = [[(.y := f(x)), x/.y] for x in range(5)] # with ":=" > > This has the advantage that leaked usage can be readily detected, removing > some forms of syntactic ambiguity. However, this would be the only place > in Python where a variable's scope is encoded into its name, making > refactoring harder. > > 4. Adding a ``where:`` to any statement to create local name bindings:: > > value = x**2 + 2*x where: > x = spam(1, 4, 7, q) > > Execution order is inverted (the indented body is performed first, followed > by the "header"). This requires a new keyword, unless an existing keyword > is repurposed (most likely ``with:``). See PEP 3150 for prior discussion > on this subject (with the proposed keyword being ``given:``). > > 5. ``TARGET from EXPR``:: > > stuff = [[y from f(x), x/y] for x in range(5)] > > This syntax has fewer conflicts than ``as`` does (conflicting only with the > ``raise Exc from Exc`` notation), but is otherwise comparable to it. Instead > of paralleling ``with expr as target:`` (which can be useful but can also be > confusing), this has no parallels, but is evocative. > > > Special-casing conditional statements > ------------------------------------- > > One of the most popular use-cases is ``if`` and ``while`` statements. Instead > of a more general solution, this proposal enhances the syntax of these two > statements to add a means of capturing the compared value:: > > if re.search(pat, text) as match: > print("Found:", match.group(0)) > > This works beautifully if and ONLY if the desired condition is based on the > truthiness of the captured value. It is thus effective for specific > use-cases (regex matches, socket reads that return `''` when done), and > completely useless in more complicated cases (eg where the condition is > ``f(x) < 0`` and you want to capture the value of ``f(x)``). It also has > no benefit to list comprehensions. > > Advantages: No syntactic ambiguities. Disadvantages: Answers only a fraction > of possible use-cases, even in ``if``/``while`` statements. > > > Special-casing comprehensions > ----------------------------- > > Another common use-case is comprehensions (list/set/dict, and genexps). As > above, proposals have been made for comprehension-specific solutions. > > 1. ``where``, ``let``, or ``given``:: > > stuff = [(y, x/y) where y = f(x) for x in range(5)] > stuff = [(y, x/y) let y = f(x) for x in range(5)] > stuff = [(y, x/y) given y = f(x) for x in range(5)] > > This brings the subexpression to a location in between the 'for' loop and > the expression. It introduces an additional language keyword, which creates > conflicts. Of the three, ``where`` reads the most cleanly, but also has the > greatest potential for conflict (eg SQLAlchemy and numpy have ``where`` > methods, as does ``tkinter.dnd.Icon`` in the standard library). > > 2. ``with NAME = EXPR``:: > > stuff = [(y, x/y) with y = f(x) for x in range(5)] > > As above, but reusing the `with` keyword. Doesn't read too badly, and needs > no additional language keyword. Is restricted to comprehensions, though, > and cannot as easily be transformed into "longhand" for-loop syntax. Has > the C problem that an equals sign in an expression can now create a name > binding, rather than performing a comparison. Would raise the question of > why "with NAME = EXPR:" cannot be used as a statement on its own. > > 3. ``with EXPR as NAME``:: > > stuff = [(y, x/y) with f(x) as y for x in range(5)] > > As per option 2, but using ``as`` rather than an equals sign. Aligns > syntactically with other uses of ``as`` for name binding, but a simple > transformation to for-loop longhand would create drastically different > semantics; the meaning of ``with`` inside a comprehension would be > completely different from the meaning as a stand-alone statement, while > retaining identical syntax. > > Regardless of the spelling chosen, this introduces a stark difference between > comprehensions and the equivalent unrolled long-hand form of the loop. It is > no longer possible to unwrap the loop into statement form without reworking > any name bindings. The only keyword that can be repurposed to this task is > ``with``, thus giving it sneakily different semantics in a comprehension than > in a statement; alternatively, a new keyword is needed, with all the costs > therein. > > > Lowering operator precedence > ---------------------------- > > There are two logical precedences for the ``:=`` operator. Either it should > bind as loosely as possible, as does statement-assignment; or it should bind > more tightly than comparison operators. Placing its precedence between the > comparison and arithmetic operators (to be precise: just lower than bitwise > OR) allows most uses inside ``while`` and ``if`` conditions to be spelled > without parentheses, as it is most likely that you wish to capture the value > of something, then perform a comparison on it:: > > pos = -1 > while pos := buffer.find(search_term, pos + 1) >= 0: > ... > > Once find() returns -1, the loop terminates. If ``:=`` binds as loosely as > ``=`` does, this would capture the result of the comparison (generally either > ``True`` or ``False``), which is less useful. > > While this behaviour would be convenient in many situations, it is also harder > to explain than "the := operator behaves just like the assignment statement", > and as such, the precedence for ``:=`` has been made as close as possible to > that of ``=``. > > > Migration path > ============== > > The semantic changes to list/set/dict comprehensions, and more so to generator > expressions, may potentially require migration of code. In many cases, the > changes simply make legal what used to raise an exception, but there are some > edge cases that were previously legal and now are not, and a few corner cases > with altered semantics. > > > The Outermost Iterable > ---------------------- > > As of Python 3.7, the outermost iterable in a comprehension is special: it is > evaluated in the surrounding context, instead of inside the comprehension. > Thus it is permitted to contain a ``yield`` expression, to use a name also > used elsewhere, and to reference names from class scope. Also, in a genexp, > the outermost iterable is pre-evaluated, but the rest of the code is not > touched until the genexp is first iterated over. Class scope is now handled > more generally (see above), but if other changes require the old behaviour, > the iterable must be explicitly elevated from the comprehension:: > > # Python 3.7 > def f(x): > return [x for x in x if x] > def g(): > return [x for x in [(yield 1)]] > # With PEP 572 > def f(x): > return [y for y in x if y] > def g(): > sent_item = (yield 1) > return [x for x in [sent_item]] > > This more clearly shows that it is g(), not the comprehension, which is able > to yield values (and is thus a generator function). The entire comprehension > is consistently in a single scope. > > The following expressions would, in Python 3.7, raise exceptions immediately. > With the removal of the outermost iterable's special casing, they are now > equivalent to the most obvious longhand form:: > > gen = (x for x in rage(10)) # NameError > gen = (x for x in 10) # TypeError (not iterable) > gen = (x for x in range(1/0)) # ZeroDivisionError > > def (): > for x in rage(10): > yield x > gen = () # No exception yet > tng = next(gen) # NameError > > > Open questions > ============== > > Importing names into comprehensions > ----------------------------------- > > A list comprehension can use and update local names, and they will retain > their values from one iteration to another. It would be convenient to use > this feature to create rolling or self-effecting data streams:: > > progressive_sums = [total := total + value for value in data] > > This will fail with UnboundLocalError due to ``total`` not being initalized. > Simply initializing it outside of the comprehension is insufficient - unless > the comprehension is in class scope:: > > class X: > total = 0 > progressive_sums = [total := total + value for value in data] > > At other scopes, it may be beneficial to have a way to fetch a value from the > surrounding scope. Should this be automatic? Should it be controlled with a > keyword? Hypothetically (and using no new keywords), this could be written:: > > total = 0 > progressive_sums = [total := total + value > import nonlocal total > for value in data] > > Translated into longhand, this would become:: > > total = 0 > def (total=total): > result = [] > for value in data: > result.append(total := total + value) > return result > progressive_sums = () > > ie utilizing the same early-binding technique that is used at class scope. > > > Frequently Raised Objections > ============================ > > Why not just turn existing assignment into an expression? > --------------------------------------------------------- > > C and its derivatives define the ``=`` operator as an expression, rather than > a statement as is Python's way. This allows assignments in more contexts, > including contexts where comparisons are more common. The syntactic similarity > between ``if (x == y)`` and ``if (x = y)`` belies their drastically different > semantics. Thus this proposal uses ``:=`` to clarify the distinction. > > > This could be used to create ugly code! > --------------------------------------- > > So can anything else. This is a tool, and it is up to the programmer to use it > where it makes sense, and not use it where superior constructs can be used. > > > With assignment expressions, why bother with assignment statements? > ------------------------------------------------------------------- > > The two forms have different flexibilities. The ``:=`` operator can be used > inside a larger expression; the ``=`` statement can be augmented to ``+=`` and > its friends. The assignment statement is a clear declaration of intent: this > value is to be assigned to this target, and that's it. > > > Why not use a sublocal scope and prevent namespace pollution? > ------------------------------------------------------------- > > Previous revisions of this proposal involved sublocal scope (restricted to a > single statement), preventing name leakage and namespace pollution. While a > definite advantage in a number of situations, this increases complexity in > many others, and the costs are not justified by the benefits. In the interests > of language simplicity, the name bindings created here are exactly equivalent > to any other name bindings, including that usage at class or module scope will > create externally-visible names. This is no different from ``for`` loops or > other constructs, and can be solved the same way: ``del`` the name once it is > no longer needed, or prefix it with an underscore. > > Names bound within a comprehension are local to that comprehension, even in > the outermost iterable, and can thus be used freely without polluting the > surrounding namespace. > > (The author wishes to thank Guido van Rossum and Christoph Groth for their > suggestions to move the proposal in this direction. [2]_) > > > Style guide recommendations > =========================== > > As this adds another way to spell some of the same effects as can already be > done, it is worth noting a few broad recommendations. These could be included > in PEP 8 and/or other style guides. > > 1. If either assignment statements or assignment expressions can be > used, prefer statements; they are a clear declaration of intent. > > 2. If using assignment expressions would lead to ambiguity about > execution order, restructure it to use statements instead. > > > Acknowledgements > ================ > > The author wishes to thank Guido van Rossum and Nick Coghlan for their > considerable contributions to this proposal, and to members of the > core-mentorship mailing list for assistance with implementation. > > > References > ========== > > .. [1] Proof of concept / reference implementation > (https://github.com/Rosuav/cpython/tree/assignment-expressions) > .. [2] Pivotal post regarding inline assignment semantics > (https://mail.python.org/pipermail/python-ideas/2018-March/049409.html) > > > Copyright > ========= > > This document has been placed in the public domain. > > > .. > Local Variables: > mode: indented-text > indent-tabs-mode: nil > sentence-end-double-space: t > fill-column: 70 > coding: utf-8 > End: > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/damalinov%40gmail.com From chris.barker at noaa.gov Fri Apr 20 01:08:48 2018 From: chris.barker at noaa.gov (Chris Barker - NOAA Federal) Date: Thu, 19 Apr 2018 22:08:48 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <874lk6bwv2.fsf@grothesque.org> References: <874lk6bwv2.fsf@grothesque.org> Message-ID: > On Apr 19, 2018, at 4:27 PM, Christoph Groth wrote: > def sync_runner(learner, f, static_hint): > while True: > points = learner.get(static_hint) > if not points: > break > learner.feed(f(points)) > > > > With assignment expressions the body of the above function could be > simplified to > > while points := learner.get(static_hint): > learner.feed(f(points)) > > making it crucially simpler. Kinda supports my assertion that what we really want is a different while loop. Would it be ridiculous if := only worked in a while statement? -CHB From rosuav at gmail.com Fri Apr 20 02:31:32 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 20 Apr 2018 16:31:32 +1000 Subject: [Python-Dev] PEP 572: Now with 25% less reference implementation! In-Reply-To: References: Message-ID: On Fri, Apr 20, 2018 at 2:45 PM, Dmitry Malinovsky wrote: > Hello Chris, and thank you for working on this PEP! > > What do you think about using variable type hints with this syntax? > I tried to search through python-dev and couldn't find a single post > discussing that question. > If I missed it somehow, could you please include its conclusions into the PEP? I'm ignoring them for the sake of the PEP, because it'd complicate the grammar for little benefit. If someone wants to add an enhancement later, that's fine; but the proposal can stand without it, and with it, it'll make for even more noise in a line full of colons. > For instance, as I understand now the parser will fail on this snippet: > > while data: bytes := stream.read(): > print("Received data:", data) > > Do brackets help? > > while (data: bytes := stream.read()): > print("Received data:", data) > > IIUC, in 3.7 It is invalid syntax to specify a type hint for a for loop item; > should brackets help? Currently they don't: > > Python 3.7.0b3+ (heads/3.7:7dcfd6c, Mar 30 2018, 21:30:34) > [Clang 9.0.0 (clang-900.0.39.2)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>> for (x: int) in [1,2,3]: > File "", line 1 > for (x: int) in [1,2,3]: > ^ > SyntaxError: invalid syntax And that's another good reason not to bother, at least for now. I'm not sure whether you can use a Py2-style type hint comment on a for loop, but if so, you should also be able to do it on a while loop or anything. Or, of course, you can just annotate the variable before the loop, if you want to. ChrisA From rob at sixty-north.com Fri Apr 20 02:32:56 2018 From: rob at sixty-north.com (Robert Smallshire) Date: Fri, 20 Apr 2018 08:32:56 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> Message-ID: If you restrict the idea to 'if' and 'while', Why not render this using the existing 'as' form for binding names, already used with 'except' and 'with': while learner.get(static_hint) as points: learner.feed(f(points)) The equivalent for 'if' helps with the regex matching case: if re.match(r"...") as m: print(m.group(1)) I considered proposing these two forms in a PEP a few years ago, but never got around to it. To my eye, they fit syntactically into the language as-is, without introducing new symbols, operators or keywords, are consistent with existing usage, and address two relatively common causes of displeasing Python code. Robert -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Fri Apr 20 02:46:05 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 20 Apr 2018 16:46:05 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> Message-ID: On Fri, Apr 20, 2018 at 1:30 PM, Stephen J. Turnbull wrote: > Christoph Groth writes: > > > Wouldn't it be a pity not to liberate assignments from their boring > > statement existence? > > Maybe not. While it would be nice to solve the loop-and-a-half > "problem" and the loop variable initialization "problem" (not everyone > agrees these are even problems, especially now that we have > comprehensions and generator expressions), as a matter of taste I like > the fact that this particular class of side effects is given weighty > statement syntax rather than more lightweight expression syntax. > > That is, I find statement syntax more readable. > If you've read the PEP, you'll see that it encourages the use of assignment statements whereever possible. If statement syntax is generally more readable, by all means, use it. That doesn't mean there aren't situations where the expression syntax is FAR more readable. Tell me, is this "more readable" than a loop with an actual condition in it? def sqrt(n): guess, nextguess = 1, n while True: if math.isclose(guess, nextguess): return guess guess = nextguess nextguess = n / guess Readable doesn't mean "corresponds closely to its disassembly", despite the way many people throw the word around. It also doesn't mean "code I like", as opposed to "code I don't like". The words for those concepts are "strongly typed" and "dynamically typed", as have been demonstrated through MANY online discussions. (But I digress.) Readable code is code which expresses an algorithm, expresses the programmer's intent. It adequately demonstrates something at a *higher* abstraction level. Does the algorithm demonstrated here include an infinite loop? No? Then it shouldn't have "while True:" in it. Now, this is a pretty obvious example. I deliberately wrote it so you could simply lift the condition straight into the while header. And I hope that everyone here agrees that this would be an improvement: def sqrt(n): guess, nextguess = 1, n while not math.isclose(guess, nextguess): guess = nextguess nextguess = n / guess return guess But what if the condition were more complicated? def read_document(file): doc = "" while (token := file.get_next_token()) != "END": doc += token return doc The loop condition is "while the token is not END", or "while get_next_token() doesn't return END", depending on your point of view. Is it "more readable" to put that condition into the while header, or to use an infinite loop and a break statement, or to duplicate a line of code before the loop and at the bottom of the loop? Which one best expresses the programmer's intention? ChrisA From rosuav at gmail.com Fri Apr 20 02:52:06 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 20 Apr 2018 16:52:06 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> Message-ID: On Fri, Apr 20, 2018 at 4:32 PM, Robert Smallshire wrote: > If you restrict the idea to 'if' and 'while', Why not render this using the > existing 'as' form for binding names, already used with 'except' and 'with': > > while learner.get(static_hint) as points: > learner.feed(f(points)) > > The equivalent for 'if' helps with the regex matching case: > > if re.match(r"...") as m: > print(m.group(1)) > > I considered proposing these two forms in a PEP a few years ago, but never > got around to it. To my eye, they fit syntactically into the language as-is, > without introducing new symbols, operators or keywords, are consistent with > existing usage, and address two relatively common causes of displeasing > Python code. And are limited to conditions that check the truthiness/falsiness of the value you care about. So that works for re.match, but not for anything that might return -1 (a lot of C APIs do that, so if you're working with a thin wrapper, that might be all you get), and it'll encourage people to use this form when "is not None" would be more appropriate (setting up for a failure if ever the API returned a falsey value), etc. It's similar if you use iter(func, None) - it's actually doing an equality check, not an identity check, even though a longhand form would be better written with "is not None". Also, are you assuming that this is binding to a name, or can it assign to other targets? if re.match(...) as m[3]: ... HINT: Saying "it should do whatever except and with do" won't answer the question. Give it a try if you don't believe me. :) ChrisA From p.f.moore at gmail.com Fri Apr 20 04:32:34 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 20 Apr 2018 09:32:34 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> Message-ID: On 20 April 2018 at 07:46, Chris Angelico wrote: > On Fri, Apr 20, 2018 at 1:30 PM, Stephen J. Turnbull > wrote: >> Christoph Groth writes: >> >> > Wouldn't it be a pity not to liberate assignments from their boring >> > statement existence? >> >> Maybe not. While it would be nice to solve the loop-and-a-half >> "problem" and the loop variable initialization "problem" (not everyone >> agrees these are even problems, especially now that we have >> comprehensions and generator expressions), as a matter of taste I like >> the fact that this particular class of side effects is given weighty >> statement syntax rather than more lightweight expression syntax. >> >> That is, I find statement syntax more readable. >> > > If you've read the PEP, you'll see that it encourages the use of > assignment statements whereever possible. If statement syntax is > generally more readable, by all means, use it. That doesn't mean there > aren't situations where the expression syntax is FAR more readable. > > Tell me, is this "more readable" than a loop with an actual condition in it? > > def sqrt(n): > guess, nextguess = 1, n > while True: > if math.isclose(guess, nextguess): return guess > guess = nextguess > nextguess = n / guess > > Readable doesn't mean "corresponds closely to its disassembly", > despite the way many people throw the word around. It also doesn't > mean "code I like", as opposed to "code I don't like". The words for > those concepts are "strongly typed" and "dynamically typed", as have > been demonstrated through MANY online discussions. (But I digress.) > Readable code is code which expresses an algorithm, expresses the > programmer's intent. It adequately demonstrates something at a > *higher* abstraction level. Does the algorithm demonstrated here > include an infinite loop? No? Then it shouldn't have "while True:" in > it. Thanks Chris - this is a very good explanation of how we can (somewhat) objectively look at "readability", and not one I'd really considered before. It's also an extremely good argument (IMO) that the loop-and-a-half construct would benefit from improvement. In my opinion, it's only partially related to the assignment expression discussion, though. Yes, assignment expressions "solve" the loop-and-a-half situation. I'm unsure how *much* I like the look of the resulting code, but I concede that's a "code I like" vs "code I don't like" situation. But assignment expressions are much more general than that, and as a general construct, they should be evaluated based on how many problems like this they solve, and whether the downsides justify it. We've already had the comprehension use case marginalised as no longer being a key use case for the proposal, because they weren't as "obviously" improved as some people had hoped. So overall, I think assignment expressions have proved to be a bit useful in some cases, and less so in others. Clearly any proposal can be picked to death with enough time to look for flaws. And part of the Zen is "Now is better than never". But I think in this case, "Although never is often better than *right* now" applies - we've had some very productive discussions, and you've done an incredible job of managing them and capturing the results, but it feels to me that the overall result is that there's likely a better solution still out there, that needs a new intuition to solve. > Now, this is a pretty obvious example. I deliberately wrote it so you > could simply lift the condition straight into the while header. And I > hope that everyone here agrees that this would be an improvement: > > def sqrt(n): > guess, nextguess = 1, n > while not math.isclose(guess, nextguess): > guess = nextguess > nextguess = n / guess > return guess > > But what if the condition were more complicated? > > def read_document(file): > doc = "" > while (token := file.get_next_token()) != "END": > doc += token > return doc > > The loop condition is "while the token is not END", or "while > get_next_token() doesn't return END", depending on your point of view. > Is it "more readable" to put that condition into the while header, or > to use an infinite loop and a break statement, or to duplicate a line > of code before the loop and at the bottom of the loop? Which one best > expresses the programmer's intention? The version that captures the value and tests it. I agree completely here. But we do have other options: def read_document(file): doc = "" for token in token_stream(file, terminator="END"): doc += token return doc (This point about rewriting to use for and an iterator applies to Chris Barker's fp.readline() example as well). Sure, token_stream might need a loop-and-a-half internally[1]. But from the user's point of view that's "low level" code, so not so important (ultimately, this is all about abstracting intent). And people maybe aren't used to writing "helper" iterators quite this freely, but that's a matter of education. So agreed - assignment expressions help with loop-and-a-half constructs. But we have other ways of dealing with them, so that's a relatively manageable situation. It's *still* all about cost-benefit trade-offs, with no clear winner (in my view). Paul [1] Although actually not - in *this* case, iter(file.get_next_token, 'END') is exactly what you need. But I concede that it's possible to demonstrate examples where that isn't the case. From rosuav at gmail.com Fri Apr 20 05:01:24 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 20 Apr 2018 19:01:24 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> Message-ID: On Fri, Apr 20, 2018 at 6:32 PM, Paul Moore wrote: >> def read_document(file): >> doc = "" >> while (token := file.get_next_token()) != "END": >> doc += token >> return doc >> >> The loop condition is "while the token is not END", or "while >> get_next_token() doesn't return END", depending on your point of view. >> Is it "more readable" to put that condition into the while header, or >> to use an infinite loop and a break statement, or to duplicate a line >> of code before the loop and at the bottom of the loop? Which one best >> expresses the programmer's intention? > > The version that captures the value and tests it. I agree completely > here. But we do have other options: > > def read_document(file): > doc = "" > for token in token_stream(file, terminator="END"): > doc += token > return doc > > (This point about rewriting to use for and an iterator applies to > Chris Barker's fp.readline() example as well). > > Sure, token_stream might need a loop-and-a-half internally[1]. But > from the user's point of view that's "low level" code, so not so > important (ultimately, this is all about abstracting intent). And > people maybe aren't used to writing "helper" iterators quite this > freely, but that's a matter of education. So agreed - assignment > expressions help with loop-and-a-half constructs. But we have other > ways of dealing with them, so that's a relatively manageable > situation. > > It's *still* all about cost-benefit trade-offs, with no clear winner > (in my view). You can always add another level of indirection. Always. Pushing something off into another function is helpful ONLY if you can usefully name that function, such that anyone who's reading the calling code can ignore the function and know everything they need to know about it. Otherwise, all you've done is force them to look elsewhere for the code. A bunch of single-use helper functions does not generally improve a module. > [1] Although actually not - in *this* case, iter(file.get_next_token, > 'END') is exactly what you need. But I concede that it's possible to > demonstrate examples where that isn't the case. The easiest way is to pick any comparison other than equality. If you want "is" / "is not", or if there are several termination conditions (so the check is "in {x, y, z}"), iter() can't help you. ChrisA From J.Demeyer at UGent.be Fri Apr 20 06:02:44 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Fri, 20 Apr 2018 12:02:44 +0200 Subject: [Python-Dev] PEP 575: Unifying function/method classes In-Reply-To: <3b065fca4bbd416b9d71afab455a4c2c@xmail101.UGent.be> References: <5ACF8552.6020607@UGent.be> <5AD1C725.5090603@UGent.be> <3b065fca4bbd416b9d71afab455a4c2c@xmail101.UGent.be> Message-ID: <5AD9BAC4.7000204@UGent.be> On 2018-04-14 23:14, Guido van Rossum wrote: > That actually sounds like a pretty big problem. I'm sure there is lots > of code that doesn't *just* duck-type nor calls inspect but uses > isinstance() to decide how to extract the desired information. I have been thinking about this some more... One solution to improve backwards compatibility would be to duplicate some classes. For example, make a separate class for bound methods in extension types, which would be literally a duplicate of the existing types.MethodType class (possibly with a different name). In other words, a bound method of an extension type would work exactly the same way as an existing bound method but it would artificially be a different class for the benefit of non-duck-typing. From christoph at grothesque.org Fri Apr 20 07:25:02 2018 From: christoph at grothesque.org (Christoph Groth) Date: Fri, 20 Apr 2018 13:25:02 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> Message-ID: <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> Steven Turnbull wrote: > Christoph Groth writes: > >> Wouldn't it be a pity not to liberate assignments from their boring >> statement existence? > > Maybe not. While it would be nice to solve the loop-and-a-half > "problem" and the loop variable initialization "problem" (not everyone > agrees these are even problems, especially now that we have > comprehensions and generator expressions), as a matter of taste I like > the fact that this particular class of side effects is given weighty > statement syntax rather than more lightweight expression syntax. I think that this is the crucial point. If it is indeed a design principle of Python that expressions should not have the side-effect of assigning names, than the whole discussion of PEP 572 could have been stopped early on. I didn't have this impression since core devs participated constructively in the discussion. > That is, I find statement syntax more readable. Many people will agree that := is more readable when used in cases where it's meant to be used (as listed in the PEP). Your objection seems to refer to the potential for "clever" misuse, like i := (a := list(iterator)).index(elem) instead of a := list(iterator) i := a.index(elem) I think that the ":=" syntax catches the eye and makes it easy to spot even hidden assignment expressions that shouldn't have been used. Note that the proposed syntax can be actually *more* readable even when used as a statement, like in equal := a == b Personally, I even slightly prefer a := 3 to the commonplace a = 3 because it visually expresses the asymmetry of the operation. (And no, Turbo Pascal was not my first programming language. :-) Christoph From p.f.moore at gmail.com Fri Apr 20 08:51:37 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 20 Apr 2018 13:51:37 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> Message-ID: On 20 April 2018 at 12:25, Christoph Groth wrote: >> Maybe not. While it would be nice to solve the loop-and-a-half >> "problem" and the loop variable initialization "problem" (not everyone >> agrees these are even problems, especially now that we have >> comprehensions and generator expressions), as a matter of taste I like >> the fact that this particular class of side effects is given weighty >> statement syntax rather than more lightweight expression syntax. > > I think that this is the crucial point. If it is indeed a design > principle of Python that expressions should not have the side-effect of > assigning names, than the whole discussion of PEP 572 could have been > stopped early on. I didn't have this impression since core devs > participated constructively in the discussion. I don't think it's a "design principle" as such, but it *is* true that until this point, functions with side effects excluded, Python's expression syntax has not included any means of assigning names, and that's expected behaviour (in the sense that when looking for where a name could have been bound, Python programmers do not look closely at the detail of expressions, because they can't be the source of an assignment). This PEP explicitly changes that, and that's a fairly radical change from current expectations. As to why core devs participated in the discussion, I can't speak for anyone else but for me: 1. PEPs deserve to be taken seriously - Chris put a lot of work into this PEP and simply dismissing it isn't giving it a fair hearing. The PEP didn't come out of nowhere, there had been previous discussions that established it was worth writing a PEP. 2. I hadn't really thought of the PEP in those terms. Now that you've mentioned it, "Python has never before had syntax that assigns names from within an expression" is quite a significant change, and one the PEP needs to examine the implications of. It's mostly been covered in the discussions by now, of course. 3. It was of interest to me. Just because I'm interested in a proposal doesn't necessarily mean it's a good idea. 4. I didn't like the idea much, and I wanted to raise my objections. Maybe they could be addressed, maybe not, but participation doesn't imply endorsement. Paul From guido at python.org Fri Apr 20 10:42:06 2018 From: guido at python.org (Guido van Rossum) Date: Fri, 20 Apr 2018 07:42:06 -0700 Subject: [Python-Dev] PEP 572: Now with 25% less reference implementation! In-Reply-To: References: Message-ID: If you want type hints you can use a variable annotation without initialization before the statement: data: bytes while data := stream.read(): print("Got", data) On Thu, Apr 19, 2018 at 9:45 PM, Dmitry Malinovsky wrote: > Hello Chris, and thank you for working on this PEP! > > What do you think about using variable type hints with this syntax? > I tried to search through python-dev and couldn't find a single post > discussing that question. > If I missed it somehow, could you please include its conclusions into the > PEP? > > For instance, as I understand now the parser will fail on this snippet: > > while data: bytes := stream.read(): > print("Received data:", data) > > Do brackets help? > > while (data: bytes := stream.read()): > print("Received data:", data) > > IIUC, in 3.7 It is invalid syntax to specify a type hint for a for loop > item; > should brackets help? Currently they don't: > > Python 3.7.0b3+ (heads/3.7:7dcfd6c, Mar 30 2018, 21:30:34) > [Clang 9.0.0 (clang-900.0.39.2)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>> for (x: int) in [1,2,3]: > File "", line 1 > for (x: int) in [1,2,3]: > ^ > SyntaxError: invalid syntax > > Thanks, > > > On 20 Apr 2018, at 09:10, Chris Angelico wrote: > > > > Working on the reference implementation for PEP 572 is turning out to > > be a massive time sink, both on my personal schedule and on the PEP's > > discussion. I can't just hold off all discussion on a topic until I > > figure out whether something is possible or not, because that could > > take me several days, even a week or more. And considering the massive > > backlash against the proposal, it seems like a poor use of my time to > > try to prove that something's impossible, find that I don't know > > enough about grammar editing to be able to say anything more than > > "well, I couldn't do it, but someone else might be able to", and then > > try to resume the discussion with no more certainty than we had > > before. > > > > So here's the PEP again, simplified. I'm fairly sure it's just going > > to be another on a growing list of rejected PEPs to my name, and I'm > > done with trying to argue some of these points. Either the rules get > > simplified, or they don't. Trying to simplify the rules and maintain > > perfect backward compatibility is just making the rules even more > > complicated. > > > > PEP 572, if accepted, *will* change the behaviour of certain > > constructs inside comprehensions, mainly due to interactions with > > class scope that make no sense unless you know how they're implemented > > internally. The official tutorial pretends that comprehensions are > > "equivalent to" longhand: > > > > https://docs.python.org/3/tutorial/datastructures.html? > highlight=equivalent#list-comprehensions > > https://docs.python.org/3/howto/functional.html?highlight=equivalent# > generator-expressions-and-list-comprehensions > > > > and this is an inaccuracy for the sake of simplicity. PEP 572 will > > make this far more accurate; the only difference is that the > > comprehension is inside a function. Current semantics are far more > > bizarre than that. > > > > Do you want absolutely 100% backward compatibility? Then reject this > > PEP. Or better still, keep using Python 3.7, and don't upgrade to 3.8, > > in case something breaks. Do you want list comprehensions that make > > better sense? Then accept that some code will need to change, if it > > tried to use the same name in multiple scopes, or tried to use ancient > > Python 2 semantics with a yield expression in the outermost iterable. > > > > I'm pretty much ready for pronouncement. > > > > https://www.python.org/dev/peps/pep-0572/ > > > > ChrisA > > > > PEP: 572 > > Title: Assignment Expressions > > Author: Chris Angelico > > Status: Draft > > Type: Standards Track > > Content-Type: text/x-rst > > Created: 28-Feb-2018 > > Python-Version: 3.8 > > Post-History: 28-Feb-2018, 02-Mar-2018, 23-Mar-2018, 04-Apr-2018, > 17-Apr-2018 > > > > > > Abstract > > ======== > > > > This is a proposal for creating a way to assign to variables within an > > expression. Additionally, the precise scope of comprehensions is > adjusted, to > > maintain consistency and follow expectations. > > > > > > Rationale > > ========= > > > > Naming the result of an expression is an important part of programming, > > allowing a descriptive name to be used in place of a longer expression, > > and permitting reuse. Currently, this feature is available only in > > statement form, making it unavailable in list comprehensions and other > > expression contexts. Merely introducing a way to assign as an expression > > would create bizarre edge cases around comprehensions, though, and to > avoid > > the worst of the confusions, we change the definition of comprehensions, > > causing some edge cases to be interpreted differently, but maintaining > the > > existing behaviour in the majority of situations. > > > > > > Syntax and semantics > > ==================== > > > > In any context where arbitrary Python expressions can be used, a **named > > expression** can appear. This is of the form ``target := expr`` where > > ``expr`` is any valid Python expression, and ``target`` is any valid > > assignment target. > > > > The value of such a named expression is the same as the incorporated > > expression, with the additional side-effect that the target is assigned > > that value:: > > > > # Handle a matched regex > > if (match := pattern.search(data)) is not None: > > ... > > > > # A more explicit alternative to the 2-arg form of iter() invocation > > while (value := read_next_item()) is not None: > > ... > > > > # Share a subexpression between a comprehension filter clause and its > output > > filtered_data = [y for x in data if (y := f(x)) is not None] > > > > > > Differences from regular assignment statements > > ---------------------------------------------- > > > > Most importantly, since ``:=`` is an expression, it can be used in > contexts > > where statements are illegal, including lambda functions and > comprehensions. > > > > An assignment statement can assign to multiple targets, left-to-right:: > > > > x = y = z = 0 > > > > The equivalent assignment expression is parsed as separate binary > operators, > > and is therefore processed right-to-left, as if it were spelled thus:: > > > > assert 0 == (x := (y := (z := 0))) > > > > Augmented assignment is not supported in expression form:: > > > >>>> x +:= 1 > > File "", line 1 > > x +:= 1 > > ^ > > SyntaxError: invalid syntax > > > > Otherwise, the semantics of assignment are identical in statement and > > expression forms. > > > > > > Alterations to comprehensions > > ----------------------------- > > > > The current behaviour of list/set/dict comprehensions and generator > > expressions has some edge cases that would behave strangely if an > assignment > > expression were to be used. Therefore the proposed semantics are changed, > > removing the current edge cases, and instead altering their behaviour > *only* > > in a class scope. > > > > As of Python 3.7, the outermost iterable of any comprehension is > evaluated > > in the surrounding context, and then passed as an argument to the > implicit > > function that evaluates the comprehension. > > > > Under this proposal, the entire body of the comprehension is evaluated in > > its implicit function. Names not assigned to within the comprehension are > > located in the surrounding scopes, as with normal lookups. As one special > > case, a comprehension at class scope will **eagerly bind** any name which > > is already defined in the class scope. > > > > A list comprehension can be unrolled into an equivalent function. With > > Python 3.7 semantics:: > > > > numbers = [x + y for x in range(3) for y in range(4)] > > # Is approximately equivalent to > > def (iterator): > > result = [] > > for x in iterator: > > for y in range(4): > > result.append(x + y) > > return result > > numbers = (iter(range(3))) > > > > Under the new semantics, this would instead be equivalent to:: > > > > def (): > > result = [] > > for x in range(3): > > for y in range(4): > > result.append(x + y) > > return result > > numbers = () > > > > When a class scope is involved, a naive transformation into a function > would > > prevent name lookups (as the function would behave like a method):: > > > > class X: > > names = ["Fred", "Barney", "Joe"] > > prefix = "> " > > prefixed_names = [prefix + name for name in names] > > > > With Python 3.7 semantics, this will evaluate the outermost iterable at > class > > scope, which will succeed; but it will evaluate everything else in a > function:: > > > > class X: > > names = ["Fred", "Barney", "Joe"] > > prefix = "> " > > def (iterator): > > result = [] > > for name in iterator: > > result.append(prefix + name) > > return result > > prefixed_names = (iter(names)) > > > > The name ``prefix`` is thus searched for at global scope, ignoring the > class > > name. Under the proposed semantics, this name will be eagerly bound; and > the > > same early binding then handles the outermost iterable as well. The list > > comprehension is thus approximately equivalent to:: > > > > class X: > > names = ["Fred", "Barney", "Joe"] > > prefix = "> " > > def (names=names, prefix=prefix): > > result = [] > > for name in names: > > result.append(prefix + name) > > return result > > prefixed_names = () > > > > With list comprehensions, this is unlikely to cause any confusion. With > > generator expressions, this has the potential to affect behaviour, as the > > eager binding means that the name could be rebound between the creation > of > > the genexp and the first call to ``next()``. It is, however, more closely > > aligned to normal expectations. The effect is ONLY seen with names that > > are looked up from class scope; global names (eg ``range()``) will still > > be late-bound as usual. > > > > One consequence of this change is that certain bugs in genexps will not > > be detected until the first call to ``next()``, where today they would be > > caught upon creation of the generator. > > > > > > Recommended use-cases > > ===================== > > > > Simplifying list comprehensions > > ------------------------------- > > > > A list comprehension can map and filter efficiently by capturing > > the condition:: > > > > results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0] > > > > Similarly, a subexpression can be reused within the main expression, by > > giving it a name on first use:: > > > > stuff = [[y := f(x), x/y] for x in range(5)] > > > > # There are a number of less obvious ways to spell this in current > > # versions of Python, such as: > > > > # Inline helper function > > stuff = [(lambda y: [y,x/y])(f(x)) for x in range(5)] > > > > # Extra 'for' loop - potentially could be optimized internally > > stuff = [[y, x/y] for x in range(5) for y in [f(x)]] > > > > # Using a mutable cache object (various forms possible) > > c = {} > > stuff = [[c.update(y=f(x)) or c['y'], x/c['y']] for x in range(5)] > > > > In all cases, the name is local to the comprehension; like iteration > variables, > > it cannot leak out into the surrounding context. > > > > > > Capturing condition values > > -------------------------- > > > > Assignment expressions can be used to good effect in the header of > > an ``if`` or ``while`` statement:: > > > > # Proposed syntax > > while (command := input("> ")) != "quit": > > print("You entered:", command) > > > > # Capturing regular expression match objects > > # See, for instance, Lib/pydoc.py, which uses a multiline spelling > > # of this effect > > if match := re.search(pat, text): > > print("Found:", match.group(0)) > > > > # Reading socket data until an empty string is returned > > while data := sock.read(): > > print("Received data:", data) > > > > # Equivalent in current Python, not caring about function return value > > while input("> ") != "quit": > > print("You entered a command.") > > > > # To capture the return value in current Python demands a four-line > > # loop header. > > while True: > > command = input("> "); > > if command == "quit": > > break > > print("You entered:", command) > > > > Particularly with the ``while`` loop, this can remove the need to have an > > infinite loop, an assignment, and a condition. It also creates a smooth > > parallel between a loop which simply uses a function call as its > condition, > > and one which uses that as its condition but also uses the actual value. > > > > > > Rejected alternative proposals > > ============================== > > > > Proposals broadly similar to this one have come up frequently on > python-ideas. > > Below are a number of alternative syntaxes, some of them specific to > > comprehensions, which have been rejected in favour of the one given > above. > > > > > > Alternative spellings > > --------------------- > > > > Broadly the same semantics as the current proposal, but spelled > differently. > > > > 1. ``EXPR as NAME``:: > > > > stuff = [[f(x) as y, x/y] for x in range(5)] > > > > Since ``EXPR as NAME`` already has meaning in ``except`` and ``with`` > > statements (with different semantics), this would create unnecessary > > confusion or require special-casing (eg to forbid assignment within the > > headers of these statements). > > > > 2. ``EXPR -> NAME``:: > > > > stuff = [[f(x) -> y, x/y] for x in range(5)] > > > > This syntax is inspired by languages such as R and Haskell, and some > > programmable calculators. (Note that a left-facing arrow ``y <- f(x)`` > is > > not possible in Python, as it would be interpreted as less-than and > unary > > minus.) This syntax has a slight advantage over 'as' in that it does > not > > conflict with ``with`` and ``except`` statements, but otherwise is > > equivalent. > > > > 3. Adorning statement-local names with a leading dot:: > > > > stuff = [[(f(x) as .y), x/.y] for x in range(5)] # with "as" > > stuff = [[(.y := f(x)), x/.y] for x in range(5)] # with ":=" > > > > This has the advantage that leaked usage can be readily detected, > removing > > some forms of syntactic ambiguity. However, this would be the only > place > > in Python where a variable's scope is encoded into its name, making > > refactoring harder. > > > > 4. Adding a ``where:`` to any statement to create local name bindings:: > > > > value = x**2 + 2*x where: > > x = spam(1, 4, 7, q) > > > > Execution order is inverted (the indented body is performed first, > followed > > by the "header"). This requires a new keyword, unless an existing > keyword > > is repurposed (most likely ``with:``). See PEP 3150 for prior > discussion > > on this subject (with the proposed keyword being ``given:``). > > > > 5. ``TARGET from EXPR``:: > > > > stuff = [[y from f(x), x/y] for x in range(5)] > > > > This syntax has fewer conflicts than ``as`` does (conflicting only > with the > > ``raise Exc from Exc`` notation), but is otherwise comparable to it. > Instead > > of paralleling ``with expr as target:`` (which can be useful but can > also be > > confusing), this has no parallels, but is evocative. > > > > > > Special-casing conditional statements > > ------------------------------------- > > > > One of the most popular use-cases is ``if`` and ``while`` statements. > Instead > > of a more general solution, this proposal enhances the syntax of these > two > > statements to add a means of capturing the compared value:: > > > > if re.search(pat, text) as match: > > print("Found:", match.group(0)) > > > > This works beautifully if and ONLY if the desired condition is based on > the > > truthiness of the captured value. It is thus effective for specific > > use-cases (regex matches, socket reads that return `''` when done), and > > completely useless in more complicated cases (eg where the condition is > > ``f(x) < 0`` and you want to capture the value of ``f(x)``). It also has > > no benefit to list comprehensions. > > > > Advantages: No syntactic ambiguities. Disadvantages: Answers only a > fraction > > of possible use-cases, even in ``if``/``while`` statements. > > > > > > Special-casing comprehensions > > ----------------------------- > > > > Another common use-case is comprehensions (list/set/dict, and genexps). > As > > above, proposals have been made for comprehension-specific solutions. > > > > 1. ``where``, ``let``, or ``given``:: > > > > stuff = [(y, x/y) where y = f(x) for x in range(5)] > > stuff = [(y, x/y) let y = f(x) for x in range(5)] > > stuff = [(y, x/y) given y = f(x) for x in range(5)] > > > > This brings the subexpression to a location in between the 'for' loop > and > > the expression. It introduces an additional language keyword, which > creates > > conflicts. Of the three, ``where`` reads the most cleanly, but also > has the > > greatest potential for conflict (eg SQLAlchemy and numpy have ``where`` > > methods, as does ``tkinter.dnd.Icon`` in the standard library). > > > > 2. ``with NAME = EXPR``:: > > > > stuff = [(y, x/y) with y = f(x) for x in range(5)] > > > > As above, but reusing the `with` keyword. Doesn't read too badly, and > needs > > no additional language keyword. Is restricted to comprehensions, > though, > > and cannot as easily be transformed into "longhand" for-loop syntax. > Has > > the C problem that an equals sign in an expression can now create a > name > > binding, rather than performing a comparison. Would raise the question > of > > why "with NAME = EXPR:" cannot be used as a statement on its own. > > > > 3. ``with EXPR as NAME``:: > > > > stuff = [(y, x/y) with f(x) as y for x in range(5)] > > > > As per option 2, but using ``as`` rather than an equals sign. Aligns > > syntactically with other uses of ``as`` for name binding, but a simple > > transformation to for-loop longhand would create drastically different > > semantics; the meaning of ``with`` inside a comprehension would be > > completely different from the meaning as a stand-alone statement, while > > retaining identical syntax. > > > > Regardless of the spelling chosen, this introduces a stark difference > between > > comprehensions and the equivalent unrolled long-hand form of the loop. > It is > > no longer possible to unwrap the loop into statement form without > reworking > > any name bindings. The only keyword that can be repurposed to this task > is > > ``with``, thus giving it sneakily different semantics in a comprehension > than > > in a statement; alternatively, a new keyword is needed, with all the > costs > > therein. > > > > > > Lowering operator precedence > > ---------------------------- > > > > There are two logical precedences for the ``:=`` operator. Either it > should > > bind as loosely as possible, as does statement-assignment; or it should > bind > > more tightly than comparison operators. Placing its precedence between > the > > comparison and arithmetic operators (to be precise: just lower than > bitwise > > OR) allows most uses inside ``while`` and ``if`` conditions to be spelled > > without parentheses, as it is most likely that you wish to capture the > value > > of something, then perform a comparison on it:: > > > > pos = -1 > > while pos := buffer.find(search_term, pos + 1) >= 0: > > ... > > > > Once find() returns -1, the loop terminates. If ``:=`` binds as loosely > as > > ``=`` does, this would capture the result of the comparison (generally > either > > ``True`` or ``False``), which is less useful. > > > > While this behaviour would be convenient in many situations, it is also > harder > > to explain than "the := operator behaves just like the assignment > statement", > > and as such, the precedence for ``:=`` has been made as close as > possible to > > that of ``=``. > > > > > > Migration path > > ============== > > > > The semantic changes to list/set/dict comprehensions, and more so to > generator > > expressions, may potentially require migration of code. In many cases, > the > > changes simply make legal what used to raise an exception, but there are > some > > edge cases that were previously legal and now are not, and a few corner > cases > > with altered semantics. > > > > > > The Outermost Iterable > > ---------------------- > > > > As of Python 3.7, the outermost iterable in a comprehension is special: > it is > > evaluated in the surrounding context, instead of inside the > comprehension. > > Thus it is permitted to contain a ``yield`` expression, to use a name > also > > used elsewhere, and to reference names from class scope. Also, in a > genexp, > > the outermost iterable is pre-evaluated, but the rest of the code is not > > touched until the genexp is first iterated over. Class scope is now > handled > > more generally (see above), but if other changes require the old > behaviour, > > the iterable must be explicitly elevated from the comprehension:: > > > > # Python 3.7 > > def f(x): > > return [x for x in x if x] > > def g(): > > return [x for x in [(yield 1)]] > > # With PEP 572 > > def f(x): > > return [y for y in x if y] > > def g(): > > sent_item = (yield 1) > > return [x for x in [sent_item]] > > > > This more clearly shows that it is g(), not the comprehension, which is > able > > to yield values (and is thus a generator function). The entire > comprehension > > is consistently in a single scope. > > > > The following expressions would, in Python 3.7, raise exceptions > immediately. > > With the removal of the outermost iterable's special casing, they are now > > equivalent to the most obvious longhand form:: > > > > gen = (x for x in rage(10)) # NameError > > gen = (x for x in 10) # TypeError (not iterable) > > gen = (x for x in range(1/0)) # ZeroDivisionError > > > > def (): > > for x in rage(10): > > yield x > > gen = () # No exception yet > > tng = next(gen) # NameError > > > > > > Open questions > > ============== > > > > Importing names into comprehensions > > ----------------------------------- > > > > A list comprehension can use and update local names, and they will retain > > their values from one iteration to another. It would be convenient to use > > this feature to create rolling or self-effecting data streams:: > > > > progressive_sums = [total := total + value for value in data] > > > > This will fail with UnboundLocalError due to ``total`` not being > initalized. > > Simply initializing it outside of the comprehension is insufficient - > unless > > the comprehension is in class scope:: > > > > class X: > > total = 0 > > progressive_sums = [total := total + value for value in data] > > > > At other scopes, it may be beneficial to have a way to fetch a value > from the > > surrounding scope. Should this be automatic? Should it be controlled > with a > > keyword? Hypothetically (and using no new keywords), this could be > written:: > > > > total = 0 > > progressive_sums = [total := total + value > > import nonlocal total > > for value in data] > > > > Translated into longhand, this would become:: > > > > total = 0 > > def (total=total): > > result = [] > > for value in data: > > result.append(total := total + value) > > return result > > progressive_sums = () > > > > ie utilizing the same early-binding technique that is used at class > scope. > > > > > > Frequently Raised Objections > > ============================ > > > > Why not just turn existing assignment into an expression? > > --------------------------------------------------------- > > > > C and its derivatives define the ``=`` operator as an expression, rather > than > > a statement as is Python's way. This allows assignments in more > contexts, > > including contexts where comparisons are more common. The syntactic > similarity > > between ``if (x == y)`` and ``if (x = y)`` belies their drastically > different > > semantics. Thus this proposal uses ``:=`` to clarify the distinction. > > > > > > This could be used to create ugly code! > > --------------------------------------- > > > > So can anything else. This is a tool, and it is up to the programmer to > use it > > where it makes sense, and not use it where superior constructs can be > used. > > > > > > With assignment expressions, why bother with assignment statements? > > ------------------------------------------------------------------- > > > > The two forms have different flexibilities. The ``:=`` operator can be > used > > inside a larger expression; the ``=`` statement can be augmented to > ``+=`` and > > its friends. The assignment statement is a clear declaration of intent: > this > > value is to be assigned to this target, and that's it. > > > > > > Why not use a sublocal scope and prevent namespace pollution? > > ------------------------------------------------------------- > > > > Previous revisions of this proposal involved sublocal scope (restricted > to a > > single statement), preventing name leakage and namespace pollution. > While a > > definite advantage in a number of situations, this increases complexity > in > > many others, and the costs are not justified by the benefits. In the > interests > > of language simplicity, the name bindings created here are exactly > equivalent > > to any other name bindings, including that usage at class or module > scope will > > create externally-visible names. This is no different from ``for`` > loops or > > other constructs, and can be solved the same way: ``del`` the name once > it is > > no longer needed, or prefix it with an underscore. > > > > Names bound within a comprehension are local to that comprehension, even > in > > the outermost iterable, and can thus be used freely without polluting the > > surrounding namespace. > > > > (The author wishes to thank Guido van Rossum and Christoph Groth for > their > > suggestions to move the proposal in this direction. [2]_) > > > > > > Style guide recommendations > > =========================== > > > > As this adds another way to spell some of the same effects as can > already be > > done, it is worth noting a few broad recommendations. These could be > included > > in PEP 8 and/or other style guides. > > > > 1. If either assignment statements or assignment expressions can be > > used, prefer statements; they are a clear declaration of intent. > > > > 2. If using assignment expressions would lead to ambiguity about > > execution order, restructure it to use statements instead. > > > > > > Acknowledgements > > ================ > > > > The author wishes to thank Guido van Rossum and Nick Coghlan for their > > considerable contributions to this proposal, and to members of the > > core-mentorship mailing list for assistance with implementation. > > > > > > References > > ========== > > > > .. [1] Proof of concept / reference implementation > > (https://github.com/Rosuav/cpython/tree/assignment-expressions) > > .. [2] Pivotal post regarding inline assignment semantics > > (https://mail.python.org/pipermail/python-ideas/2018-March/049409.html > ) > > > > > > Copyright > > ========= > > > > This document has been placed in the public domain. > > > > > > .. > > Local Variables: > > mode: indented-text > > indent-tabs-mode: nil > > sentence-end-double-space: t > > fill-column: 70 > > coding: utf-8 > > End: > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > https://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > damalinov%40gmail.com > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Fri Apr 20 11:07:22 2018 From: chris.barker at noaa.gov (Chris Barker - NOAA Federal) Date: Fri, 20 Apr 2018 08:07:22 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> Message-ID: > Personally, I even slightly prefer > > a := 3 > > to the commonplace > > a = 3 > because it visually expresses the asymmetry of the operation. Careful here! That?s a fine argument for using := in a new language, but people using := when they don?t need an expression because they like the symbol better is a reason NOT to do this. And yes, normally aspirated Pascal WAS my first programming language. :-) -CHB From ethan at stoneleaf.us Fri Apr 20 11:35:03 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 20 Apr 2018 08:35:03 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> Message-ID: <5ADA08A7.2050500@stoneleaf.us> On 04/20/2018 08:07 AM, Chris Barker - NOAA Federal wrote: >On 04/20/2018 04:25 AM, Christoph Groth wrote: >> Personally, I even slightly prefer >> >> a := 3 >> >> to the commonplace >> >> a = 3 >> because it visually expresses the asymmetry of the operation. > > Careful here! That?s a fine argument for using := in a new language, > but people using := when they don?t need an expression because they > like the symbol better is a reason NOT to do this. Unless it's a bug magnet, that doesn't strike me as a good reason NOT to do this. -- ~Ethan~ From rosuav at gmail.com Fri Apr 20 11:49:25 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 21 Apr 2018 01:49:25 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> Message-ID: On Fri, Apr 20, 2018 at 10:51 PM, Paul Moore wrote: > 2. I hadn't really thought of the PEP in those terms. Now that you've > mentioned it, "Python has never before had syntax that assigns names > from within an expression" is quite a significant change, and one the > PEP needs to examine the implications of. It's mostly been covered in > the discussions by now, of course. Depending on your definition of "assignment", a lambda function could count as a means of assigning a variable in a subexpression. But yes, there is no convenient way to assign to something in a wider scope. ChrisA From guido at python.org Fri Apr 20 11:50:09 2018 From: guido at python.org (Guido van Rossum) Date: Fri, 20 Apr 2018 08:50:09 -0700 Subject: [Python-Dev] PEP 572: Now with 25% less reference implementation! In-Reply-To: References: Message-ID: Maybe annotations should get a brief mention in the Rejected Ideas section, with your explanation here added. (And maybe my response.) On Thu, Apr 19, 2018 at 11:31 PM, Chris Angelico wrote: > On Fri, Apr 20, 2018 at 2:45 PM, Dmitry Malinovsky > wrote: > > Hello Chris, and thank you for working on this PEP! > > > > What do you think about using variable type hints with this syntax? > > I tried to search through python-dev and couldn't find a single post > > discussing that question. > > If I missed it somehow, could you please include its conclusions into > the PEP? > > I'm ignoring them for the sake of the PEP, because it'd complicate the > grammar for little benefit. If someone wants to add an enhancement > later, that's fine; but the proposal can stand without it, and with > it, it'll make for even more noise in a line full of colons. > > > For instance, as I understand now the parser will fail on this snippet: > > > > while data: bytes := stream.read(): > > print("Received data:", data) > > > > Do brackets help? > > > > while (data: bytes := stream.read()): > > print("Received data:", data) > > > > IIUC, in 3.7 It is invalid syntax to specify a type hint for a for loop > item; > > should brackets help? Currently they don't: > > > > Python 3.7.0b3+ (heads/3.7:7dcfd6c, Mar 30 2018, 21:30:34) > > [Clang 9.0.0 (clang-900.0.39.2)] on darwin > > Type "help", "copyright", "credits" or "license" for more > information. > > >>> for (x: int) in [1,2,3]: > > File "", line 1 > > for (x: int) in [1,2,3]: > > ^ > > SyntaxError: invalid syntax > > And that's another good reason not to bother, at least for now. I'm > not sure whether you can use a Py2-style type hint comment on a for > loop, but if so, you should also be able to do it on a while loop or > anything. Or, of course, you can just annotate the variable before the > loop, if you want to. > > ChrisA > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Apr 20 11:58:11 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 21 Apr 2018 01:58:11 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> Message-ID: On 20 April 2018 at 21:25, Christoph Groth wrote: > Steven Turnbull wrote: >> >> Christoph Groth writes: >> >>> Wouldn't it be a pity not to liberate assignments from their boring >>> statement existence? >> >> >> Maybe not. While it would be nice to solve the loop-and-a-half >> "problem" and the loop variable initialization "problem" (not everyone >> agrees these are even problems, especially now that we have >> comprehensions and generator expressions), as a matter of taste I like >> the fact that this particular class of side effects is given weighty >> statement syntax rather than more lightweight expression syntax. > > > I think that this is the crucial point. If it is indeed a design > principle of Python that expressions should not have the side-effect of > assigning names, than the whole discussion of PEP 572 could have been > stopped early on. I didn't have this impression since core devs > participated constructively in the discussion. There were a couple of factors at play there. Firstly, python-ideas and python-dev play different roles in the process, with python-ideas focused on "Help PEP authors put forward the most compelling proposal possible", and then python-dev providing the far more stringent filter of "Do we sincerely believe the long term improvements in language learnability and code maintainability arising from this change will outweigh the inevitable near term costs?" (python-ideas does consider the latter question as well, but we're more willing to spend time on ideas that only reach the level "Maybe? Depending on your point of view?"). Secondly, the original PEP proposed sublocal scopes *precisely* to help preserve that property by limiting the impact of any name binding side effects to a single statement (albeit an arbtirarily long nested suite in the case of compound statements). My own enthusiasm for the idea largely waned after folks successfully campaigned for "we'd prefer side effects to introducing a new kind of scope" (and while I'm definitely sympathetic to the "Python's name lookup scoping rules are already excessively complicated" point of view, I also think that if "=" and ":=" both target the same kind of scope, there isn't enough new expressiveness introduced by the latter to justify the syntactic complexity of adding it). Cheers, Nick. P.S. While I'm not planning to work on it myself anytime soon, I think the sublocal scoping semantics proposed in earlier versions of PEP 572 would provide a *much* better developer experience for PEP 3150's "given" clause (which is currently deferred indefinitely, as even I don't particularly like the current incarnation of that proposal). -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Fri Apr 20 12:08:34 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 21 Apr 2018 02:08:34 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> Message-ID: On 21 April 2018 at 01:49, Chris Angelico wrote: > On Fri, Apr 20, 2018 at 10:51 PM, Paul Moore wrote: >> 2. I hadn't really thought of the PEP in those terms. Now that you've >> mentioned it, "Python has never before had syntax that assigns names >> from within an expression" is quite a significant change, and one the >> PEP needs to examine the implications of. It's mostly been covered in >> the discussions by now, of course. > > Depending on your definition of "assignment", a lambda function could > count as a means of assigning a variable in a subexpression. But yes, > there is no convenient way to assign to something in a wider scope. We used to sort of have one (Python 2 list comprehensions), and the behaviour was sufficiently unpopular that Py3 introduced an implicitly nested scope to keep the iteration variable name from leaking :) That history is a non-trivial part of why I advocated more strongly for the original sublocal scoping version of the proposal: with tighter lexical scoping for expression level assignments, I think we'd be able to avoid most of the downsides that come from reinstating the ability for expressions to bind and rebind names. However, given how those original discussions went, I now think the only way that option might be successfully pitched to people would be to propose a statement-level variant of it first (perhaps in the form of a heavily revised variant of PEP 3150's given clause), and then only propose "expression level name binding with implicit sublocal scopes" after the semantics of sublocal scoping were already established elsewhere. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From status at bugs.python.org Fri Apr 20 12:09:55 2018 From: status at bugs.python.org (Python tracker) Date: Fri, 20 Apr 2018 18:09:55 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20180420160955.62D8E5670D@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2018-04-13 - 2018-04-20) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 6587 (+15) closed 38484 (+32) total 45071 (+47) Open issues with patches: 2573 Issues opened (30) ================== #33274: minidom removeAttributeNode returns None https://bugs.python.org/issue33274 opened by iter #33275: glob.glob should explicitly note that results aren't sorted https://bugs.python.org/issue33275 opened by Ben FrantzDale #33276: Clarify that __path__ can't be set to just anything https://bugs.python.org/issue33276 opened by brett.cannon #33277: Deprecate __loader__, __package__, __file__, and __cached__ on https://bugs.python.org/issue33277 opened by brett.cannon #33278: libexpat uses HAVE_SYSCALL_GETRANDOM instead of HAVE_GETRANDOM https://bugs.python.org/issue33278 opened by lemburg #33280: Update link to Tcl/Tk 8.6 man pages in tkinter.rst https://bugs.python.org/issue33280 opened by adelfino #33281: ctypes.util.find_library not working on macOS https://bugs.python.org/issue33281 opened by Ian Burgwin #33282: Subprocess Popen communicate hung if stdin not given to Popen, https://bugs.python.org/issue33282 opened by JatinGoel #33284: Increase test coverage for numbers.py https://bugs.python.org/issue33284 opened by Barry Devlin #33286: Conflict between tqdm and multiprocessing on windows https://bugs.python.org/issue33286 opened by schwemro #33289: askcolor is returning floats for r,g,b values instead of ints https://bugs.python.org/issue33289 opened by Bryan.Oakley #33294: Support complex expressions for py-print command. https://bugs.python.org/issue33294 opened by Martin Li??ka #33295: ERROR: test_sites_no_connection_close (test.test_urllib2net.Ot https://bugs.python.org/issue33295 opened by inada.naoki #33296: datetime.datetime.utcfromtimestamp call decimal causes precisi https://bugs.python.org/issue33296 opened by anglister #33297: Mention Pillow package on tkinter.rst to work with more image https://bugs.python.org/issue33297 opened by adelfino #33300: Bad usage example in id() DocString https://bugs.python.org/issue33300 opened by gneff #33301: Add __contains__ to pathlib https://bugs.python.org/issue33301 opened by Alok Singh #33302: The search for pyvenv.cfg doesn't match PEP 405 https://bugs.python.org/issue33302 opened by mattheww #33303: ElementTree Comment text isn't escaped https://bugs.python.org/issue33303 opened by johnburnett #33305: Improve syntax error for numbers with leading zero https://bugs.python.org/issue33305 opened by steven.daprano #33306: Improving SyntaxError for unmatched parentheses https://bugs.python.org/issue33306 opened by serhiy.storchaka #33309: Unittest Mock objects do not freeze arguments they are called https://bugs.python.org/issue33309 opened by slacknate #33311: cgitb: remove parentheses when the error is in module https://bugs.python.org/issue33311 opened by sblondon #33312: ubsan undefined behavior sanitizer flags struct _dictkeysobjec https://bugs.python.org/issue33312 opened by gregory.p.smith #33314: Bad rendering in the documentation for the os module https://bugs.python.org/issue33314 opened by pablogsal #33315: Allow queue.Queue to be used in type annotations https://bugs.python.org/issue33315 opened by sproshev #33316: PyThread_release_lock always fails https://bugs.python.org/issue33316 opened by Ivan.Pozdeev #33317: `repr()` of string in NFC and NFD forms does not differ https://bugs.python.org/issue33317 opened by pekka.klarck #33318: Move folding tuples of constants into compiler.c from peephole https://bugs.python.org/issue33318 opened by serhiy.storchaka #33319: `subprocess.run` documentation doesn't tell is using `stdout=P https://bugs.python.org/issue33319 opened by pekka.klarck Most recent 15 issues with no replies (15) ========================================== #33319: `subprocess.run` documentation doesn't tell is using `stdout=P https://bugs.python.org/issue33319 #33318: Move folding tuples of constants into compiler.c from peephole https://bugs.python.org/issue33318 #33316: PyThread_release_lock always fails https://bugs.python.org/issue33316 #33315: Allow queue.Queue to be used in type annotations https://bugs.python.org/issue33315 #33314: Bad rendering in the documentation for the os module https://bugs.python.org/issue33314 #33311: cgitb: remove parentheses when the error is in module https://bugs.python.org/issue33311 #33309: Unittest Mock objects do not freeze arguments they are called https://bugs.python.org/issue33309 #33306: Improving SyntaxError for unmatched parentheses https://bugs.python.org/issue33306 #33303: ElementTree Comment text isn't escaped https://bugs.python.org/issue33303 #33302: The search for pyvenv.cfg doesn't match PEP 405 https://bugs.python.org/issue33302 #33301: Add __contains__ to pathlib https://bugs.python.org/issue33301 #33297: Mention Pillow package on tkinter.rst to work with more image https://bugs.python.org/issue33297 #33294: Support complex expressions for py-print command. https://bugs.python.org/issue33294 #33289: askcolor is returning floats for r,g,b values instead of ints https://bugs.python.org/issue33289 #33284: Increase test coverage for numbers.py https://bugs.python.org/issue33284 Most recent 15 issues waiting for review (15) ============================================= #33318: Move folding tuples of constants into compiler.c from peephole https://bugs.python.org/issue33318 #33316: PyThread_release_lock always fails https://bugs.python.org/issue33316 #33314: Bad rendering in the documentation for the os module https://bugs.python.org/issue33314 #33312: ubsan undefined behavior sanitizer flags struct _dictkeysobjec https://bugs.python.org/issue33312 #33311: cgitb: remove parentheses when the error is in module https://bugs.python.org/issue33311 #33306: Improving SyntaxError for unmatched parentheses https://bugs.python.org/issue33306 #33305: Improve syntax error for numbers with leading zero https://bugs.python.org/issue33305 #33297: Mention Pillow package on tkinter.rst to work with more image https://bugs.python.org/issue33297 #33295: ERROR: test_sites_no_connection_close (test.test_urllib2net.Ot https://bugs.python.org/issue33295 #33294: Support complex expressions for py-print command. https://bugs.python.org/issue33294 #33284: Increase test coverage for numbers.py https://bugs.python.org/issue33284 #33280: Update link to Tcl/Tk 8.6 man pages in tkinter.rst https://bugs.python.org/issue33280 #33266: 2to3 doesn't parse all valid string literals https://bugs.python.org/issue33266 #33263: Asyncio server enters an invalid state after a request with SO https://bugs.python.org/issue33263 #33262: Deprecate shlex.split(None) to read from stdin. https://bugs.python.org/issue33262 Top 10 most discussed issues (10) ================================= #33312: ubsan undefined behavior sanitizer flags struct _dictkeysobjec https://bugs.python.org/issue33312 11 msgs #32232: building extensions as builtins is broken in 3.7 https://bugs.python.org/issue32232 8 msgs #33261: inspect.isgeneratorfunction fails on hand-created methods https://bugs.python.org/issue33261 8 msgs #24882: ThreadPoolExecutor doesn't reuse threads until #threads == max https://bugs.python.org/issue24882 6 msgs #33295: ERROR: test_sites_no_connection_close (test.test_urllib2net.Ot https://bugs.python.org/issue33295 6 msgs #11594: 2to3 does not preserve line endings https://bugs.python.org/issue11594 5 msgs #33257: Race conditions in Tkinter with non-threaded Tcl https://bugs.python.org/issue33257 5 msgs #14573: json iterencode can not handle general iterators https://bugs.python.org/issue14573 4 msgs #33131: Upgrade to pip 10 for Python 3.7 https://bugs.python.org/issue33131 4 msgs #33280: Update link to Tcl/Tk 8.6 man pages in tkinter.rst https://bugs.python.org/issue33280 4 msgs Issues closed (31) ================== #11485: Default SDK value on MacOSX needs changing https://bugs.python.org/issue11485 closed by ned.deily #14499: Extension module builds fail with Xcode 4.3 on OS X 10.7 due t https://bugs.python.org/issue14499 closed by ned.deily #19019: Investigate using Apple clang for building OS X installers https://bugs.python.org/issue19019 closed by ned.deily #24414: MACOSX_DEPLOYMENT_TARGET set incorrectly by configure in Pytho https://bugs.python.org/issue24414 closed by ned.deily #32726: macOS installer and framework enhancements and changes for 3.7 https://bugs.python.org/issue32726 closed by ned.deily #33176: Allow memoryview.toreadonly() https://bugs.python.org/issue33176 closed by pitrou #33184: Update OpenSSL to 1.1.0h / 1.0.2o https://bugs.python.org/issue33184 closed by steve.dower #33185: Python 3.7.0b3 fails in pydoc where b2 did not. https://bugs.python.org/issue33185 closed by ncoghlan #33189: pygettext doesn't work with f-strings https://bugs.python.org/issue33189 closed by serhiy.storchaka #33205: GROWTH_RATE prevents dict shrinking https://bugs.python.org/issue33205 closed by inada.naoki #33264: Remove to-be-deprecated urllib.request.urlretrieve function re https://bugs.python.org/issue33264 closed by orsenthil #33269: InteractiveConsole behaves differently on terminal, within scr https://bugs.python.org/issue33269 closed by terry.reedy #33270: tags for anonymous code objects should be interned https://bugs.python.org/issue33270 closed by serhiy.storchaka #33271: Exception handling should match subclasses, not subtypes https://bugs.python.org/issue33271 closed by serhiy.storchaka #33279: Py_BuildValue is causing crashes with the "L" and "K" format c https://bugs.python.org/issue33279 closed by serhiy.storchaka #33283: Mention PNG as a supported image format by Tcl/Tk https://bugs.python.org/issue33283 closed by serhiy.storchaka #33285: pip upgrade runtime crash https://bugs.python.org/issue33285 closed by paul.moore #33287: "pip iinstall packageName.py" fails https://bugs.python.org/issue33287 closed by berker.peksag #33288: Spam https://bugs.python.org/issue33288 closed by zach.ware #33290: Python.org macOS pkg installs pip3 as pip https://bugs.python.org/issue33290 closed by ned.deily #33291: issue in the default setup.py which comes with the Python 3.6. https://bugs.python.org/issue33291 closed by brett.cannon #33292: Fix secrets.randbelow docstring https://bugs.python.org/issue33292 closed by serhiy.storchaka #33293: Using datetime.datetime.utcnow().timestamp() in Python3.6.0 ca https://bugs.python.org/issue33293 closed by ned.deily #33298: Wrap only constants with _PyCode_ConstantKey() in the compiler https://bugs.python.org/issue33298 closed by serhiy.storchaka #33299: Return an object itself for some types in _PyCode_ConstantKey( https://bugs.python.org/issue33299 closed by serhiy.storchaka #33304: Syntax Error on leading 0 in integer tokens https://bugs.python.org/issue33304 closed by steven.daprano #33307: socket.send() fails to send large amount of bytes https://bugs.python.org/issue33307 closed by christian.heimes #33308: parser.st2list(..., col_info=True) triggers a SystemError https://bugs.python.org/issue33308 closed by serhiy.storchaka #33310: Update references to PIL-style arrays https://bugs.python.org/issue33310 closed by skrah #33313: pwritev, preadv and posix_spawn are missing from "What's new i https://bugs.python.org/issue33313 closed by serhiy.storchaka #33320: Clarify who can add the "skip issue" label on GitHub https://bugs.python.org/issue33320 closed by adelfino From christoph at grothesque.org Fri Apr 20 12:17:34 2018 From: christoph at grothesque.org (Christoph Groth) Date: Fri, 20 Apr 2018 18:17:34 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: (Chris Barker's message of "Fri, 20 Apr 2018 08:07:22 -0700") References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> Message-ID: <87h8o5x36p.fsf@grothesque.org> Chris Barker - NOAA Federal wrote: > > Personally, I even slightly prefer > > > > a := 3 > > > > to the commonplace > > > > a = 3 > > because it visually expresses the asymmetry of the operation. > > Careful here! That?s a fine argument for using := in a new language, > but people using := when they don?t need an expression because they > like the symbol better is a reason NOT to do this. Perhaps you are right and it is indeed unrealistic to expect people to (eventually) shift to using := for simple assignments after 28 years of Python... Then I think it would be also OK to introduce a fully general ":=" but discourage its use in assignment statements. However, it seems strange to forbid the use of one expression (namely ":=") as a statement while all other expressions are allowed. (So there seems no alternative to accepting both = and := in statements, and if I understand you correctly you consider this a problem.) One way or the other, I'd like to underline a point that I made yesterday: I believe that it's important for sanity that taking any existing assignment statement and replacing all occurrences of "=" by ":=" does not have any effect on the program. PEP 572 currently proposes to make ":=" a binary operator that is evaluated from right to left. Christoph From solipsis at pitrou.net Fri Apr 20 12:25:18 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 20 Apr 2018 18:25:18 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> Message-ID: <20180420182518.58c2e2de@fsol> On Fri, 20 Apr 2018 13:25:02 +0200 Christoph Groth wrote: > > I think that this is the crucial point. If it is indeed a design > principle of Python that expressions should not have the side-effect of > assigning names, than the whole discussion of PEP 572 could have been > stopped early on. I didn't have this impression since core devs > participated constructively in the discussion. python-dev and python-ideas are two different mailing-lists with different participants, so there's a selection bias here. People who are willing follow lengthy python-ideas discussions may be more tolerant to adventurous language changes. Many core developers (including myself) don't read python-ideas routinely. Regards Antoine. From python-dev at mgmiller.net Fri Apr 20 12:50:29 2018 From: python-dev at mgmiller.net (Mike Miller) Date: Fri, 20 Apr 2018 09:50:29 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> Message-ID: <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> On 2018-04-19 23:52, Chris Angelico wrote: > And are limited to conditions that check the truthiness/falsiness of > the value you care about. So that works for re.match, but not for > anything that might return -1 (a lot of C APIs do that, so if you're > working with a thin wrapper, that might be all you get), and it'll > encourage people to use this form when "is not None" would be more > appropriate (setting up for a failure if ever the API returned a From the previously discussed code, it might look like this: while (file.get_next_token() as token) != -1: doc += token Shouldn't be needed often, but I find it readable enough. More generally, I've been -0 on this idea because I've come to appreciate Python's less-clever i.e. "dumb" loop syntax, and ":=" combined with assignment-expressions doesn't feel like Python at all but rather Pascal and C had a love-child, haha. I could mildly support the "as" syntax however, since it is so darn readable and has analogues in other places. That leaves what to do with "with". Guess I missed the part in the discussion where we couldn't fit the syntax into it. Would requiring parens here not work? with (expr() as name) as conman: pass This should rarely be necessary or useful, correct? Perhaps disallow for now. On assignment to names/subscripts, just names sounds simpler for the first round. Also the current "while" itself could be a bit simpler by making the expression optional and slightly less verbose: while: points = learner.get(static_hint) if not points: break Thanks for the hard work, -Mike From christoph at grothesque.org Fri Apr 20 14:15:04 2018 From: christoph at grothesque.org (Christoph Groth) Date: Fri, 20 Apr 2018 20:15:04 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: (Nick Coghlan's message of "Sat, 21 Apr 2018 01:58:11 +1000") References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> Message-ID: <87bmedwxqv.fsf@grothesque.org> Nick Coghlan wrote: > I also think that if "=" and ":=" both target the same kind of scope, > there isn't enough new expressiveness introduced by the latter to > justify the syntactic complexity of adding it. OK, but then how about introducing assignment expressions with the "=" operator but *requiring* extra parens (similar to how modern C compilers warn about assignment expressions without parens), e.g. while (obj = get()): process(obj) The semantics of assignment expressions could be exactly what I proposed for ":=", i.e. completely consistent with assignment statements. Assignment statements could be either left as they are or could be treated as expressions. That second choice would have consequences for interactive sessions: >>> a = 3 3 ---------------- The above would bring the benefits of assignment expressions in a minimally invasive but safe way. Moreover, it would not feel like Pascal! The only downside is that "=" stands out less than ":=" so that the presence of side-effects would be somewhat less visible. Christoph From ethan at stoneleaf.us Fri Apr 20 14:22:21 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 20 Apr 2018 11:22:21 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <87bmedwxqv.fsf@grothesque.org> References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> <87bmedwxqv.fsf@grothesque.org> Message-ID: <5ADA2FDD.6030307@stoneleaf.us> On 04/20/2018 11:15 AM, Christoph Groth wrote: > Nick Coghlan wrote: > >> I also think that if "=" and ":=" both target the same kind of scope, >> there isn't enough new expressiveness introduced by the latter to >> justify the syntactic complexity of adding it. > > OK, but then how about introducing assignment expressions with the "=" > operator but *requiring* extra parens (similar to how modern C compilers > warn about assignment expressions without parens), e.g. Using a single "=" for assignment expressions isn't going to happen. Period. -- ~Ethan~ From christoph at grothesque.org Fri Apr 20 14:31:14 2018 From: christoph at grothesque.org (Christoph Groth) Date: Fri, 20 Apr 2018 20:31:14 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <5ADA2FDD.6030307@stoneleaf.us> Message-ID: <877ep1wwzx.fsf@grothesque.org> Ethan Furman wrote: > On 04/20/2018 11:15 AM, Christoph Groth wrote: > > Nick Coghlan wrote: > > > >> I also think that if "=" and ":=" both target the same kind of scope, > >> there isn't enough new expressiveness introduced by the latter to > >> justify the syntactic complexity of adding it. > > > > OK, but then how about introducing assignment expressions with the "=" > > operator but *requiring* extra parens (similar to how modern C > > compilers warn about assignment expressions without parens), e.g. > > Using a single "=" for assignment expressions isn't going to happen. > Period. Huh, I didn't want to irritate anyone! Guido wrote [1] on python-ideas: > I also think it's fair to at least reconsider adding inline > assignment, with the "traditional" semantics (possibly with mandatory > parentheses). This would be easier to learn and understand for people > who are familiar with it from other languages (C++, Java, JavaScript). I interpreted this in the way that he at least doesn't rule out "= with parens" completely. Perhaps he meant ":= with parens", but that would seem redundant. [1] https://mail.python.org/pipermail/python-ideas/2018-March/049409.html From ethan at stoneleaf.us Fri Apr 20 14:47:17 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 20 Apr 2018 11:47:17 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <877ep1wwzx.fsf@grothesque.org> References: <877ep1wwzx.fsf@grothesque.org> Message-ID: <5ADA35B5.2010103@stoneleaf.us> On 04/20/2018 11:31 AM, Christoph Groth wrote: > Ethan Furman wrote: >> On 04/20/2018 11:15 AM, Christoph Groth wrote: >>> OK, but then how about introducing assignment expressions with the "=" >>> operator but *requiring* extra parens (similar to how modern C >>> compilers warn about assignment expressions without parens), e.g. >> >> Using a single "=" for assignment expressions isn't going to happen. >> Period. > > Huh, I didn't want to irritate anyone! No worries. It's just not going to happen. ;) > Guido wrote [1] on python-ideas: > >> I also think it's fair to at least reconsider adding inline >> assignment, with the "traditional" semantics (possibly with mandatory >> parentheses). This would be easier to learn and understand for people >> who are familiar with it from other languages (C++, Java, JavaScript). > > I interpreted this in the way that he at least doesn't rule out "= with > parens" completely. Perhaps he meant ":= with parens", but that would > seem redundant. Ah. I believe he was referring to not having a statement-local binding, but a normal binding instead (so either local or global depending on where the expression occurred). -- ~Ethan~ From guido at python.org Fri Apr 20 14:55:22 2018 From: guido at python.org (Guido van Rossum) Date: Fri, 20 Apr 2018 11:55:22 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <5ADA35B5.2010103@stoneleaf.us> References: <877ep1wwzx.fsf@grothesque.org> <5ADA35B5.2010103@stoneleaf.us> Message-ID: Christoph's interpretation is correct. I don't rule that out. I also separately proposed := as something that more people could get behind, though perhaps it's all moot, and perhaps the PEP would gain clarity if it went back to proposing "=". (Mostly kidding.) On Fri, Apr 20, 2018 at 11:47 AM, Ethan Furman wrote: > On 04/20/2018 11:31 AM, Christoph Groth wrote: > >> Ethan Furman wrote: >> >>> On 04/20/2018 11:15 AM, Christoph Groth wrote: >>> >> > OK, but then how about introducing assignment expressions with the "=" >>>> operator but *requiring* extra parens (similar to how modern C >>>> compilers warn about assignment expressions without parens), e.g. >>>> >>> >>> Using a single "=" for assignment expressions isn't going to happen. >>> Period. >>> >> >> Huh, I didn't want to irritate anyone! >> > > No worries. It's just not going to happen. ;) > > > Guido wrote [1] on python-ideas: >> >> I also think it's fair to at least reconsider adding inline >>> assignment, with the "traditional" semantics (possibly with mandatory >>> parentheses). This would be easier to learn and understand for people >>> who are familiar with it from other languages (C++, Java, JavaScript). >>> >> >> I interpreted this in the way that he at least doesn't rule out "= with >> parens" completely. Perhaps he meant ":= with parens", but that would >> seem redundant. >> > > Ah. I believe he was referring to not having a statement-local binding, > but a normal binding instead (so either local or global depending on where > the expression occurred). > > -- > ~Ethan~ > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido% > 40python.org > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Fri Apr 20 15:03:02 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 21 Apr 2018 05:03:02 +1000 Subject: [Python-Dev] PEP 572: Now with 25% less reference implementation! In-Reply-To: References: Message-ID: On Sat, Apr 21, 2018 at 1:50 AM, Guido van Rossum wrote: > Maybe annotations should get a brief mention in the Rejected Ideas section, > with your explanation here added. (And maybe my response.) Good idea. Actually, since it's a syntactic difference, I've promoted it to the "Differences" section. ChrisA From rosuav at gmail.com Fri Apr 20 15:38:52 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 21 Apr 2018 05:38:52 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <87h8o5x36p.fsf@grothesque.org> References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> <87h8o5x36p.fsf@grothesque.org> Message-ID: On Sat, Apr 21, 2018 at 2:17 AM, Christoph Groth wrote: > Chris Barker - NOAA Federal wrote: > >> > Personally, I even slightly prefer >> > >> > a := 3 >> > >> > to the commonplace >> > >> > a = 3 >> > because it visually expresses the asymmetry of the operation. >> >> Careful here! That?s a fine argument for using := in a new language, >> but people using := when they don?t need an expression because they >> like the symbol better is a reason NOT to do this. > > Perhaps you are right and it is indeed unrealistic to expect people to > (eventually) shift to using := for simple assignments after 28 years of > Python... It's not just 28 years of Python. It's also that other languages use "=" for assignment. While this is by no means a clinching argument, it does have some weight; imagine if Python used "=" for comparison and ":=" for assignment - anyone who works simultaneously with multiple languages is going to constantly type the wrong operator. (I get this often enough with comment characters, but my editor will usually tell me straight away if I type "// blah" in Python, whereas it won't always tell me that I used "x = 1" when I wanted one of the other forms.) > One way or the other, I'd like to underline a point that I made > yesterday: I believe that it's important for sanity that taking any > existing assignment statement and replacing all occurrences of "=" by > ":=" does not have any effect on the program. > > PEP 572 currently proposes to make ":=" a binary operator that is > evaluated from right to left. This is one of the points that I was halfway through working on when I finally gave up on working on a reference implementation for a likely-doomed PEP. It might be possible to make := take an entire sequence of assignables and then set them left to right; however, this would be a lot more complicated, and I'm not even sure I want that behaviour. I don't want to encourage people to replace all "=" with ":=" just for the sake of it. The consistency is good if it can be achieved, but you shouldn't actually DO that sort of thing normally. Consider: one of the important reasons to define the assignment order is so you can reference a subscript and also use it. For instance: idx, items[idx] = new_idx, new_val But you don't need that with :=, because you can: items[idx := new_idx] = new_val (and you can use := for the second one if you wish). And actually, this one wouldn't even change, because it's using tuple unpacking, not the assignment order of chained assignments. I cannot think of any situation where you'd want to write this: idx = items[idx] = f() inside an expression, and thus need to write it as: g(items[idx] := idx := f()) So I have no problem with a style guide saying "yeah just don't do that", and the PEP saying "if you do this, the semantics won't be absolutely identical to '='". Which it now does. Now, if someone else wants to work on the reference implementation, they're welcome to create this feature and then see whether they like it. But since I can't currently prove it's possible, I'm not going to specify it in the PEP. ChrisA From rosuav at gmail.com Fri Apr 20 15:43:19 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 21 Apr 2018 05:43:19 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> References: <874lk6bwv2.fsf@grothesque.org> <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> Message-ID: On Sat, Apr 21, 2018 at 2:50 AM, Mike Miller wrote: > > On 2018-04-19 23:52, Chris Angelico wrote: >> >> And are limited to conditions that check the truthiness/falsiness of >> the value you care about. So that works for re.match, but not for >> anything that might return -1 (a lot of C APIs do that, so if you're >> working with a thin wrapper, that might be all you get), and it'll >> encourage people to use this form when "is not None" would be more >> appropriate (setting up for a failure if ever the API returned a > > > From the previously discussed code, it might look like this: > > while (file.get_next_token() as token) != -1: > doc += token Except that that's now a feature of expressions, NOT of the loop construct. And then you're left with: why not permit this everywhere? > That leaves what to do with "with". Guess I missed the part in the > discussion where we couldn't fit the syntax into it. Would requiring parens > here not work? > > with (expr() as name) as conman: > pass > > This should rarely be necessary or useful, correct? Perhaps disallow for > now. What would these mean? from contextlib import closing with open(fn) as f: with (open(fn) as f): with closing(urlopen(url)) as dl: with closing(urlopen(url) as dl): with (closing(urlopen(url)) as dl): One of these is not like the others... > Also the current "while" itself could be a bit simpler by making the > expression optional and slightly less verbose: > > while: > points = learner.get(static_hint) > if not points: > break As an alias for "while True"? Not a lot of benefit. I'd rather do something like this: while "get more learners": which at least tells the next reader that there is a condition, even if not a coded one. ChrisA From mertz at gnosis.cx Fri Apr 20 16:04:20 2018 From: mertz at gnosis.cx (David Mertz) Date: Fri, 20 Apr 2018 20:04:20 +0000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> <87h8o5x36p.fsf@grothesque.org> Message-ID: It's horrors like this: g(items[idx] := idx := f()) That make me maybe +0 if the PEP only allowed simple name targets, but decisively -1 for any assignment target in the current PEP. I would much rather never have to read awful constructs like that than get the minor convenience of: if (val := some_expensive_func()) > 0: x = call_something(val) On Fri, Apr 20, 2018, 3:39 PM Chris Angelico wrote: > On Sat, Apr 21, 2018 at 2:17 AM, Christoph Groth > wrote: > > Chris Barker - NOAA Federal wrote: > > > >> > Personally, I even slightly prefer > >> > > >> > a := 3 > >> > > >> > to the commonplace > >> > > >> > a = 3 > >> > because it visually expresses the asymmetry of the operation. > >> > >> Careful here! That?s a fine argument for using := in a new language, > >> but people using := when they don?t need an expression because they > >> like the symbol better is a reason NOT to do this. > > > > Perhaps you are right and it is indeed unrealistic to expect people to > > (eventually) shift to using := for simple assignments after 28 years of > > Python... > > It's not just 28 years of Python. It's also that other languages use > "=" for assignment. While this is by no means a clinching argument, it > does have some weight; imagine if Python used "=" for comparison and > ":=" for assignment - anyone who works simultaneously with multiple > languages is going to constantly type the wrong operator. (I get this > often enough with comment characters, but my editor will usually tell > me straight away if I type "// blah" in Python, whereas it won't > always tell me that I used "x = 1" when I wanted one of the other > forms.) > > > One way or the other, I'd like to underline a point that I made > > yesterday: I believe that it's important for sanity that taking any > > existing assignment statement and replacing all occurrences of "=" by > > ":=" does not have any effect on the program. > > > > PEP 572 currently proposes to make ":=" a binary operator that is > > evaluated from right to left. > > This is one of the points that I was halfway through working on when I > finally gave up on working on a reference implementation for a > likely-doomed PEP. It might be possible to make := take an entire > sequence of assignables and then set them left to right; however, this > would be a lot more complicated, and I'm not even sure I want that > behaviour. I don't want to encourage people to replace all "=" with > ":=" just for the sake of it. The consistency is good if it can be > achieved, but you shouldn't actually DO that sort of thing normally. > > Consider: one of the important reasons to define the assignment order > is so you can reference a subscript and also use it. For instance: > > idx, items[idx] = new_idx, new_val > > But you don't need that with :=, because you can: > > items[idx := new_idx] = new_val > > (and you can use := for the second one if you wish). And actually, > this one wouldn't even change, because it's using tuple unpacking, not > the assignment order of chained assignments. I cannot think of any > situation where you'd want to write this: > > idx = items[idx] = f() > > inside an expression, and thus need to write it as: > > g(items[idx] := idx := f()) > > So I have no problem with a style guide saying "yeah just don't do > that", and the PEP saying "if you do this, the semantics won't be > absolutely identical to '='". Which it now does. > > Now, if someone else wants to work on the reference implementation, > they're welcome to create this feature and then see whether they like > it. But since I can't currently prove it's possible, I'm not going to > specify it in the PEP. > > ChrisA > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/mertz%40gnosis.cx > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Fri Apr 20 16:12:28 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 21 Apr 2018 06:12:28 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> <87h8o5x36p.fsf@grothesque.org> Message-ID: On Sat, Apr 21, 2018 at 6:04 AM, David Mertz wrote: > It's horrors like this: > > g(items[idx] := idx := f()) > > That make me maybe +0 if the PEP only allowed simple name targets, but > decisively -1 for any assignment target in the current PEP. But that's my point: you shouldn't need to write that. Can anyone give me a situation where that kind of construct is actually useful? Much more common would be to use := inside the square brackets, which makes the whole thing a lot more sane. You can ALWAYS write stupid code. Nobody can or will stop you. ChrisA From guido at python.org Fri Apr 20 16:59:47 2018 From: guido at python.org (Guido van Rossum) Date: Fri, 20 Apr 2018 13:59:47 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> <87h8o5x36p.fsf@grothesque.org> Message-ID: Does the PEP currently propose to *allow* that horrible example? I thought Tim Peters successfully pleaded to *only* allow a single "NAME := ". You don't have to implement this restriction -- we know it's possible to implement, and if specifying this alone were to pull enough people from -1 to +0 there's a lot of hope! On Fri, Apr 20, 2018 at 1:12 PM, Chris Angelico wrote: > On Sat, Apr 21, 2018 at 6:04 AM, David Mertz wrote: > > It's horrors like this: > > > > g(items[idx] := idx := f()) > > > > That make me maybe +0 if the PEP only allowed simple name targets, but > > decisively -1 for any assignment target in the current PEP. > > But that's my point: you shouldn't need to write that. Can anyone give > me a situation where that kind of construct is actually useful? Much > more common would be to use := inside the square brackets, which makes > the whole thing a lot more sane. > > You can ALWAYS write stupid code. Nobody can or will stop you. > > ChrisA > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Fri Apr 20 17:04:01 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 21 Apr 2018 07:04:01 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> <87h8o5x36p.fsf@grothesque.org> Message-ID: On Sat, Apr 21, 2018 at 6:59 AM, Guido van Rossum wrote: > Does the PEP currently propose to *allow* that horrible example? I thought > Tim Peters successfully pleaded to *only* allow a single "NAME := ". > You don't have to implement this restriction -- we know it's possible to > implement, and if specifying this alone were to pull enough people from -1 > to +0 there's a lot of hope! I don't see much value in restricting the assignment target to names only, but if that's what it takes, it can be restricted, at least initially. As to chaining... well, since the entire construct (target := expr) is an expression, it can be used on the right of :=, so short of outright forbidding it, there's not a lot to be done. ChrisA From guido at python.org Fri Apr 20 17:28:02 2018 From: guido at python.org (Guido van Rossum) Date: Fri, 20 Apr 2018 14:28:02 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> <87h8o5x36p.fsf@grothesque.org> Message-ID: On Fri, Apr 20, 2018 at 2:04 PM, Chris Angelico wrote: > On Sat, Apr 21, 2018 at 6:59 AM, Guido van Rossum > wrote: > > Does the PEP currently propose to *allow* that horrible example? I > thought > > Tim Peters successfully pleaded to *only* allow a single "NAME := > ". > > You don't have to implement this restriction -- we know it's possible to > > implement, and if specifying this alone were to pull enough people from > -1 > > to +0 there's a lot of hope! > > I don't see much value in restricting the assignment target to names > only, but if that's what it takes, it can be restricted, at least > initially. All of this is an exercise in listening and compromise, not in solving puzzles. > As to chaining... well, since the entire construct (target > := expr) is an expression, it can be used on the right of :=, so short > of outright forbidding it, there's not a lot to be done. > It would be more work but it can definitely be done (perhaps by introducing a syntactic construct of intermediate precedence). People could write "a := (b := foo())" but that way they resolve the ambiguity. Although if we restrict targets to just names there's less concern about ambiguity. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.peters at gmail.com Fri Apr 20 17:33:04 2018 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 20 Apr 2018 16:33:04 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> <87h8o5x36p.fsf@grothesque.org> Message-ID: [Guido, about g(items[idx] := idx := f()) ] > Does the PEP currently propose to *allow* that horrible example? I thought > Tim Peters successfully pleaded to *only* allow a single "NAME := ". I was "successful" only in that the two of us agreed that would be far less disruptive, and quite possibly an actual improvement ;-) But I only argued for limiting assignment expressions to the form identifier ":=" expression I expected that, given that expressions "naturally nest", chained targets could still be specified: a := b := c:= 5 but since they're all plain names there's no way to tell whether the bindings occur "left to right" or "right to left" short of staring at the generated code. I have no use case for chaining plain-name targets in assignment expressions, but didn't see a good reason to torture the implementation to forbid it. I expected chaining would just be an unused-in-practice possibility. Much like, e.g., a in b in c in d is an unused-in-practice possibility. And I'll take this opportunity to repeat the key point for me: I tried hard, but never found a single case based on staring at real code where allowing _fancier_ (than "plain name") targets would be a real improvement. In every case I thought it _might_ help, it turned out that it really didn't unless Python _also_ grew an analog to C's "comma operator" (take only the last result from a sequence of expressions). I'll also note that I asked if anyone else had a real-life example, and got no responses. There were lots of "real life" cases where plain-name targets allowed for code improvement, though. > You don't have to implement this restriction -- we know it's possible to > implement, and if specifying this alone were to pull enough people from -1 > to +0 there's a lot of hope! Given my experience with _trying_ to find use cases for fancier targets, and getting burned every time, I'm on the minus side of the current PEP, because - best I can tell - all the extra complexity would create an "attractive nuisance" :-( From python-dev at mgmiller.net Fri Apr 20 17:54:35 2018 From: python-dev at mgmiller.net (Mike Miller) Date: Fri, 20 Apr 2018 14:54:35 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> Message-ID: On 2018-04-20 12:43, Chris Angelico wrote: > Except that that's now a feature of expressions, NOT of the loop > construct. And then you're left with: why not permit this everywhere? Sorry, I didn't understand. Didn't mean to imply it couldn't be used everywhere. > What would these mean? My expectations: with open(fn) as f: # current behavior with (open(fn) as f): # syntax error, missing clause with closing(urlopen(url)) as dl: # current behavior with closing(urlopen(url) as dl): # syntax error, missing clause with (closing(urlopen(url)) as dl): # syntax error, missing clause In other words, the with statement would continue to require an as clause outside of the parentheses. A double name binding doesn't seem very useful however. -Mike From jelle.zijlstra at gmail.com Fri Apr 20 17:59:09 2018 From: jelle.zijlstra at gmail.com (Jelle Zijlstra) Date: Fri, 20 Apr 2018 14:59:09 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> Message-ID: 2018-04-20 14:54 GMT-07:00 Mike Miller : > > In other words, the with statement would continue to require an as clause > outside of the parentheses. A double name binding doesn't seem very useful > however. > > The with statement does not require an as clause. -------------- next part -------------- An HTML attachment was scrubbed... URL: From python-dev at mgmiller.net Fri Apr 20 18:07:56 2018 From: python-dev at mgmiller.net (Mike Miller) Date: Fri, 20 Apr 2018 15:07:56 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> Message-ID: <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> On 2018-04-20 14:59, Jelle Zijlstra wrote: > In other words, the with statement would continue to require an as clause > outside of the parentheses. A double name binding doesn't seem very useful > however. > > The with statement does not require an as clause. Sorry, more precisely a contenxt-manager object to be returned. So perhaps this "with" issue may not be one at all. From python at mrabarnett.plus.com Fri Apr 20 19:17:50 2018 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 21 Apr 2018 00:17:50 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> <87h8o5x36p.fsf@grothesque.org> Message-ID: <70f57020-7d99-a401-d273-00587422256e@mrabarnett.plus.com> On 2018-04-20 22:33, Tim Peters wrote: [snip] > And I'll take this opportunity to repeat the key point for me: I > tried hard, but never found a single case based on staring at real > code where allowing _fancier_ (than "plain name") targets would be a > real improvement. In every case I thought it _might_ help, it turned > out that it really didn't unless Python _also_ grew an analog to C's > "comma operator" (take only the last result from a sequence of > expressions). I'll also note that I asked if anyone else had a > real-life example, and got no responses. > Could a semicolon in a parenthesised expression be an equivalent to C's "comma operator"? [snip] From rosuav at gmail.com Fri Apr 20 20:15:08 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 21 Apr 2018 10:15:08 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> References: <874lk6bwv2.fsf@grothesque.org> <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> Message-ID: On Sat, Apr 21, 2018 at 8:07 AM, Mike Miller wrote: > On 2018-04-20 14:59, Jelle Zijlstra wrote: >> In other words, the with statement would continue to require an as >> clause >> outside of the parentheses. A double name binding doesn't seem very >> useful >> however. >> >> The with statement does not require an as clause. > > Sorry, more precisely a contenxt-manager object to be returned. So perhaps > this "with" issue may not be one at all. That's completely different, and isn't a syntactic point. They may bomb with AttributeError at run time, but they also may not. > My expectations: > > with open(fn) as f: # current behavior > with (open(fn) as f): # syntax error, missing clause > with closing(urlopen(url)) as dl: # current behavior > with closing(urlopen(url) as dl): # syntax error, missing clause > with (closing(urlopen(url)) as dl): # syntax error, missing clause The second and fifth could be special cased as either the same as first and third, or as SyntaxErrors. (But which?) The fourth one is very tricky. If 'expr as name' is allowed inside arbitrary expressions, why shouldn't it be allowed there? The disconnect between viable syntax and useful statements is problematic here. ChrisA From anthony.flury at btinternet.com Fri Apr 20 21:38:04 2018 From: anthony.flury at btinternet.com (Anthony Flury) Date: Sat, 21 Apr 2018 02:38:04 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> Message-ID: I am entirely new to this list, but if I can I would like share my comments : * I do think this proposal := has merit in my opinion; it does make some code more readable. * I think readability is only improved if : * target is restricted to a simple name - I don't see a benefit in more complex targets * chaining is not allowed - I think the construct : ??? ??? ??? while (line := input.read_row()) is not None: ??? ??? ??? ??? ??? process_line(line) ??? ??? Is readable, but : ??? ??? ??? while (current_line := line := input.read_row()) is not None: ??? ??? ??? ??? ??? line = process_line(line) ????? is not obvious - and certainly isn't any more obvious than : ??? ??? ??? while (line := input.read_row()) is not None: ??? ??? ??? ??? ? ? current_line = line ?? ??? ??? ??? ???? line = process_line(line) * The current expectations of how comprehensions work should also be honored; I don't claim to have fully followed all of the discussions around this, but it seems to me that comprehensions work in a particular way because of a concerted effect (especially in Python 3) to make them that way. They are self contained and don't leak values in their containing scope. Similarly I think that setting variables within a comprehension is just for the benefit of readable code within the comprehension - i.e. : stuff = [[y, x/y] for x in range(5) for y in [f(x)]] ??? ??? ??? ??? can become : stuff = [[y := f(x), x/y] for x in range(5)] So - overall from me a conditional +1 - conditions as above; if they are not possible then -1 from me. -- Anthony Flury email : *Anthony.flury at btinternet.com* Twitter : *@TonyFlury * From tim.peters at gmail.com Fri Apr 20 21:44:46 2018 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 20 Apr 2018 20:44:46 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> <87h8o5x36p.fsf@grothesque.org> Message-ID: [Chris Angelico ] > I don't see much value in restricting the assignment target to names > only, but if that's what it takes, it can be restricted, at least > initially. I believe this point was made most clearly before by Terry Reedy, but it bears repeating :-) This is from the PEP's motivation: """ Naming the result of an expression is an important part of programming, allowing a descriptive name to be used in place of a longer expression, and permitting reuse. """ As "head arguments" go, that's a good one! But restricting assignment expressions to identifier ":=" expression satisfies it. If what's of value is to name the result of an expression, that single case handles that and _only_ that. In a sense, it's "the simplest thing that could possibly work", and that's generally a good thing to aim for. Python assignment _statements_ are way more complex than that. Besides just giving names to expression results, they can also implicitly invoke arbitrarily complex __setitem__ and __setattr__ methods on targets, rely on all sorts of side effects across chained assignments, and support funky syntax for magically iterating over an expression's iterable result. While that can all be useful _in_ an assignment statement, the PEP's motivation doesn't say a word about why any of _that_ would also be useful buried inside an assignment expression. There doesn't appear to be a good "head argument" for why, besides "why not?". That's not enough. I think it's no coincidence that every example of an _intended_ use is of the simple identifier ":=" expression form. There are no examples of fancier targets in the PEP, and - more importantly - also none I saw in the hundreds of mailing-list messages since this started. Except for a few of mine, where I tried to demonstrate why _trying_ fancier targets in examples derived from real code made the original "loop and a half" code _worse_ And where other people were illustrating how incomprehensibly code _could_ be written (which isn't a real interest of mine). Short course: e.g., while a general assignment expression can "unpack" an iterable expression result, giving names to its elements, there's no clean way to _use_ the names bound by the unpacking _in_ the "if" or "while" tests. That's fine for `for` loops (only the _body_ of the loop needs the names), but in conditional constructs you typically want to use the names _in_ the condition being tested. if ((a, b, c) := func_returning_triple()) and b > 0: process(a+b, b+c, a+c) seems to be as good as it gets, but inherently relies on "a trick": that a 3-tuple is always truthy, regardless of content. OTOH, if ((a, b, c) := func_returning_triple())[1] > 0: doesn't rely on a trick, but can't use the name `b` in the test(!). if [((a, b, c) := func_returning_triple()), b > 0][-1]:: manages to avoid "a trick", and to use the natural `b > 0`, but is ... strained ;-) So, to my eyes, this is a clear improvement over all of those: a, b, c = func_returning_triple() if b > 0: process(a+b, b+c, a+c) Of course I could be cherry-picking a bad example there, but that's not the intent: I'm still waiting for anyone to post an example where a "fancy" assignment-expression target would actually make code clearer. I haven't found one. There are lots of examples when the target is a plain single name. Why the stark difference? I don't need deep theoretical reasons to see that there _is_ one, or to conclude that - in the absence of compelling use cases - complex assignment-expression targets are probably a Poor Idea. From brett at python.org Fri Apr 20 17:52:11 2018 From: brett at python.org (Brett Cannon) Date: Fri, 20 Apr 2018 21:52:11 +0000 Subject: [Python-Dev] Introducing python.zulipchat.com Message-ID: As an experiment we have gotten an instance of Zulip running for Python's development at https://python.zulipchat.com (IOW this is for discussing the development *of* Python only*)*. As Guido has put it you can view Zulip like "hyper-interactive email" as we have streams corresponding to equivalent mailing lists and all messages fall under a topic so conversations are on-topic. The instance is currently rather open at the suggestion of Zulip, so people can create new streams, add bots, etc. There are already bots I have added for commit notifications, Travis failures, and deployments of Bedevere and The Knights Who Say Ni. The invite message when you sign up mentions all this, the CoC, etc., so do give it a read. Otherwise feel free to join and help us decide if this is useful enough to make a permanent thing for Python's development! -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.peters at gmail.com Fri Apr 20 22:15:32 2018 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 20 Apr 2018 21:15:32 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <70f57020-7d99-a401-d273-00587422256e@mrabarnett.plus.com> References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> <87h8o5x36p.fsf@grothesque.org> <70f57020-7d99-a401-d273-00587422256e@mrabarnett.plus.com> Message-ID: [Tim] >> And I'll take this opportunity to repeat the key point for me: I >> tried hard, but never found a single case based on staring at real >> code where allowing _fancier_ (than "plain name") targets would be a >> real improvement. In every case I thought it _might_ help, it turned >> out that it really didn't unless Python _also_ grew an analog to C's >> "comma operator" (take only the last result from a sequence of >> expressions). I'll also note that I asked if anyone else had a >> real-life example, and got no responses. [MRAB ] > Could a semicolon in a parenthesised expression be an equivalent to C's > "comma operator"? I expect it could, but I it's been many years since I tried hacking Python's grammar, and I wouldn't want a comma operator anyway ;-) To recycle a recently-posted example, instead of one of these 3: if ((a, b, c) := func_returning_triple()) and b > 0: process(a+b, b+c, a+c) if ((a, b, c) := func_returning_triple())[1] > 0: .... if [((a, b, c) := func_returning_triple()), b > 0][-1]:: ... it would allow this instead: if ((a, b, c) := func_returning_triple(); b > 0): ... That's better than any of the first three, but I'm not sure it's better than the original a, b, c = func_returning_triple() if b > 0: ... It _may_ be more readable in other complex-target examples, though. It's also what's wanted in one of the running plain-name target examples, _not_ involving a conditional context: r1, r2 = (D := sqrt(b**-4*a*c); a2 := 2*a; ((-b+D)/a2), (-b-D)/a2)) And if I saw enough code like that, I'd write a PEP suggesting that Python introduce separate assignment statements where name bindings persisted across statement boundaries ;-) From rosuav at gmail.com Fri Apr 20 22:30:36 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 21 Apr 2018 12:30:36 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> Message-ID: On Sat, Apr 21, 2018 at 11:38 AM, Anthony Flury via Python-Dev wrote: > I am entirely new to this list, but if I can I would like share my comments > : > > * I do think this proposal := has merit in my > opinion; it does make some code more readable. > > * I think readability is only improved if : There's that word "readability" again. Sometimes I wish the Zen of Python didn't use it, because everyone seems to think that "readable" means "code I like". > * The current expectations of how comprehensions work should also be > honored; I don't claim to have fully followed all of the discussions > around this, but it seems to me that comprehensions work in a > particular way because of a concerted effect (especially in Python > 3) to make them that way. They are self contained and don't leak > values in their containing scope. Similarly I think that setting > variables within a comprehension is just for the benefit of readable > code within the comprehension - i.e. : > > stuff = [[y, x/y] for x in range(5) for y in [f(x)]] > > can become : > > stuff = [[y := f(x), x/y] for x in range(5)] > > So - overall from me a conditional +1 - conditions as above; if they are not > possible then -1 from me. Perfectly self-contained. They do everything in their own scope. Except ... except that they don't. $ python3.7 Python 3.7.0a4+ (heads/master:95e4d58913, Jan 27 2018, 06:21:05) [GCC 6.3.0 20170516] on linux Type "help", "copyright", "credits" or "license" for more information. >>> class X: ... x = ["spam", "ham"] ... print(["1: "+x for x in x]) ... print(["2: "+x for _ in range(1) for x in x]) ... ['1: spam', '1: ham'] Traceback (most recent call last): File "", line 1, in File "", line 4, in X File "", line 4, in UnboundLocalError: local variable 'x' referenced before assignment >>> class Y: ... prefix = "3: " ... print([prefix + x for x in ["spam", "ham"]]) ... Traceback (most recent call last): File "", line 1, in File "", line 3, in Y File "", line 3, in NameError: name 'prefix' is not defined Introducing expression assignments will make these oddities even more obvious. You'd be able to demonstrate things like this at function scope, not just with a class. But the oddities are there, and they are inherent to the current definition of a comprehension. That's why the changes are being recommended. They will simplify the peculiarities of comprehensions, make them far closer to a naive transformation into longhand, and extremely close to a non-naive-but-still-simple transformation into longhand. ChrisA From ncoghlan at gmail.com Sat Apr 21 02:17:42 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 21 Apr 2018 16:17:42 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> <87h8o5x36p.fsf@grothesque.org> Message-ID: On 21 April 2018 at 07:33, Tim Peters wrote: > I expected that, given that expressions "naturally nest", chained > targets could still be specified: > > a := b := c:= 5 > > but since they're all plain names there's no way to tell whether the > bindings occur "left to right" or "right to left" short of staring at > the generated code. The fact class namespaces are ordered by default now allow us to demonstrate the order of multiple target assignments and tuple unpacking without staring at generated code: >>> class AssignmentOrder: ... a = b = c = 0 ... d, e, f = range(3) ... >>> class ReversedAssignmentOrder: ... c = b = a = 0 ... f, e, d = range(3) ... >>> [attr for attr in AssignmentOrder.__dict__ if not attr.startswith("_")] ['a', 'b', 'c', 'd', 'e', 'f'] >>> [attr for attr in ReversedAssignmentOrder.__dict__ if not attr.startswith("_")] ['c', 'b', 'a', 'f', 'e', 'd'] So that's a situation where "name = alias = value" could end up matching "alias := name := value" (Even in earlier versions, you can illustrate the same assignment ordering behaviour with the enum module, and there it makes even more of a difference, as it affects which name binding is considered the canonical name, and which are considered aliases). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From steve at pearwood.info Sat Apr 21 03:11:18 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 21 Apr 2018 17:11:18 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> Message-ID: <20180421071117.GW11616@ando.pearwood.info> On Sat, Apr 21, 2018 at 12:30:36PM +1000, Chris Angelico wrote: > There's that word "readability" again. Sometimes I wish the Zen of > Python didn't use it, because everyone seems to think that "readable" > means "code I like". In fairness, if one can't read code, then one can hardly be expected to like it. But there's plenty of code I can read that I don't like. However your point still stands: since we don't have an objective and empirical measure of readability, we cannot help but be subjective about it. And with such subjective judgements, it is very, very hard to divorce personal opinions about what we like from subjective estimates of how readable something is. [...] > Perfectly self-contained. They do everything in their own scope. > Except ... except that they don't. [...] They being comprehensions inside class scopes. Class scopes are already a bit weird, and don't *quite* work the same as non-class scopes even without introducing comprehensions: py> class X: ... a = 99 ... b = lambda: a+1 ... c = b() ... Traceback (most recent call last): File "", line 1, in File "", line 4, in X File "", line 3, in NameError: name 'a' is not defined I get bitten by this all the time. No, I tell a lie: I *hardly ever* get bitten by this. Judging by the number of questions about it on StackOverflow and the Python-List and Tutor mailing lists, I'd say that I'm not unusual here. > Introducing expression assignments will make these oddities even more > obvious. You'd be able to demonstrate things like this at function > scope, not just with a class. In what way? And are you absolutely sure they will be oddities? To give an analogy, I don't think this is an oddity: py> def func(a=1, b=a+1): ... pass ... Traceback (most recent call last): File "", line 1, in NameError: name 'a' is not defined If anyone expected that the default value for b could make use of the default value for a, the answer is: no, Python function declarations don't work that way. Maybe they could, if we wanted them to, but we don't, so they don't. So can you explain specifically what odd function-scope behaviour you are referring to? Give an example please? -- Steve From rosuav at gmail.com Sat Apr 21 03:46:44 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 21 Apr 2018 17:46:44 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <20180421071117.GW11616@ando.pearwood.info> References: <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <20180421071117.GW11616@ando.pearwood.info> Message-ID: On Sat, Apr 21, 2018 at 5:11 PM, Steven D'Aprano wrote: > On Sat, Apr 21, 2018 at 12:30:36PM +1000, Chris Angelico wrote: >> Introducing expression assignments will make these oddities even more >> obvious. You'd be able to demonstrate things like this at function >> scope, not just with a class. > > In what way? > > And are you absolutely sure they will be oddities? To give an analogy, I > don't think this is an oddity: > > py> def func(a=1, b=a+1): > ... pass > ... > Traceback (most recent call last): > File "", line 1, in > NameError: name 'a' is not defined > > If anyone expected that the default value for b could make use of the > default value for a, the answer is: no, Python function declarations > don't work that way. Maybe they could, if we wanted them to, but we > don't, so they don't. > > So can you explain specifically what odd function-scope behaviour you > are referring to? Give an example please? doubled_items = [x for x in (items := get_items()) if x * 2 in items] This will leak 'items' into the surrounding scope (but not 'x'). [x for x in x if x] # This works [x for y in x if x := y] # UnboundLocalError (x for x in 5) # TypeError (x for _ in [1] for x in 5) # Works I'm sure you can come up with more examples. The outermost iterable is special and magical. ChrisA From ncoghlan at gmail.com Sat Apr 21 03:52:19 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 21 Apr 2018 17:52:19 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <20180421071117.GW11616@ando.pearwood.info> References: <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <20180421071117.GW11616@ando.pearwood.info> Message-ID: On 21 April 2018 at 17:11, Steven D'Aprano wrote: > So can you explain specifically what odd function-scope behaviour you > are referring to? Give an example please? Once we allow name binding as an expression, there are three main cases to consider in comprehensions: 1. Name binding in the result expression 2. Name binding a filter expression 3. Name binding in an iterable expression The first two cases are fine (they happen in the implicit nested scope, and hence don't affect the scope containing the comprehension), but the behaviour in the third case bothered people, because it broke down into two distinct subcases: 3a. For the outermost iterable, the binding always happens in the surrounding scope, and hence will not be accessible from the rest of the comprehension when used at class scope. 3b. For any nested iterables, the binding happens in the implicit nested scope, as for other comprehension subexpressions In the original version of PEP 572 (the one with sublocal scopes), the consequences of 3a were just that you couldn't meaningfully use assignment expressions in the outermost iterable expression of a comprehension, since neither the implicitly nested scope nor the surrounding scope would be able to see them. Too bad, so sad, don't do that then (since it's pointless). In the revised version of PEP 572 that just used regular local assignment, the side effects of 3a were more concerning, since they meant that we'd be bringing back the comprehension variable leakage problem, albeit in a far more esoteric form. Even less defensibly, the construct would just work at function scope, work, but define an unexpected module attribute at module scope, and simply not work at all at class scope. Hence the changes to the PEP to move even the evaluation of the outermost iterable inside the implicitly nested scope, rather than leaving it outside the way it is now. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From p.f.moore at gmail.com Sat Apr 21 04:49:07 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 21 Apr 2018 09:49:07 +0100 Subject: [Python-Dev] Introducing python.zulipchat.com In-Reply-To: References: Message-ID: On 20 April 2018 at 22:52, Brett Cannon wrote: > As an experiment we have gotten an instance of Zulip running for Python's > development at https://python.zulipchat.com (IOW this is for discussing the > development of Python only). As Guido has put it you can view Zulip like > "hyper-interactive email" as we have streams corresponding to equivalent > mailing lists and all messages fall under a topic so conversations are > on-topic. > > The instance is currently rather open at the suggestion of Zulip, so people > can create new streams, add bots, etc. There are already bots I have added > for commit notifications, Travis failures, and deployments of Bedevere and > The Knights Who Say Ni. > > The invite message when you sign up mentions all this, the CoC, etc., so do > give it a read. Otherwise feel free to join and help us decide if this is > useful enough to make a permanent thing for Python's development! Just a usability note - the sign in procedure seems very weird. I tried to log in, but didn't want to create another independent account so I tried "Sign in with Google" and "Sign in with Github". Both took me round a loop of authorising the access, then said "no account found for your email address" and I ended up back at the sign in page. I'm not sure what I did wrong, but the first time I did this, I just gave up. This time, I've sent this message... Paul From anthony.flury at btinternet.com Sat Apr 21 04:38:04 2018 From: anthony.flury at btinternet.com (Anthony Flury) Date: Sat, 21 Apr 2018 09:38:04 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <20180421071117.GW11616@ando.pearwood.info> Message-ID: <6281453e-6468-3b44-b237-d1147d02f51f@btinternet.com> On 21/04/18 08:46, Chris Angelico wrote: > doubled_items = [x for x in (items := get_items()) if x * 2 in items] > > This will leak 'items' into the surrounding scope (but not 'x'). At the risk of stating the obvious - wasn't there work in Python 3 to prevent leakage from comprehensions ? > [x for x in x if x] # This works > [x for y in x if x := y] # UnboundLocalError The standard library example given earlier notwithstanding, I can see no benefit in using the same name as the iterator and the loop target name. To be honest I have trouble parsing that first version, and keeping track of which x is which (especially which x is being used in the conditional clause) : surely this would be better : [x_item for x_item in x if x_item] Your 2nd example makes no sense to me as to the intention of the code - the re-use of the name x is confusing at best. -- Anthony Flury email : *Anthony.flury at btinternet.com* Twitter : *@TonyFlury * From jcgoble3 at gmail.com Sat Apr 21 04:55:02 2018 From: jcgoble3 at gmail.com (Jonathan Goble) Date: Sat, 21 Apr 2018 08:55:02 +0000 Subject: [Python-Dev] Introducing python.zulipchat.com In-Reply-To: References: Message-ID: On Sat, Apr 21, 2018, 4:51 AM Paul Moore wrote: > Just a usability note - the sign in procedure seems very weird. I > tried to log in, but didn't want to create another independent account > so I tried "Sign in with Google" and "Sign in with Github". Both took > me round a loop of authorising the access, then said "no account found > for your email address" and I ended up back at the sign in page. > > I'm not sure what I did wrong, but the first time I did this, I just > gave up. This time, I've sent this message... > Try clicking on "Sign up" below those buttons to get the "sign up with Google/GitHub" buttons. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Sat Apr 21 05:12:56 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 21 Apr 2018 10:12:56 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> Message-ID: On 21 April 2018 at 03:30, Chris Angelico wrote: > There's that word "readability" again. Sometimes I wish the Zen of > Python didn't use it, because everyone seems to think that "readable" > means "code I like". Readability is subjective, yes. But it's not the same as "liking". If a significant number of people say that they find a piece of code hard to read/understand, then that's a problem. It's all too easy to say "you don't have to write code like that", but as someone who has been a maintenance programmer for his whole career, I can tell you that people don't always have that luxury. And supporting code that's written in a language that prioritises "readability" (whatever that may mean) is a much easier task than supporting code written in a language that doesn't. There's a reason far fewer people write systems in Perl these days, and it's not because you can't write clear and maintainable code in Perl... I think that enough people have flagged up readability concerns, that the PEP should take that seriously. One improvement would be to limit the proposal to assignment of simple names (there have been far fewer complaints about readability for such examples). Another would be to simply address the concern more seriously than the current "This can be used to create ugly code" section. How about a heading like "Code that uses this construct could be difficult to maintain", with a discussion that acknowledges that maintenance programmers usually didn't write the code, and sometimes don't have the freedom to rewrite working code. It could mention other easy-to-misuse constructs like over-complicated comprehensions, and point out that in practice things haven't turned out as bad with those as was feared. Paul. From p.f.moore at gmail.com Sat Apr 21 05:16:08 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 21 Apr 2018 10:16:08 +0100 Subject: [Python-Dev] Introducing python.zulipchat.com In-Reply-To: References: Message-ID: OK, got there now. Thanks for the help. That's a lousy sign-up experience, though... Paul On 21 April 2018 at 09:55, Jonathan Goble wrote: > On Sat, Apr 21, 2018, 4:51 AM Paul Moore wrote: >> >> Just a usability note - the sign in procedure seems very weird. I >> tried to log in, but didn't want to create another independent account >> so I tried "Sign in with Google" and "Sign in with Github". Both took >> me round a loop of authorising the access, then said "no account found >> for your email address" and I ended up back at the sign in page. >> >> I'm not sure what I did wrong, but the first time I did this, I just >> gave up. This time, I've sent this message... > > > Try clicking on "Sign up" below those buttons to get the "sign up with > Google/GitHub" buttons. From rosuav at gmail.com Sat Apr 21 06:12:35 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 21 Apr 2018 20:12:35 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <6281453e-6468-3b44-b237-d1147d02f51f@btinternet.com> References: <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <20180421071117.GW11616@ando.pearwood.info> <6281453e-6468-3b44-b237-d1147d02f51f@btinternet.com> Message-ID: On Sat, Apr 21, 2018 at 6:38 PM, Anthony Flury via Python-Dev wrote: > On 21/04/18 08:46, Chris Angelico wrote: >> >> doubled_items = [x for x in (items := get_items()) if x * 2 in items] >> >> This will leak 'items' into the surrounding scope (but not 'x'). > > At the risk of stating the obvious - wasn't there work in Python 3 to > prevent leakage from comprehensions ? >> >> [x for x in x if x] # This works >> [x for y in x if x := y] # UnboundLocalError > > > The standard library example given earlier notwithstanding, I can see no > benefit in using the same name as the iterator and the loop target name. To > be honest I have trouble parsing that first version, and keeping track of > which x is which (especially which x is being used in the conditional > clause) : surely this would be better : [x_item for x_item in x if x_item] > > Your 2nd example makes no sense to me as to the intention of the code - the > re-use of the name x is confusing at best. > I agree. The change in behaviour caused by PEP 572 is basically only going to be visible if you reuse a name, or in a very few other cases like yield expressions: def gen(): yield [x for x in (yield 1)] g = gen() next(g) g.send(range(5)) Once again, the outermost iterable is bizarre in this way. ChrisA From rosuav at gmail.com Sat Apr 21 06:18:15 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 21 Apr 2018 20:18:15 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> Message-ID: On Sat, Apr 21, 2018 at 7:12 PM, Paul Moore wrote: > On 21 April 2018 at 03:30, Chris Angelico wrote: >> There's that word "readability" again. Sometimes I wish the Zen of >> Python didn't use it, because everyone seems to think that "readable" >> means "code I like". > > Readability is subjective, yes. But it's not the same as "liking". If > a significant number of people say that they find a piece of code hard > to read/understand, then that's a problem. It's all too easy to say > "you don't have to write code like that", but as someone who has been > a maintenance programmer for his whole career, I can tell you that > people don't always have that luxury. And supporting code that's > written in a language that prioritises "readability" (whatever that > may mean) is a much easier task than supporting code written in a > language that doesn't. There's a reason far fewer people write systems > in Perl these days, and it's not because you can't write clear and > maintainable code in Perl... But you haven't answered anything about what "readable" means. Does it mean "if I look at this code, I can predict what dis.dis() would output"? Or does it mean "this code clearly expresses an algorithm and the programmer's intent"? Frequently I hear people complain that something is unreadable because it fails the former check. I'm much more interested in the latter check. For instance, this line of code expresses the concept "generate the squares of odd numbers": [x*x for x in range(100) if x % 2] But it doesn't clearly express the disassembly. Is that a problem? Are list comprehensions a bad feature for that reason? I don't think so. ChrisA From steve at pearwood.info Sat Apr 21 08:26:44 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 21 Apr 2018 22:26:44 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <20180421071117.GW11616@ando.pearwood.info> Message-ID: <20180421122643.GX11616@ando.pearwood.info> On Sat, Apr 21, 2018 at 05:46:44PM +1000, Chris Angelico wrote: > On Sat, Apr 21, 2018 at 5:11 PM, Steven D'Aprano wrote: > > So can you explain specifically what odd function-scope behaviour you > > are referring to? Give an example please? > > doubled_items = [x for x in (items := get_items()) if x * 2 in items] > > This will leak 'items' into the surrounding scope (but not 'x'). The "not x" part is odd, I agree, but it's a popular feature to have comprehensions run in a separate scope, so that's working as designed. The "leak items" part is the behaviour I desire, so that's not odd, it's sensible *wink* The reason I want items to "leak" into the surrounding scope is mostly so that the initial value for it can be set with a simple assignment outside the comprehension: items = (1, 2, 3) [ ... items := items*2 ... ] and the least magical way to do that is to just make items an ordinary local variable. > [x for x in x if x] # This works The oddity is that this does work, and there's no assignment expression in sight. Given that x is a local variable of the comprehension `for x in ...` it ought to raise UnboundLocalError, as the expanded equivalent does: def demo(): result = [] for x in x: # ought to raise UnboundLocalError if x: result.append(x) return result That the comprehension version runs (rather than raising) is surprising but I wouldn't call it a bug. Nor would I say it was a language guarantee that we have to emulate in similar expressions. In the absence of either explicit documentation of this behaviour, or Guido or one of the senior core developers explicitly stating that it is intentional behaviour that should be considered a language promise, I'd call it an accident of implementation. In which case, the fact that your next example: > [x for y in x if x := y] # UnboundLocalError "correctly" raises, as does the expanded version: def demo(): result = [] for y in x: # ought to raise UnboundLocalError x = y # since x is a local if x: result.append(x) return result shouldn't be seen as a problem. The code is different, so why should it behave the same? > (x for x in 5) # TypeError > (x for _ in [1] for x in 5) # Works Now that last one is more than just odd, it is downright bizarre. Or at least it would, if it did work: py> list((x for _ in [1] for x in 5)) Traceback (most recent call last): File "", line 1, in File "", line 1, in TypeError: 'int' object is not iterable Are you sure about this example? In any case, since this has no assignment expression in it, I don't see why it is relevant. -- Steve From christoph at grothesque.org Sat Apr 21 08:28:32 2018 From: christoph at grothesque.org (Christoph Groth) Date: Sat, 21 Apr 2018 14:28:32 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: Message-ID: <874lk4wxov.fsf@grothesque.org> Tim Peters wrote: > [Chris Angelico ] > > I don't see much value in restricting the assignment target to names > > only, but if that's what it takes, it can be restricted, at least > > initially. > > I believe this point was made most clearly before by Terry Reedy, but > it bears repeating :-) This is from the PEP's motivation: > > """ > Naming the result of an expression is an important part of > programming, allowing a descriptive name to be used in place of a > longer expression, and permitting reuse. > """ > > As "head arguments" go, that's a good one! But restricting assignment > expressions to > > identifier ":=" expression > > satisfies it. If what's of value is to name the result of an > expression, that single case handles that and _only_ that. In a > sense, it's "the simplest thing that could possibly work", and that's > generally a good thing to aim for. > > (...) Tim, thanks for this clear analysis. Here's the best use case of more general assignment expressions that I can come up with (from real code I'm currently working on): class Basis: def __init__(self, parent, periods=()): self._parent = parent if len(self._periods := np.asarray(periods, int)): ... else: # In absence of periods, treat them as an (0, n)-shaped array. # This avoids a special code path below. self._periods = np.empty((0, len(parent.periods)), int) But since this is a weak counterexample, it actually serves to strengthen your point that identifier ":=" expression is all that is needed. Such minimal assignment expressions have the (IMHO important) advantage of not being inconsistent with assignment statements. ---------------- Still, it seems weird to have two different ways of binding names in the language where one would be sufficient (i.e. the old one would remain only for backwards compatibility). From the point of view of someone who's new to the language that's two things to learn instead of just one. From anthony.flury at btinternet.com Sat Apr 21 07:48:45 2018 From: anthony.flury at btinternet.com (Anthony Flury) Date: Sat, 21 Apr 2018 12:48:45 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> Message-ID: <4c32f15d-a312-c186-6b26-cf14768f8d70@btinternet.com> On 21/04/18 11:18, Chris Angelico wrote: >> But you haven't answered anything about what "readable" means. Does it >> mean "if I look at this code, I can predict what dis.dis() would >> output"? Or does it mean "this code clearly expresses an algorithm and >> the programmer's intent"? Frequently I hear people complain that >> something is unreadable because it fails the former check. I'm much >> more interested in the latter check. For instance, this line of code >> expresses the concept "generate the squares of odd numbers": >> >> [x*x for x in range(100) if x % 2] >> >> But it doesn't clearly express the disassembly. Is that a problem? Are >> list comprehensions a bad feature for that reason? I don't think so. >> >> ChrisA For what it worth - readability for me is all about understanding the intent. I don't care (most of the time) about how the particular code construct is actually implemented. When I am maintaining code (or trying to) I need to understand what the developer intended (or in the case of a bug, the gap between the outcome and the intention). One of the challenges about readability is it partially depends on skill level - for a beginner the comprehension may well be baffling where as someone with more skills would? understand it - almost intuitively; as an example: I have been using Python for 7 years - and comprehensions with more than one for loop still are not intuitive for me, I can't read them without an amount of deep thought about how the loops work together. -- -- Anthony Flury email : *Anthony.flury at btinternet.com* Twitter : *@TonyFlury * From rosuav at gmail.com Sat Apr 21 10:45:36 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 22 Apr 2018 00:45:36 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <20180421122643.GX11616@ando.pearwood.info> References: <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <20180421071117.GW11616@ando.pearwood.info> <20180421122643.GX11616@ando.pearwood.info> Message-ID: On Sat, Apr 21, 2018 at 10:26 PM, Steven D'Aprano wrote: > On Sat, Apr 21, 2018 at 05:46:44PM +1000, Chris Angelico wrote: >> On Sat, Apr 21, 2018 at 5:11 PM, Steven D'Aprano wrote: > >> > So can you explain specifically what odd function-scope behaviour you >> > are referring to? Give an example please? >> >> doubled_items = [x for x in (items := get_items()) if x * 2 in items] >> >> This will leak 'items' into the surrounding scope (but not 'x'). > > The "not x" part is odd, I agree, but it's a popular feature to have > comprehensions run in a separate scope, so that's working as designed. > > The "leak items" part is the behaviour I desire, so that's not odd, it's > sensible *wink* > > The reason I want items to "leak" into the surrounding scope is mostly > so that the initial value for it can be set with a simple assignment > outside the comprehension: > > items = (1, 2, 3) > [ ... items := items*2 ... ] > > and the least magical way to do that is to just make items an ordinary > local variable. You can't have your cake and eat it too. Iteration variables and names bound by assignment expressions are both set inside the comprehension. Either they both are local, or they both leak - or else we have a weird rule like "the outermost iterable is magical and special". >> [x for x in x if x] # This works > > The oddity is that this does work, and there's no assignment expression > in sight. > > Given that x is a local variable of the comprehension `for x in ...` it > ought to raise UnboundLocalError, as the expanded equivalent does: > > > def demo(): > result = [] > for x in x: # ought to raise UnboundLocalError > if x: > result.append(x) > return result > > > That the comprehension version runs (rather than raising) is surprising > but I wouldn't call it a bug. Nor would I say it was a language > guarantee that we have to emulate in similar expressions. See, that's the problem. That is NOT how the comprehension expands. It actually expands to this: def demo(it): result = [] for x in it: if x: result.append(x) return result demo(iter(x)) PEP 572 corrects this by making it behave the way that you, and many other people, expect. Current behaviour is surprising because the outermost iterable is special and magical. >> (x for x in 5) # TypeError >> (x for _ in [1] for x in 5) # Works > > Now that last one is more than just odd, it is downright bizarre. Or at > least it would, if it did work: > > py> list((x for _ in [1] for x in 5)) > Traceback (most recent call last): > File "", line 1, in > File "", line 1, in > TypeError: 'int' object is not iterable > > > Are you sure about this example? Yes, I'm sure. You may notice that I didn't iterate over the genexps in my example. The first one will bomb out, even without iteration; the second one gives a valid generator object which, if iterated over (or even stepped once), will bomb. This is because, again, the outermost iterable is special and magical. > In any case, since this has no assignment expression in it, I don't see > why it is relevant. Because an assignment expression in the outermost iterable would, if the semantics are preserved, bind in the surrounding scope. It would be FAR more logical to have it bind in the inner scope. Consider these two completely different results: def f(*prefix): print([p + name for p in prefix for name in locals()]) print([p + name for name in locals() for p in prefix]) >>> f("* ", "$ ") ['* .0', '* p', '$ .0', '$ p', '$ name'] ['* prefix', '$ prefix'] The locals() as seen by the outermost iterable are f's locals, and any assignment expression there would be part of f's locals. The locals() as seen by any other iterable, by a condition, or by the primary expression, are the list comp's locals, and any assignment expression there would be part of the list comp's locals. ChrisA From mertz at gnosis.cx Sat Apr 21 11:44:48 2018 From: mertz at gnosis.cx (David Mertz) Date: Sat, 21 Apr 2018 15:44:48 +0000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <20180421071117.GW11616@ando.pearwood.info> <20180421122643.GX11616@ando.pearwood.info> Message-ID: It feels very strange that the PEP tries to do two almost entirely unrelated things. Assignment expressions are one thing, with merits and demerits discussed at length. But "fixing" comprehension scoping is pretty much completely orthogonal. Sure, it might be a good idea. And yes there are interactions between the behaviors. However, trying to shoehorn the one issue into a PEP on a different topic makes all of it harder to accept. The "broken" scoping in some slightly strange edge cases can and has been shown in lots of examples that don't use assignment expressions. Whether or not that should be changed needn't be linked to the real purpose of this PEP. On Sat, Apr 21, 2018, 10:46 AM Chris Angelico wrote: > On Sat, Apr 21, 2018 at 10:26 PM, Steven D'Aprano > wrote: > > On Sat, Apr 21, 2018 at 05:46:44PM +1000, Chris Angelico wrote: > >> On Sat, Apr 21, 2018 at 5:11 PM, Steven D'Aprano > wrote: > > > >> > So can you explain specifically what odd function-scope behaviour you > >> > are referring to? Give an example please? > >> > >> doubled_items = [x for x in (items := get_items()) if x * 2 in items] > >> > >> This will leak 'items' into the surrounding scope (but not 'x'). > > > > The "not x" part is odd, I agree, but it's a popular feature to have > > comprehensions run in a separate scope, so that's working as designed. > > > > The "leak items" part is the behaviour I desire, so that's not odd, it's > > sensible *wink* > > > > The reason I want items to "leak" into the surrounding scope is mostly > > so that the initial value for it can be set with a simple assignment > > outside the comprehension: > > > > items = (1, 2, 3) > > [ ... items := items*2 ... ] > > > > and the least magical way to do that is to just make items an ordinary > > local variable. > > You can't have your cake and eat it too. Iteration variables and names > bound by assignment expressions are both set inside the comprehension. > Either they both are local, or they both leak - or else we have a > weird rule like "the outermost iterable is magical and special". > > >> [x for x in x if x] # This works > > > > The oddity is that this does work, and there's no assignment expression > > in sight. > > > > Given that x is a local variable of the comprehension `for x in ...` it > > ought to raise UnboundLocalError, as the expanded equivalent does: > > > > > > def demo(): > > result = [] > > for x in x: # ought to raise UnboundLocalError > > if x: > > result.append(x) > > return result > > > > > > That the comprehension version runs (rather than raising) is surprising > > but I wouldn't call it a bug. Nor would I say it was a language > > guarantee that we have to emulate in similar expressions. > > See, that's the problem. That is NOT how the comprehension expands. It > actually expands to this: > > def demo(it): > result = [] > for x in it: > if x: > result.append(x) > return result > demo(iter(x)) > > PEP 572 corrects this by making it behave the way that you, and many > other people, expect. Current behaviour is surprising because the > outermost iterable is special and magical. > > >> (x for x in 5) # TypeError > >> (x for _ in [1] for x in 5) # Works > > > > Now that last one is more than just odd, it is downright bizarre. Or at > > least it would, if it did work: > > > > py> list((x for _ in [1] for x in 5)) > > Traceback (most recent call last): > > File "", line 1, in > > File "", line 1, in > > TypeError: 'int' object is not iterable > > > > > > Are you sure about this example? > > Yes, I'm sure. You may notice that I didn't iterate over the genexps > in my example. The first one will bomb out, even without iteration; > the second one gives a valid generator object which, if iterated over > (or even stepped once), will bomb. This is because, again, the > outermost iterable is special and magical. > > > In any case, since this has no assignment expression in it, I don't see > > why it is relevant. > > Because an assignment expression in the outermost iterable would, if > the semantics are preserved, bind in the surrounding scope. It would > be FAR more logical to have it bind in the inner scope. Consider these > two completely different results: > > def f(*prefix): > print([p + name for p in prefix for name in locals()]) > print([p + name for name in locals() for p in prefix]) > > >>> f("* ", "$ ") > ['* .0', '* p', '$ .0', '$ p', '$ name'] > ['* prefix', '$ prefix'] > > The locals() as seen by the outermost iterable are f's locals, and any > assignment expression there would be part of f's locals. The locals() > as seen by any other iterable, by a condition, or by the primary > expression, are the list comp's locals, and any assignment expression > there would be part of the list comp's locals. > > ChrisA > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/mertz%40gnosis.cx > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sat Apr 21 11:57:00 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 22 Apr 2018 01:57:00 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <20180421071117.GW11616@ando.pearwood.info> <20180421122643.GX11616@ando.pearwood.info> Message-ID: On 22 April 2018 at 01:44, David Mertz wrote: > It feels very strange that the PEP tries to do two almost entirely unrelated > things. Assignment expressions are one thing, with merits and demerits > discussed at length. > > But "fixing" comprehension scoping is pretty much completely orthogonal. > Sure, it might be a good idea. And yes there are interactions between the > behaviors. However, trying to shoehorn the one issue into a PEP on a > different topic makes all of it harder to accept. > > The "broken" scoping in some slightly strange edge cases can and has been > shown in lots of examples that don't use assignment expressions. Whether or > not that should be changed needn't be linked to the real purpose of this > PEP. The reason it's covered in the PEP is because the PEP doesn't want to lock in the current "binds the name in the surrounding scope" semantics when assignment expressions are used in the outermost iterable in a comprehension. However, resolving that question *could* be postponed more simply by making that a SyntaxError, rather than trying to move the expression evaluation inside the implicitly nested scope. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From mertz at gnosis.cx Sat Apr 21 12:08:32 2018 From: mertz at gnosis.cx (David Mertz) Date: Sat, 21 Apr 2018 12:08:32 -0400 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <20180421071117.GW11616@ando.pearwood.info> <20180421122643.GX11616@ando.pearwood.info> Message-ID: It could also be postponed simply by saying assignment expressions follow the same semantics as other bindings in comprehensions... which are subject to change pending PEP XXXX (i.e. some different number). On the other hand, I am one who doesn't really care about assignment expressions in comprehensions and only see the real benefit for 'if' and 'while' statements. I'm sure if it's added, I'll wind up using them in comprehensions, but I've been perfectly happy with this for years: stuff = [[y, x/y] for x in range(5) for y in [f(x)]] There's nothing quite analogous in current Python for: while (command := input("> ")) != "quit": print("You entered:", command) On Sat, Apr 21, 2018, 11:57 AM Nick Coghlan wrote: > On 22 April 2018 at 01:44, David Mertz wrote: > > It feels very strange that the PEP tries to do two almost entirely > unrelated > > things. Assignment expressions are one thing, with merits and demerits > > discussed at length. > > > > But "fixing" comprehension scoping is pretty much completely orthogonal. > > Sure, it might be a good idea. And yes there are interactions between the > > behaviors. However, trying to shoehorn the one issue into a PEP on a > > different topic makes all of it harder to accept. > > > > The "broken" scoping in some slightly strange edge cases can and has been > > shown in lots of examples that don't use assignment expressions. Whether > or > > not that should be changed needn't be linked to the real purpose of this > > PEP. > > The reason it's covered in the PEP is because the PEP doesn't want to > lock in the current "binds the name in the surrounding scope" > semantics when assignment expressions are used in the outermost > iterable in a comprehension. > > However, resolving that question *could* be postponed more simply by > making that a SyntaxError, rather than trying to move the expression > evaluation inside the implicitly nested scope. > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Sat Apr 21 12:31:53 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 22 Apr 2018 02:31:53 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <20180421071117.GW11616@ando.pearwood.info> <20180421122643.GX11616@ando.pearwood.info> Message-ID: <20180421163153.GY11616@ando.pearwood.info> On Sat, Apr 21, 2018 at 03:44:48PM +0000, David Mertz wrote: > It feels very strange that the PEP tries to do two almost entirely > unrelated things. Assignment expressions are one thing, with merits and > demerits discussed at length. > > But "fixing" comprehension scoping is pretty much completely orthogonal. This. > Sure, it might be a good idea. And yes there are interactions between the > behaviors. However, trying to shoehorn the one issue into a PEP on a > different topic makes all of it harder to accept. Indeed. *And* harder to understand. As I see it, we ought to just decide on the semantics of assignment- expressions as they relate to comprehensions: do they bind to the comprehension scope, or the local scope? I prefer the second, for the reasons I stated earlier. A third (slightly more complex) choice would be that they remain bound to the comprehension (like the loop variable) but they are initialised from any surrounding scope. I'd be happy with that as a "best of both worlds" compromise: # don't leak from comprehensions x = 1 [(x := y+1) for y in items if x%2 == 0] assert x == 1 # but still support running totals and similar use-cases total = 0 [(total := total + y) for y in items] # and you can still get UnboundLocalError del total [(total := total + y) for y in items] # total has no initial value This is not entirely unprecedented in Python: it is analogous (although not identical) to binding default values to parameters: def running_total(items, total=total): # Here total is local to the function, but the default # is taken from the surrounding scope. Cleaning up the odd interactions involved in comprehensions could be done separately, or later, or not at all. After all, this PEP isn't *introducing* those oddities. As Chris' earlier examples show, they already exist. -- Steve From stephen at xemacs.org Sat Apr 21 12:45:31 2018 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 22 Apr 2018 01:45:31 +0900 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> Message-ID: <23259.27307.864811.86992@turnbull.sk.tsukuba.ac.jp> Chris Angelico writes: > There's that word "readability" again. Sometimes I wish the Zen of > Python didn't use it, because everyone seems to think that "readable" > means "code I like". Hey, man, that hurts. Some of us not only *have* precise statements of the aspects of readability we invoked, but we *provided* them in our posts. I sympathize that you get tired of the repetitions of various claims about readability, as well as the proliferation of purely subjective claims about it, but that doesn't mean they deserve to be dismissed this way. That said, subjectivity is a real problem, and it's not a PEP protagonist's responsibility to deal with it. I would like to recommend that posters who want to make claims about "readability" remember that it's one aspect of Pythonic language design out of many, and that it is a complex concept itself. A claim about readability is a claim about an aspect of the *value* of a construct, but the word "readability" alone is too subjective to explain *why* it has (or lacks) that value. If you can't describe, let alone define, what you mean by "readable", but still feel strongly enough to post, think whether you really mean that word. If so, apologize for that lack, because PEP protagonists have no obligation to figure out what you can't. If not, find another word to express the feeling. Or postpone posting until you have a description or better word. Language design has a lot of these words involving complexity and subjectivity: readability, expressiveness, power. Remember that your "vote" counts even when subjective: you don't have to justify your likes. But if it's personal, you can express that succinctly with +1/-1. If you want to claim that the subjective feeling is common to many users of Python, you need to communicate it. Try to define the aspect you're using. Do so explicitly, unless the definition you're using was given already in a recent post and is current in the thread. Steve From steve at pearwood.info Sat Apr 21 13:41:26 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 22 Apr 2018 03:41:26 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <20180421071117.GW11616@ando.pearwood.info> <20180421122643.GX11616@ando.pearwood.info> Message-ID: <20180421174125.GZ11616@ando.pearwood.info> On Sun, Apr 22, 2018 at 12:45:36AM +1000, Chris Angelico wrote: > > The reason I want items to "leak" into the surrounding scope is mostly > > so that the initial value for it can be set with a simple assignment > > outside the comprehension: > > > > items = (1, 2, 3) > > [ ... items := items*2 ... ] > > > > and the least magical way to do that is to just make items an ordinary > > local variable. > > You can't have your cake and eat it too. Iteration variables and names > bound by assignment expressions are both set inside the comprehension. You say that as if it were a law of physics, rather than an implementation choice. > Either they both are local, or they both leak - or else we have a > weird rule like "the outermost iterable is magical and special". We *already* have the rule that the outermost iterable is special, except it isn't a rule precisely, since (as far as I know) it isn't documented anywhere, nor was it ever planned as a feature. It's just an accidental(?) consequence of the implementation choices made. py> spam = [(1,2), (3, 4)] py> [spam for x in spam for spam in x] # first loop is magic [1, 2, 3, 4] but: py> spam = [(1,2), (3, 4)] py> [spam for _ in [1] for x in spam for spam in x] Traceback (most recent call last): File "", line 1, in File "", line 1, in UnboundLocalError: local variable 'spam' referenced before assignment However the second example worked fine in Python 2. Changing the implementation of comprehensions to be like generator expressions and avoid leaking the loop variables seems to have had the (accidental?) side-effect of making the first loop magical. [...] > PEP 572 corrects this by making it behave the way that you, and many > other people, expect. Current behaviour is surprising because the > outermost iterable is special and magical. This shouldn't be PEP 572's job. It's unfair on you to be shouldered with sheparding through what is effectively two complex PEPs ("assignment-expressions" plus "fix comprehension scoping") in one. Especially if you had no idea at the start that this is what is involved. And it's even more unfair on those who may not care two hoots about assignment-expressions, but would be really interested in comprehension scoping if only they knew we were talking about that. And it makes this PEP harder work for readers who don't care about comprehension scoping. I think that we should be able to make any of the following choices (or no choice at all) regarding comprehensions: * no change: semantics remains underspecified, defined by "whatever the implementation does"; * lock in the current behaviour as a language promise; * change the behaviour and make it a language promise; regardless of what is decided about PEP 572. [...] > > Are you sure about this example? > > Yes, I'm sure. You may notice that I didn't iterate over the genexps > in my example. No, I didn't notice that was your intent. I thought it was just short-hand. > The first one will bomb out, even without iteration; And that's yet another oddity, one I didn't think of. It's downright bizarre that these two genexps behave differently: spam = [1, 2] eggs = 12 (x+y for x in spam for y in eggs) # okay (x+y for y in eggs for x in spam) # TypeError and I'd be surprised to learn that this behavour was planned in advance. ("Early binding and ahead-of-time type-testing for the first loop, late binding and just-in-time type-testing for the second loop. All in favour?") But it is what it is, and who knows, maybe we decide we *want* this behaviour, bizarre as it is. It isn't clear to me that: 1. it's necessarily "broken" and needs fixing; 2. if if does need fixing, it needs to be fixed *right* now; 3. that acceptance or rejection of PEP 572 needs to hinge on the decision about comprehensions; 4. and especially that a change to comprehensions ought to be smuggled in via an unrelated PEP. (I know that 4 is not your intention, but that's the way it may appear.) -- Steve From matthew at woodcraft.me.uk Sat Apr 21 13:47:13 2018 From: matthew at woodcraft.me.uk (Matthew Woodcraft) Date: Sat, 21 Apr 2018 18:47:13 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On 2018-04-17 08:46, Chris Angelico wrote: > Having survived four rounds in the boxing ring at python-ideas, PEP > 572 is now ready to enter the arena of python-dev. I would like to suggest one more motivating example for "Capturing condition values": multiple regex matches with 'elif'. if match := re.search(pat1, text): print("Found one:", match.group(0)) elif match := re.search(pat2, text): print("Found two:", match.group(0)) elif match := re.search(pat3, text): print("Found three:", match.group(0)) Without assignment expressions, you have an annoying choice between a cascade of 'else's with an ever-increasing indent and evaluating all the matches up front (so doing unnecessary work). -M- From tim.peters at gmail.com Sat Apr 21 14:02:00 2018 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 21 Apr 2018 13:02:00 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: [Matthew Woodcraft ] > I would like to suggest one more motivating example for "Capturing > condition values": multiple regex matches with 'elif'. > > if match := re.search(pat1, text): > print("Found one:", match.group(0)) > elif match := re.search(pat2, text): > print("Found two:", match.group(0)) > elif match := re.search(pat3, text): > print("Found three:", match.group(0)) > > Without assignment expressions, you have an annoying choice between a > cascade of 'else's with an ever-increasing indent and evaluating all the > matches up front (so doing unnecessary work). That's a reasonable use, but would more likely be written like so today: for tag, pat in (("one", pat1), ("two", pat2), ("three", pat3). ("four", pat4), ...): match = re.search(pat, text) if match: print("Found", tag + ":", match.group(0)) break Which would still read a bit nicer if the first two loop body lines could be collapsed to if match := re.search(pat, text): From mcepl at cepl.eu Sat Apr 21 13:53:42 2018 From: mcepl at cepl.eu (=?UTF-8?Q?Mat=C4=9Bj?= Cepl) Date: Sat, 21 Apr 2018 19:53:42 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions References: <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <20180421071117.GW11616@ando.pearwood.info> Message-ID: On 2018-04-21, 07:46 GMT, Chris Angelico wrote: > doubled_items = [x for x in (items := get_items()) if x * 2 in > items] Aside from other concerns expressed elsewhere by other people, do you really like this? I know and agree that ?readability? is a subjective term, but it is my firm persuasion that whenever I need to think about what particular list comprehension means, it is the moment I should write a separate function or normal cycle. I think we should encourage writing simple (I didn?t use the r* word here, you see!) code than something which has potential to slide us towards Perl. Best, Mat?j -- https://matej.ceplovi.cz/blog/, Jabber: mcepl at ceplovi.cz GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 A day without sunshine is like night. From tim.peters at gmail.com Sat Apr 21 15:20:21 2018 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 21 Apr 2018 14:20:21 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <874lk4wxov.fsf@grothesque.org> References: <874lk4wxov.fsf@grothesque.org> Message-ID: [Christoph Groth ] > Tim, thanks for this clear analysis. Here's the best use case of more > general assignment expressions that I can come up with (from real code > I'm currently working on): > > class Basis: > def __init__(self, parent, periods=()): > self._parent = parent > if len(self._periods := np.asarray(periods, int)): > ... > else: > # In absence of periods, treat them as an (0, n)-shaped array. > # This avoids a special code path below. > self._periods = np.empty((0, len(parent.periods)), int) > > But since this is a weak counterexample, it actually serves to > strengthen your point that > > identifier ":=" expression > > is all that is needed. That's a decent example. In truth, I have no real objection to binding an attribute - but am willing to throw out a bit of soap with the bathwater if doing so can avoid throwing the baby out too ;-) > Such minimal assignment expressions have the (IMHO important) advantage > of not being inconsistent with assignment statements. > > ---------------- > > Still, it seems weird to have two different ways of binding names in the > language where one would be sufficient (i.e. the old one would remain > only for backwards compatibility). From the point of view of someone > who's new to the language that's two things to learn instead of just > one. But they're very different in a key respect. the value of an assignment expression is the value assigned. Asking "what's the value of a statement?" doesn't even make sense in Python (whether an assignment statement or any other kind of statement). For that reason, _if_ a PEP is reworked to suggest a "binding expression" (I'd prefer the name change to nudge people away from conflating it with the far more general assignment statement), the usage pragmatics are clear: use a binding expression if the context requires using the value bound, else use a simple assignment statement. ":=" doesn't _just_ mean "bind the simple name on the left" in that world, but also "and return the value of the expression on the right". For that reason, e.g., i = 1 would be strongly preferred to i := 1 as a standalone line, except perhaps when typing at an interactive shell (where you may _want_ to see the value being bound - but usually don't). From tim.peters at gmail.com Sat Apr 21 15:27:11 2018 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 21 Apr 2018 14:27:11 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> <87h8o5x36p.fsf@grothesque.org> Message-ID: [Tim] >> I expected that, given that expressions "naturally nest", chained >> targets could still be specified: >> >> a := b := c:= 5 >> >> but since they're all plain names there's no way to tell whether the >> bindings occur "left to right" or "right to left" short of staring at >> the generated code. [Nick Coghlan ] > The fact class namespaces are ordered by default now allow us to > demonstrate the order of multiple target assignments and tuple > unpacking without staring at generated code: > > >>> class AssignmentOrder: > ... a = b = c = 0 > ... d, e, f = range(3) > ... > >>> class ReversedAssignmentOrder: > ... c = b = a = 0 > ... f, e, d = range(3) > ... > >>> [attr for attr in AssignmentOrder.__dict__ if not attr.startswith("_")] > ['a', 'b', 'c', 'd', 'e', 'f'] > >>> [attr for attr in ReversedAssignmentOrder.__dict__ if not attr.startswith("_")] > ['c', 'b', 'a', 'f', 'e', 'd'] > > So that's a situation where "name = alias = value" could end up > matching "alias := name := value" Cool! So this is really a killer-strong argument for getting rid of classes - way overdue, too ;-) > (Even in earlier versions, you can illustrate the same assignment > ordering behaviour with the enum module, and there it makes even more > of a difference, as it affects which name binding is considered the > canonical name, and which are considered aliases). So if binding expressions can be chained, they'll need to ape "left-to-right" binding order. Or they can't be allowed to chain to begin with. Either way would be fine by me. From matthew at woodcraft.me.uk Sat Apr 21 15:35:51 2018 From: matthew at woodcraft.me.uk (Matthew Woodcraft) Date: Sat, 21 Apr 2018 20:35:51 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On 2018-04-21 19:02, Tim Peters wrote: > [Matthew Woodcraft ] >> I would like to suggest one more motivating example for "Capturing >> condition values": multiple regex matches with 'elif'. >> >> if match := re.search(pat1, text): >> print("Found one:", match.group(0)) >> elif match := re.search(pat2, text): >> print("Found two:", match.group(0)) >> elif match := re.search(pat3, text): >> print("Found three:", match.group(0)) >> >> Without assignment expressions, you have an annoying choice between a >> cascade of 'else's with an ever-increasing indent and evaluating all the >> matches up front (so doing unnecessary work). > > That's a reasonable use, but would more likely be written like so today: > > for tag, pat in (("one", pat1), ("two", pat2), ("three", pat3). > ("four", pat4), ...): > match = re.search(pat, text) > if match: > print("Found", tag + ":", match.group(0)) > break Well, that's a reason to make the example a bit more realistic, then. Say: if match := re.search(pat1, text): do_something_with(match.group(0)) elif match := re.search(pat2, text): do_something_else_with(match.group(0), match.group(1)) elif match := re.search(pat3, text): do_some_other_things_with(match.group(0)) and_also_with(match.group(1), match.group(2)) -M- From christoph at grothesque.org Sat Apr 21 18:44:31 2018 From: christoph at grothesque.org (Christoph Groth) Date: Sun, 22 Apr 2018 00:44:31 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: (Tim Peters's message of "Sat, 21 Apr 2018 14:20:21 -0500") References: <874lk4wxov.fsf@grothesque.org> Message-ID: <87sh7ouqls.fsf@grothesque.org> Tim Peters wrote: > [Christoph Groth ] > > Still, it seems weird to have two different ways of binding names in > > the language where one would be sufficient (i.e. the old one would > > remain only for backwards compatibility). From the point of view of > > someone who's new to the language that's two things to learn instead > > of just one. > > But they're very different in a key respect. the value of an > assignment expression is the value assigned. Asking "what's the value > of a statement?" doesn't even make sense in Python (whether an > assignment statement or any other kind of statement). There are also no function call statements in Python. People are happily using function call expressions as statements when not interested in their value. I hope to have shown [1] that the same could be done for assignments. A consistent value can be defined for any assignment statement. So, all assignment statements could be redefined as expressions and the language would continue to work and even be (perfectly?) backwards-compatible. Syntax-wise, if replacing = by := everywhere is unthinkable, as it seems, there's still the possibility (not completely ruled out by Guido ;-) to use = for assignment expressions but require extra parens for safety. Thus, it seems to me that redefining assignments as expressions everywhere is a feasible, if radical, idea. Compared to a dedicated syntax for "binding expressions" it would be conceptually simpler, but would provide more possibilities to shoot oneself in the foot. [1] https://mail.python.org/pipermail/python-dev/2018-April/152780.html From brett at python.org Sat Apr 21 19:05:04 2018 From: brett at python.org (Brett Cannon) Date: Sat, 21 Apr 2018 23:05:04 +0000 Subject: [Python-Dev] Introducing python.zulipchat.com In-Reply-To: References: Message-ID: The Zulip project maintainers are active on our instance so after you join go to the Zulip stream and start a topic about this. On Sat, Apr 21, 2018, 02:16 Paul Moore, wrote: > OK, got there now. Thanks for the help. That's a lousy sign-up > experience, though... > Paul > > On 21 April 2018 at 09:55, Jonathan Goble wrote: > > On Sat, Apr 21, 2018, 4:51 AM Paul Moore wrote: > >> > >> Just a usability note - the sign in procedure seems very weird. I > >> tried to log in, but didn't want to create another independent account > >> so I tried "Sign in with Google" and "Sign in with Github". Both took > >> me round a loop of authorising the access, then said "no account found > >> for your email address" and I ended up back at the sign in page. > >> > >> I'm not sure what I did wrong, but the first time I did this, I just > >> gave up. This time, I've sent this message... > > > > > > Try clicking on "Sign up" below those buttons to get the "sign up with > > Google/GitHub" buttons. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Sat Apr 21 21:13:09 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 22 Apr 2018 11:13:09 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: <20180422011308.GA11616@ando.pearwood.info> On Sat, Apr 21, 2018 at 08:35:51PM +0100, Matthew Woodcraft wrote: > Well, that's a reason to make the example a bit more realistic, then. > > Say: > > if match := re.search(pat1, text): > do_something_with(match.group(0)) > elif match := re.search(pat2, text): > do_something_else_with(match.group(0), match.group(1)) > elif match := re.search(pat3, text): > do_some_other_things_with(match.group(0)) > and_also_with(match.group(1), match.group(2)) I don't think that a bunch of generic "do_something_with" functions is precisely "realistic". If I saw something like that, I'd try very hard to find a way to refactor it into code like this: for handler in handlers: if handler.match(text): handler.process() break else: # handle no-match case here where the knowledge of what to search for, where to search for it, how to search for it, and what to do when found, was encapsulated in the handler objects. Your tastes may vary. But your point is well-taken that the version with binding assignment (thanks Tim!) is nicer to read than the current procedural version: match = re.search(pat1, text) if match: do_something_with(match.group(0)) else: match = re.search(pat2, text) if match: do_something_else_with(match.group(0), match.group(1)) else: match = = re.search(pat3, text) do_some_other_things_with(match.group(0)) and_also_with(match.group(1), match.group(2)) I just don't think it counts as a motivating use-case distinct from the single match case. -- Steve From python-dev at mgmiller.net Sat Apr 21 22:04:18 2018 From: python-dev at mgmiller.net (Mike Miller) Date: Sat, 21 Apr 2018 19:04:18 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> Message-ID: <9bcde4df-2772-f26e-924a-97544d6bbccc@mgmiller.net> Round 2 (Changed order, see below): 1. with open(fn) as f: # current behavior 2. with (open(fn) as f): # same 3. with closing(urlopen(url)) as dl: # current behavior 5. with (closing(urlopen(url)) as dl): # same 4. with closing(urlopen(url) as dl): # urlopener named early On 2018-04-20 17:15, Chris Angelico wrote: > The second and fifth could be special cased as either the same as > first and third, or as SyntaxErrors. (But which?) If they are expressions, they should be the same once evaluated, no? (I had a brief episode where I wrote that "as" was required with "with", instead of the CM object, sorry. :) > The fourth one is very tricky. If 'expr as name' is allowed inside arbitrary > expressions, why shouldn't it be allowed there? Yes, they should be allowed there. > The disconnect between viable syntax and useful statements is problematic here. Number 4 appears to name the urlopener early. Since closing() returns it as well, might it work anyway? Might be missing something else, but #4 looks like a mistake with the layout of the parentheses, which can happen anywhere. I don't get the sense it will happen often. Cheers, -Mike From guido at python.org Sat Apr 21 22:05:26 2018 From: guido at python.org (Guido van Rossum) Date: Sat, 21 Apr 2018 19:05:26 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <20180422011308.GA11616@ando.pearwood.info> References: <20180422011308.GA11616@ando.pearwood.info> Message-ID: On Sat, Apr 21, 2018 at 6:13 PM, Steven D'Aprano wrote: > On Sat, Apr 21, 2018 at 08:35:51PM +0100, Matthew Woodcraft wrote: > > > Well, that's a reason to make the example a bit more realistic, then. > > > > Say: > > > > if match := re.search(pat1, text): > > do_something_with(match.group(0)) > > elif match := re.search(pat2, text): > > do_something_else_with(match.group(0), match.group(1)) > > elif match := re.search(pat3, text): > > do_some_other_things_with(match.group(0)) > > and_also_with(match.group(1), match.group(2)) > > I don't think that a bunch of generic "do_something_with" functions is > precisely "realistic". > > If I saw something like that, I'd try very hard to find a way to > refactor it into code like this: > > for handler in handlers: > if handler.match(text): > handler.process() > break > else: > # handle no-match case here > > where the knowledge of what to search for, where to search for it, how > to search for it, and what to do when found, was encapsulated in the > handler objects. Your tastes may vary. > > But your point is well-taken that the version with binding assignment > (thanks Tim!) is nicer to read than the current procedural version: > > match = re.search(pat1, text) > if match: > do_something_with(match.group(0)) > else: > match = re.search(pat2, text) > if match: > do_something_else_with(match.group(0), match.group(1)) > else: > match = = re.search(pat3, text) > do_some_other_things_with(match.group(0)) > and_also_with(match.group(1), match.group(2)) > > I just don't think it counts as a motivating use-case distinct from the > single match case. > The version of this code found in reality is not as regular as the example quoted, and the rebuttal "but I would rewrite it with a loop" shoots a straw man. To me the if-elif-elif portion of the example is very much a separate motivation, since being able to put the assignment in the elif clause avoids runaway indentation. I've regretted not being able to use elif in this kind of situation many times, whereas in the single match case I don't find it a burden to assign the variable in a separate statement preceding the if-clause. (I guess this is a case of "flat is better than nested" -- thanks Tim! :-) -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.peters at gmail.com Sat Apr 21 22:22:19 2018 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 21 Apr 2018 21:22:19 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <87sh7ouqls.fsf@grothesque.org> References: <874lk4wxov.fsf@grothesque.org> <87sh7ouqls.fsf@grothesque.org> Message-ID: [Christoph Groth ] >> > Still, it seems weird to have two different ways of binding names in >> > the language where one would be sufficient (i.e. the old one would >> > remain only for backwards compatibility). From the point of view of >> > someone who's new to the language that's two things to learn instead >> > of just one. [Tim] >> But they're very different in a key respect. the value of an >> assignment expression is the value assigned. Asking "what's the value >> of a statement?" doesn't even make sense in Python (whether an >> assignment statement or any other kind of statement). [Christoph] > There are also no function call statements in Python. People are > happily using function call expressions as statements when not > interested in their value. Sure. > I hope to have shown [1] that the same could be done for assignments. A > consistent value can be defined for any assignment statement. So, all > assignment statements could be redefined as expressions and the language > would continue to work and even be (perfectly?) backwards-compatible. Except for shells. When I type, e.g., >>> xs = sorted(iterator_returning_a_billion_strings) I really don't want to wait for hours before I can type again ;-) In the same way now, when someone calls a function at a shell but doesn't want to see its result, they do something like >>> xxx = function(a, b, c) knowing that an assignment statement never displays any output on its own. If an assignment statement did return a result, almost all shells would display it. Shells typically don't care at all what you typed at them, they just care whether or not executing the compiled code returns None: result = execute_code() if result is not None: display(repr(result)) There's also that you're not considering the other half: that every existing assignment statement could be viewed as being as expression does not imply that every existing assignment statement could be used everywhere an expression can be used. Syntax matters, and function call argument lists in particular already bristle with their own meanings for commas, equal signs, and asterisks. The language was designed with "and the twain shall never meet" in mind ;-) For example, what would f(a=b) mean? The worst possible ;-) answer is "well, since a=b is fine as an assignment statement, it must mean that we bind the value of `b` to name `a` and then pass b's value to `f()` as its first positional argument". That reading would break countless lines of code using keyword arguments. If you're willing to concede that's too much breakage to bear, then you have to identify and spell out "the rules" for every case in which something that "looks like an assignment expression really isn't, depending on context". But since I have no interest in pursuing this, I'll stop there :-) > Syntax-wise, if replacing = by := everywhere is unthinkable, as it > seems, there's still the possibility (not completely ruled out by Guido > ;-) to use = for assignment expressions but require extra parens for > safety. That would be received less well than the current PEP. The people it would hurt the most are newcomers from other languages who habitually put _every_ "if" and "while" test in parentheses, because that's what they're used to doing (e.g., in C). Many of us still remember our initial relief when we realized we'd never piss away hours debugging an assert(n=1) or if (x=0.0) typo/thinko again. Reintroducing that possibility would get an instant -1 from me, because I don't want to debug that same mistake for other people on Stackoverflow either - my time there is wholly consumed by explaining why .1 + .2 doesn't display exactly "0.3" ;-) > Thus, it seems to me that redefining assignments as expressions > everywhere is a feasible, if radical, idea. Compared to a dedicated > syntax for "binding expressions" it would be conceptually simpler, but > would provide more possibilities to shoot oneself in the foot. As above, it wouldn't remain so simple after hammering out the detailed rules for deciding when and where something that "looks like an assignment expression" really is one. For an example of a fine language that makes no distinction between "statements" and "expressions" at all, Icon is top on my list. That _can_ work out fine - but Icon was designed that way from the start. And, of course, like every sane language that has wholly general assignment expressions, Icon uses ";=" as the assignment operator, and "=" for numeric equality testing ;-) > [1] https://mail.python.org/pipermail/python-dev/2018-April/152780.html From tim.peters at gmail.com Sat Apr 21 22:40:41 2018 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 21 Apr 2018 21:40:41 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> Message-ID: [Matthew Woodcraft] >>> Well, that's a reason to make the example a bit more realistic, then. >>> >>> Say: >>> >>> if match := re.search(pat1, text): >>> do_something_with(match.group(0)) >>> elif match := re.search(pat2, text): >>> do_something_else_with(match.group(0), match.group(1)) >>> elif match := re.search(pat3, text): >>> do_some_other_things_with(match.group(0)) >>> and_also_with(match.group(1), match.group(2)) [Steven D'Aprano ] >> I don't think that a bunch of generic "do_something_with" functions is >> precisely "realistic". >> >> If I saw something like that, I'd try very hard to find a way to >> refactor it into code like this: >> >> for handler in handlers: >> if handler.match(text): >> handler.process() >> break >> else: >> # handle no-match case here >> >> where the knowledge of what to search for, where to search for it, how >> to search for it, and what to do when found, was encapsulated in the >> handler objects. Your tastes may vary. >> >> But your point is well-taken that the version with binding assignment >> (thanks Tim!) is nicer to read than the current procedural version: >> >> match = re.search(pat1, text) >> if match: >> do_something_with(match.group(0)) >> else: >> match = re.search(pat2, text) >> if match: >> do_something_else_with(match.group(0), match.group(1)) >> else: >> match = = re.search(pat3, text) >> do_some_other_things_with(match.group(0)) >> and_also_with(match.group(1), match.group(2)) >> >> I just don't think it counts as a motivating use-case distinct from the >> single match case. [Guido] > The version of this code found in reality is not as regular as the example > quoted, and the rebuttal "but I would rewrite it with a loop" shoots a straw > man. To me the if-elif-elif portion of the example is very much a separate > motivation, since being able to put the assignment in the elif clause avoids > runaway indentation. I've regretted not being able to use elif in this kind > of situation many times, whereas in the single match case I don't find it a > burden to assign the variable in a separate statement preceding the > if-clause. (I guess this is a case of "flat is better than nested" -- thanks > Tim! :-) Au contraire - thank you for forcing me to channel you succinctly lo those many years ago ;-) And for pointing out this real use case, which I'm not sure has been stressed before. The PEP could clearly use more motivating examples, and this is a fine class of them. Few things are more maddening than runaway cascading indentation :-( And noting again that a simple "binding expression" (my neologism for `identifier ":=" expression`, to break the reflexive horror at imagining the full complexity of assignment statements being allowed everywhere expressions are allowed) is sufficient to address it. From rosuav at gmail.com Sat Apr 21 22:57:36 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 22 Apr 2018 12:57:36 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <9bcde4df-2772-f26e-924a-97544d6bbccc@mgmiller.net> References: <874lk6bwv2.fsf@grothesque.org> <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <9bcde4df-2772-f26e-924a-97544d6bbccc@mgmiller.net> Message-ID: On Sun, Apr 22, 2018 at 12:04 PM, Mike Miller wrote: > Round 2 (Changed order, see below): > > 1. with open(fn) as f: # current behavior > 2. with (open(fn) as f): # same > > 3. with closing(urlopen(url)) as dl: # current behavior > 5. with (closing(urlopen(url)) as dl): # same > > 4. with closing(urlopen(url) as dl): # urlopener named early > > > On 2018-04-20 17:15, Chris Angelico wrote: >> >> The second and fifth could be special cased as either the same as >> first and third, or as SyntaxErrors. (But which?) > > > If they are expressions, they should be the same once evaluated, no? > > (I had a brief episode where I wrote that "as" was required with "with", > instead of the CM object, sorry. :) > >> The fourth one is very tricky. If 'expr as name' is allowed inside >> arbitrary >> expressions, why shouldn't it be allowed there? > > > Yes, they should be allowed there. > >> The disconnect between viable syntax and useful statements is problematic >> here. > > > Number 4 appears to name the urlopener early. Since closing() returns it as > well, might it work anyway? > > Might be missing something else, but #4 looks like a mistake with the layout > of the parentheses, which can happen anywhere. I don't get the sense it > will happen often. It's actually semantically identical to option 3, but *not* semantically identical to option 5, unless there is a magical special case that says that a 'with' statement is permitted to have parentheses for no reason. The 'closing' context manager returns the *inner* CM, not the closing CM itself. If we rewrite these into approximate equivalents without the 'with' statement, what we have is this: > 1. with open(fn) as f: # current behavior file = open(fn) f = file.__enter__() assert file is f # passes for file objects > 2. with (open(fn) as f): # same f = open(fn) f.__enter__() # The return value from enter is discarded > 3. with closing(urlopen(url)) as dl: # current behavior downloader = urlopen(url) closer = closing(downloader) dl = closer.__enter__() assert dl is downloader # passes for closing objects > 5. with (closing(urlopen(url)) as dl): # same downloader = urlopen(url) dl = closing(downloader) dl.__enter__() # Return value from __enter__ is discarded > 4. with closing(urlopen(url) as dl): # urlopener named early dl = urlopen(url) closer = closing(dl) closer.__enter__() # Return value is discarded again Notice how there are five distinctly different cases here. When people say there's a single obvious way to solve the "with (expr as name):" case, they generally haven't thought about all the different possibilities. (And I haven't mentioned the possibility that __enter__ returns something that you can't easily reference from inside the expression, though it's not materially different from closing().) There are a few ways to handle it. One is to create a special case in the grammar for 'with' statement parentheses: with_stmt: 'with' with_item (',' with_item)* ':' suite with_item: (test ['as' expr]) | ('(' test ['as' expr] ')') which will mean that these two do the same thing: with spam as ham: with (spam as ham): but this won't: with ((spam as ham)): And even with that special case, the use of 'as' inside a 'with' statement is subtly different from its behaviour anywhere else, so it would be confusing. So a better way is to straight-up disallow 'as' expressions inside 'with' headers (meaning you get a SyntaxError if the behaviour would be different from the unparenthesized form). Still confusing ("why can't I do this?"). And another way is to just not use 'as' at all, and pick a different syntax. That's why the PEP now recommends ':='. ChrisA From ncoghlan at gmail.com Sun Apr 22 01:33:26 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 22 Apr 2018 15:33:26 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <20180421163153.GY11616@ando.pearwood.info> References: <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <20180421071117.GW11616@ando.pearwood.info> <20180421122643.GX11616@ando.pearwood.info> <20180421163153.GY11616@ando.pearwood.info> Message-ID: On 22 April 2018 at 02:31, Steven D'Aprano wrote: > This is not entirely unprecedented in Python: it is analogous > (although not identical) to binding default values to parameters: > > def running_total(items, total=total): > # Here total is local to the function, but the default > # is taken from the surrounding scope. The stronger precedent for "look up elsewhere until first use" is class scopes: >>> x = "global" >>> class C: ... print(x) ... x = "class attribute to be" ... print(x) ... global class attribute to be However, that has its own quirks, in that it ignores function scopes entirely: >>> def f(): ... x = "function local" ... class C: ... print(x) ... x = "class attribute to be" ... print(x) ... >>> f() global class attribute to be Whereas if you don't rebind the name in the class body, the class scope can see the function local as you'd expect: >>> def f2(): ... x = "function local" ... class C: ... print(x) ... >>> f2() function local While I haven't explicitly researched the full history, my assumption is that references from class scopes prior to a local name rebinding are an edge case that https://www.python.org/dev/peps/pep-0227/ didn't fully account for, so they retain their original pre-PEP-227 behaviour. Cheers, Nick. P.S. It may be becoming clearer why the earlier iterations of PEP 572 proposed sublocal scoping semantics for the new name binding expression: it not only gives greater differentiation from traditional assignments and limits the potential for obviously unwanted side effects like accidentally clobbering a name that's already in use, it also sidesteps a lot of these quirky name resolution issues that arise when you use full local name bindings. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Sun Apr 22 01:48:05 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 22 Apr 2018 15:48:05 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <20180421174125.GZ11616@ando.pearwood.info> References: <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <20180421071117.GW11616@ando.pearwood.info> <20180421122643.GX11616@ando.pearwood.info> <20180421174125.GZ11616@ando.pearwood.info> Message-ID: On 22 April 2018 at 03:41, Steven D'Aprano wrote: > We *already* have the rule that the outermost iterable is special, > except it isn't a rule precisely, since (as far as I know) it isn't > documented anywhere, nor was it ever planned as a feature. It's a deliberate feature of generator expressions: https://www.python.org/dev/peps/pep-0289/#the-details Covered in the language reference here: https://docs.python.org/3/reference/expressions.html#generator-expressions It's then inherited by comprehensions through the intentional semantic equivalences between: [x for x in iterable] <-> list(x for x in iterable) {x for x in iterable} <-> set(x for x in iterable) {k(x):v(x) for x in iterable} <-> set((k(x), v(x) for x in iterable)) The consequences of those equivalences weren't historically spelled out in the language reference, but we fixed that omission when deprecating the use of "yield" & "yield from" in comprehensions for 3.7: https://github.com/python/cpython/commit/73a7e9b10b2ec9636e3c6396cf7b3695f8ed1856 (we had to in order to explain why you *can* still use "yield" and "yield from" in the outermost iterable - it's because the restriction only applies to the implicitly nested scope). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From levkivskyi at gmail.com Sun Apr 22 04:16:14 2018 From: levkivskyi at gmail.com (Ivan Levkivskyi) Date: Sun, 22 Apr 2018 09:16:14 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> <87h8o5x36p.fsf@grothesque.org> Message-ID: Replying also to the list. On 22 April 2018 at 09:14, Ivan Levkivskyi wrote: > On 20 April 2018 at 21:59, Guido van Rossum wrote: > >> Does the PEP currently propose to *allow* that horrible example? I >> thought Tim Peters successfully pleaded to *only* allow a single "NAME := >> ". You don't have to implement this restriction -- we know it's >> possible to implement, and if specifying this alone were to pull enough >> people from -1 to +0 there's a lot of hope! >> >> > * FWIW I an -1 on anything but a simple name. > > * Also Tim proposed a good idea to call these "binding expressions". > Because in contrasts the different purposes. Binding expressions would be > probably typically used to (temporarily) name an expression, while > assignment statements are actually creating "variables" -- long living > names intended to be accessed externally to a class/module. The latter > access can be programmed to trigger arbitrary complex code (see properties, > __getattr__/__setattr__, etc). > > * Re implementing restrictions: there is a CST -> AST step that will allow > to easily prohibit unwanted forms (FWIW this is how unpacking an chaining > is prohibited for annotated assignments). > > * Re using plain "=": Although I am still using this in C quite often, I > was bitten badly by this several times when I was younger, I don't want a > similar experience when _learning_ Python. > > Modulo these points I would be +0 on the PEP. > > -- > Ivan > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From olegs at traiana.com Sun Apr 22 03:47:09 2018 From: olegs at traiana.com (Oleg Sivokon) Date: Sun, 22 Apr 2018 07:47:09 +0000 Subject: [Python-Dev] https://bugs.python.org/issue33127 breaks pip / easy_install / pipenv etc in corporate networks on MS Windows using self-signed certificates In-Reply-To: <15c254a5-23c6-9d13-00a7-4602bef4dc93@python.org> References: , <15c254a5-23c6-9d13-00a7-4602bef4dc93@python.org> Message-ID: On 17Apr2018 0246, Oleg Sivokon wrote: > It is common practice in corporate networks that connect MS Windows ... > If you are referring to Python on Windows, this was never true. We've > always relied on OpenSSL and at best will read locally installed > certificates (and by default, most certificates are not locally > installed). This should not have changed recently, and certainly not > with the bug you reference. I was simply parroting whatever our IT people told me. I don't use MS Windows, and know very little about administration of this OS. I'll be happy to tell them what you just wrote. > I'm asking that this be made configurable / possible to disable using simple means, perhaps an environment variable / registry key or similar. > I'm not clear on what you're asking for. The only thing we can disable > is reading OS certificates into OpenSSL, and that would be the opposite > of what you are having trouble with. I'm still investigating what the actual problem was, and what exactly changed. It might have been related to PyPI using new hosts, but, to be absolutely honest, pip and similar tools don't make it easy to debug this problem. The problem with these tools is that they lose all context information about SSL errors, so it's not possible to know what the exact problem was. Setting up a development environment on MS Windows to try to debug Python interpreter in order to discover this information so far has been frustratingly painful (it's been about a decade since I used MS Windows for anything related to programming). > PS. I still cannot register to the bug tracker (never received a confirmation email), this is why you are reading this email. > I would guess it ended up in a junk mail folder, though that may be > controlled by your organization rather than anywhere you can get to it. > Perhaps using an alternate email address will be easiest? No, it was simply never received (maybe it was somehow filtered out by the MS Exchange filters, I know very little about it, but it never made it to my mailbox). Whatever the case, I will never know that because, apparently, our IT are either too lazy or don't know what they are doing... This communication and all information contained in or attached to it is confidential, intended solely for the addressee, may be legally privileged and is the intellectual property of one of the companies of NEX Group plc ("NEX") or third parties. If you are not the intended addressee or receive this message in error, please immediately delete all copies of it and notify the sender. We have taken precautions to minimise the risk of transmitting software viruses, but we advise you to carry out your own virus checks on any attachments. We do not accept liability for any loss or damage caused by software viruses. NEX reserves the right to monitor all communications. We do not accept any legal responsibility for the content of communications, and no communication shall be considered legally binding. Furthermore, if the content of this communication is personal or unconnected with our business, we accept no liability or responsibility for it. NEX Group plc is a public limited company registered in England and Wales under number 10013770 and certain of its affiliates are authorised and regulated by regulatory authorities. For further regulatory information please see www.NEX.com. From christoph at grothesque.org Sun Apr 22 05:29:06 2018 From: christoph at grothesque.org (Christoph Groth) Date: Sun, 22 Apr 2018 11:29:06 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: (Tim Peters's message of "Sat, 21 Apr 2018 21:22:19 -0500") References: <874lk4wxov.fsf@grothesque.org> <87sh7ouqls.fsf@grothesque.org> Message-ID: <87o9ibvbbx.fsf@grothesque.org> Tim Peters wrote: > [Christoph Groth ] > > I hope to have shown [1] that the same could be done for > > assignments. A consistent value can be defined for any assignment > > statement. So, all assignment statements could be redefined as > > expressions and the language would continue to work and even be > > (perfectly?) backwards-compatible. > > Except for shells. When I type, e.g., > > >>> xs = sorted(iterator_returning_a_billion_strings) > > I really don't want to wait for hours before I can type again ;-) In > the same way now, when someone calls a function at a shell but doesn't > want to see its result, they do something like > > >>> xxx = function(a, b, c) Yes, that's a serious problem with making all assignments expressions. Assignments are so common in interactive use that displaying their values could be quickly annoying. There are several possible solutions. For example, the IPython shell interprets a trailing semicolon as "do not show the result of the expression". A better solution seems to be to only treat assignments that are surrounded by the mandatory parens as expressions and keep the old-style assignments as statements, e.g. >>> a = 3 >>> (a = 3) # currently a SyntaxError 3 So, strictly speaking, there would be distinct assignment statements and expressions, but it would be still easy conceptually because one could say: Any valid assignment statement can be turned into an expression by surrounding it with parentheses. There is no difference in semantics. > There's also that you're not considering the other half: that every > existing assignment statement could be viewed as being as expression > does not imply that every existing assignment statement could be used > everywhere an expression can be used. Syntax matters, and function > call argument lists in particular already bristle with their own > meanings for commas, equal signs, and asterisks. The language was > designed with "and the twain shall never meet" in mind ;-) For > example, what would > > f(a=b) > > mean? It would, of course, mean the same as it does now. (Otherwise backwards compatibility would be broken.) However, f((a=b)) which currently is a syntax error, would mean: bind the value of 'b' to the name 'a' and call 'f' with the value of that expression. The extra parens would be required around any assignment expression. I believe that this also solves all the other problems that you raise with regard to commas etc. So, you see, promoting assignments to expressions is indeed feasible. The advantages are the conceptual simplicity, and the familiar syntax. The syntax is also a disadvantage, because it is somewhat ugly: while (item = get()): process(item) There's also potential for misuse, but firstly that is something that is not unheard of in Python and secondly assignment expressions could be (at least initially) limited to only a subset of the forms that are allowed for assignment statements. If I had to choose between the above and ":= binding expressions", I guess I would tend to prefer the latter because they are sufficient, nicer looking and offer less potential for trouble. But I think that it is worth to fully discuss the above idea as well. IMHO it should be at least briefly mentioned in the "rejected ideas" of PEP 572, because it is arguably the most self-evident way to introduce name-binding expressions into the language. From rosuav at gmail.com Sun Apr 22 05:32:42 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 22 Apr 2018 19:32:42 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <87o9ibvbbx.fsf@grothesque.org> References: <874lk4wxov.fsf@grothesque.org> <87sh7ouqls.fsf@grothesque.org> <87o9ibvbbx.fsf@grothesque.org> Message-ID: On Sun, Apr 22, 2018 at 7:29 PM, Christoph Groth wrote: > If I had to choose between the above and ":= binding expressions", I > guess I would tend to prefer the latter because they are sufficient, > nicer looking and offer less potential for trouble. But I think that it > is worth to fully discuss the above idea as well. IMHO it should be at > least briefly mentioned in the "rejected ideas" of PEP 572, because it > is arguably the most self-evident way to introduce name-binding > expressions into the language. It's in the FAQ. ChrisA From levkivskyi at gmail.com Sun Apr 22 06:19:15 2018 From: levkivskyi at gmail.com (Ivan Levkivskyi) Date: Sun, 22 Apr 2018 11:19:15 +0100 Subject: [Python-Dev] PEP 561 implemented and minor clarification In-Reply-To: References: Message-ID: On 12 April 2018 at 09:59, Ethan Smith wrote: > Hello, > > I've updated PEP 561 to clarify that any installed stub package should > supersede an installed inline package. In other words if there is: > > /global/site-packages/pkg/ > /user/site-packages/pkg-stubs/ > > Even if pkg in the global site packages is found first and marks that it > supports types, the stub package should supersede it. > > I also point to mypy's docs on its implementation of the PEP (which can be > read about here: https://mypy.readthedocs.io/en/latest/installed_packages. > html). The implementation has been merged into master and will be > available in the 0.590 release. > > This clarification totally makes sense for me. I could easily imagine a scenario where are third party provides more advanced/precise/detailed types for a package that already supports typing. Also thanks for writing the implementation, hopefully, this PEP will be accepted soon and it will solve one of the major problems in typing ecosystem. -- Ivan -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Sun Apr 22 07:47:51 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 22 Apr 2018 12:47:51 +0100 Subject: [Python-Dev] Introducing python.zulipchat.com In-Reply-To: References: Message-ID: On 22 April 2018 at 00:05, Brett Cannon wrote: > The Zulip project maintainers are active on our instance so after you join > go to the Zulip stream and start a topic about this. I did - "Zulip -> Sign up". I don't know of a way to put a link to that topic here, but I assume it's findable in the UI. There's been no replies yet. Paul From ncoghlan at gmail.com Sun Apr 22 10:35:25 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 23 Apr 2018 00:35:25 +1000 Subject: [Python-Dev] https://bugs.python.org/issue33127 breaks pip / easy_install / pipenv etc in corporate networks on MS Windows using self-signed certificates In-Reply-To: References: Message-ID: On 17 April 2018 at 19:46, Oleg Sivokon wrote: > It is common practice in corporate networks that connect MS Windows machines to redirect all (encrypted included) traffic through company's router. If this has only started happening recently, then the cause is more likely to be related to PyPI switching over to requiring TLS v1.2+ for all client access: a sufficiently outdated man-in-the-middle SSL/TLS proxy may not be able to meet PyPI's client security requirements. See https://pyfound.blogspot.com.au/2017/01/time-to-upgrade-your-python-tls-v12.html for more details. (If that's the problem, it would affect all versions of Python equally though). Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Sun Apr 22 10:39:23 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 23 Apr 2018 00:39:23 +1000 Subject: [Python-Dev] Introducing python.zulipchat.com In-Reply-To: References: Message-ID: On 22 April 2018 at 21:47, Paul Moore wrote: > On 22 April 2018 at 00:05, Brett Cannon wrote: >> The Zulip project maintainers are active on our instance so after you join >> go to the Zulip stream and start a topic about this. > > I did - "Zulip -> Sign up". I don't know of a way to put a link to > that topic here, but I assume it's findable in the UI. For anyone else with the same question: clicking on the topic title in either the side navbar or the stream overview will give you a view specific to the topic that also serves as a shareable link. In this case: https://python.zulipchat.com/#narrow/stream/116410-zulip/topic/Sign.20up Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From kirillbalunov at gmail.com Sun Apr 22 07:10:48 2018 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Sun, 22 Apr 2018 14:10:48 +0300 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> <87h8o5x36p.fsf@grothesque.org> Message-ID: 2018-04-21 4:44 GMT+03:00 Tim Peters : > [Chris Angelico ] > > I don't see much value in restricting the assignment target to names > > only, but if that's what it takes, it can be restricted, at least > > initially. > > I believe this point was made most clearly before by Terry Reedy, but > it bears repeating :-) This is from the PEP's motivation: > > """ > Naming the result of an expression is an important part of > programming, allowing a descriptive name to be used in place of a > longer expression, and permitting reuse. > """ > > As "head arguments" go, that's a good one! But restricting assignment > expressions to > > identifier ":=" expression > > satisfies it. If what's of value is to name the result of an > expression, that single case handles that and _only_ that. In a > sense, it's "the simplest thing that could possibly work", and that's > generally a good thing to aim for. > > Python assignment _statements_ are way more complex than that. > Besides just giving names to expression results, they can also > implicitly invoke arbitrarily complex __setitem__ and __setattr__ > methods on targets, rely on all sorts of side effects across chained > assignments, and support funky syntax for magically iterating over an > expression's iterable result. > > While that can all be useful _in_ an assignment statement, the PEP's > motivation doesn't say a word about why any of _that_ would also be > useful buried inside an assignment expression. There doesn't appear > to be a good "head argument" for why, besides "why not?". That's not > enough. > I agree with you. During the discussion on python-ideas there was not explicitly suggested to limit assignment target to name only but that was often implicitly implied. So explicit is better than implicit :) The main reason for such criticism was related to the fact that almost all of the examples from the PEP use `name := expression` form. Also it was noted that 99% of use-cases where this feature will be _nice_ to have is `while` and `if` statements (including ternary from). Although one of the prerequisites for writing this PEP was the use of the assignment expression in the lists, it will rarely be used in them, and even more rarely it will be a justified usage of. In addition, in the case of the general assignment expression and the chosen operator `: =`, which solves the problem of distinctness from `==`, I see no reason, or more precisely how to explain, why will not other forms `+=`, `*=` become expressions either? And then we are faced with with all the beauty of side effects, sequnce points, ... And while in Python it's much easier to resolve this - Python will no longer be Python. I'm glad that this does not happen. Since the discussion moves towards a simplified form - `binding expression`, where assignment target can be name only. Will you be _happy_ with the choice of `:=` operator? Which is perceived as `=`, but with very limited capabilities. Therefore, as I see it, with this _limited power_ it is one of design goals to make the syntax forms of `assignment statement` and `assignment expression` to be distinct and `:=` does not help with this. This does not mean that this new syntax form should not be convenient, but it should be different from the usual `=` form. Otherwise, the question about ".1 + .2" will have competitors :-) > I think it's no coincidence that every example of an _intended_ use is > of the simple > > identifier ":=" expression > > form. There are no examples of fancier targets in the PEP, and - more > importantly - also none I saw in the hundreds of mailing-list messages > since this started. Except for a few of mine, where I tried to > demonstrate why _trying_ fancier targets in examples derived from real > code made the original "loop and a half" code _worse_ And where other > people were illustrating how incomprehensibly code _could_ be written > (which isn't a real interest of mine). With kind regards, -gdg -------------- next part -------------- An HTML attachment was scrubbed... URL: From kirillbalunov at gmail.com Sun Apr 22 12:22:31 2018 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Sun, 22 Apr 2018 19:22:31 +0300 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> <87h8o5x36p.fsf@grothesque.org> Message-ID: 2018-04-22 14:10 GMT+03:00 Kirill Balunov : > > Although one of the prerequisites for writing this PEP was the use of the > assignment expression in the lists > Sorry, typo: in compehensions/generators. > it will rarely be used in them, and even more rarely it will be a > justified usage of. > With kind regards, -gdg > -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Sun Apr 22 15:01:16 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 22 Apr 2018 20:01:16 +0100 Subject: [Python-Dev] Introducing python.zulipchat.com In-Reply-To: References: Message-ID: On 22 April 2018 at 15:39, Nick Coghlan wrote: > On 22 April 2018 at 21:47, Paul Moore wrote: >> On 22 April 2018 at 00:05, Brett Cannon wrote: >>> The Zulip project maintainers are active on our instance so after you join >>> go to the Zulip stream and start a topic about this. >> >> I did - "Zulip -> Sign up". I don't know of a way to put a link to >> that topic here, but I assume it's findable in the UI. > > For anyone else with the same question: clicking on the topic title in > either the side navbar or the stream overview will give you a view > specific to the topic that also serves as a shareable link. In this > case: > > https://python.zulipchat.com/#narrow/stream/116410-zulip/topic/Sign.20up > lol, I'm so used to web apps *not* providing URLs specific to where you are in them (I'm not even sure what the term for this is - "doing the web properly"? :-)) that it never even occurred to me that the link in my browser bar was usable! Paul From wes.turner at gmail.com Sun Apr 22 15:08:16 2018 From: wes.turner at gmail.com (Wes Turner) Date: Sun, 22 Apr 2018 15:08:16 -0400 Subject: [Python-Dev] Introducing python.zulipchat.com In-Reply-To: References: Message-ID: On Sunday, April 22, 2018, Paul Moore wrote: > On 22 April 2018 at 15:39, Nick Coghlan wrote: > > On 22 April 2018 at 21:47, Paul Moore wrote: > >> On 22 April 2018 at 00:05, Brett Cannon wrote: > >>> The Zulip project maintainers are active on our instance so after you > join > >>> go to the Zulip stream and start a topic about this. > >> > >> I did - "Zulip -> Sign up". I don't know of a way to put a link to > >> that topic here, but I assume it's findable in the UI. > > > > For anyone else with the same question: clicking on the topic title in > > either the side navbar or the stream overview will give you a view > > specific to the topic that also serves as a shareable link. In this > > case: > > > > https://python.zulipchat.com/#narrow/stream/116410-zulip/ > topic/Sign.20up > > > > lol, I'm so used to web apps *not* providing URLs specific to where > you are in them (I'm not even sure what the term for this is - "doing > the web properly"? :-)) that it never even occurred to me that the > link in my browser bar was usable! "Deep linking" with a "#fragment-identifier". https://en.wikipedia.org/wiki/Deep_linking https://en.wikipedia.org/wiki/Fragment_identifier > > Paul > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > wes.turner%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From python-dev at mgmiller.net Sun Apr 22 15:46:45 2018 From: python-dev at mgmiller.net (Mike Miller) Date: Sun, 22 Apr 2018 12:46:45 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <9bcde4df-2772-f26e-924a-97544d6bbccc@mgmiller.net> Message-ID: On 2018-04-21 19:57, Chris Angelico wrote: Thanks for being patient. Looks like the crux of the issue is that "with ? as" binds the result of the enter function rather than the context-manager object, as it might first appear. Therefore it's not compatible with how "as" is used for direct name bindings after "import" statements or this sub-proposal. Additionally, "except Class as instance" names the instance rather than the class. So, the "as" keyword is already operating at an intuitive level rather than idealistic perfection. Three different semantics for import/with/except, correct? This sub-proposal lines up with the import use, I believe. Given that there are no use cases for using assignment-expressions in the import/with/except statements, and it could be documented that if one insists an extra set of parens could make it work: with (callable() as cm_obj) as enter_result_obj: pass It doesn't feel like this issue should be a blocker. TL;DR - Been feebly trying to make the argument that everyday "intuitive consistency" (where the expression will be used) is more important than avoiding theoretical problems. I've never seen complex with/except statements in the wild and don't expect this feature to significantly alter that. -Mike From python-dev at mgmiller.net Sun Apr 22 16:22:42 2018 From: python-dev at mgmiller.net (Mike Miller) Date: Sun, 22 Apr 2018 13:22:42 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <9bcde4df-2772-f26e-924a-97544d6bbccc@mgmiller.net> Message-ID: <2611cc3d-0a7b-20a8-8a22-8732a9698f1f@mgmiller.net> On 2018-04-22 12:37, Chris Angelico wrote: > Kinda, except that that's not quite a match either. But mainly, the > comparison with 'with' and 'except' is dangerously incompatible. Hmm, looks very close conceptually, though mechanics are different. Dangerous feels like an exaggeration however. I've made the argument that occurrences would be very rare, but if I'm wrong, the code should blow up on its first run. Perhaps a sanity check could be put in? There is a section of your PEP that argues against the "bad code could potentially be written" argument, and think it applies here. > Maybe not, but why not just use ':=' to avoid that? Don't hate it but feels like Pascal and C and not "Pythonic." Too many colons, avoiding the questions about the difference between "=" and ":=". Expression first is another win. People know how to use "as". > Intuitive consistency isn't enough to handle complex cases. > Programming languages that favour intuitive consistency end up with a > million special cases. Ok, but I think we have all the tools we need here, there's just an extra place to stub your toe out in the weeds. To turn the question around, are we really worried that this awkward code (or some variant) is about to be written? with (cm_obj := callable()) as enter_result_obj: cm_obj.write() # AttributeError If not, I argue it is a theoretical problem that, if hit, blows up immediately. -Mike From rosuav at gmail.com Sun Apr 22 17:33:35 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Apr 2018 07:33:35 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <2611cc3d-0a7b-20a8-8a22-8732a9698f1f@mgmiller.net> References: <874lk6bwv2.fsf@grothesque.org> <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <9bcde4df-2772-f26e-924a-97544d6bbccc@mgmiller.net> <2611cc3d-0a7b-20a8-8a22-8732a9698f1f@mgmiller.net> Message-ID: On Mon, Apr 23, 2018 at 6:22 AM, Mike Miller wrote: > On 2018-04-22 12:37, Chris Angelico wrote: >> Kinda, except that that's not quite a match either. But mainly, the >> comparison with 'with' and 'except' is dangerously incompatible. > > Hmm, looks very close conceptually, though mechanics are different. > > Dangerous feels like an exaggeration however. I've made the argument that > occurrences would be very rare, but if I'm wrong, the code should blow up on > its first run. Perhaps a sanity check could be put in? with open(fn) as f: with (open(fn) as f): These two do the same thing, but only because a file object's __enter__ returns self. So it's dangerous, because it WILL work... and people will get into the habit of parenthesizing to permit a 'with' statement to go across line breaks. And then they'll use a different context manager, like closing(), or a PsycoPG2 database connection (I think), where it returns something else. And it'll work, until they go over multiple lines, and then suddenly the semantics change. It's as bad as writing JavaScript code like this: function f(x) { return x + 1; } and then transforming it to this: function f(x) { return x + 1; } and having it change in behaviour. (Yes, it happens. Welcome to JavaScript, where implicit semicolons are a thing.) >> Intuitive consistency isn't enough to handle complex cases. >> Programming languages that favour intuitive consistency end up with a >> million special cases. > > Ok, but I think we have all the tools we need here, there's just an extra > place to stub your toe out in the weeds. > > To turn the question around, are we really worried that this awkward code > (or some variant) is about to be written? > > with (cm_obj := callable()) as enter_result_obj: > cm_obj.write() # AttributeError > > If not, I argue it is a theoretical problem that, if hit, blows up > immediately. Were it to blow up immediately, I wouldn't be too bothered. ChrisA From steve.dower at python.org Sun Apr 22 18:13:32 2018 From: steve.dower at python.org (Steve Dower) Date: Sun, 22 Apr 2018 15:13:32 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> Message-ID: This example makes me want ?if expr as name:? (same semantics as ?with?, and the name is always bound to the expression result regardless of truthiness), but doesn?t move me on assignment expressions. Cheers, Steve Top-posted from my Windows phone From: Guido van Rossum Sent: Saturday, April 21, 2018 19:09 To: Steven D'Aprano Cc: Python-Dev Subject: Re: [Python-Dev] PEP 572: Assignment Expressions On Sat, Apr 21, 2018 at 6:13 PM, Steven D'Aprano wrote: On Sat, Apr 21, 2018 at 08:35:51PM +0100, Matthew Woodcraft wrote: > Well, that's a reason to make the example a bit more realistic, then. > > Say: > > if match := re.search(pat1, text): >? ? ?do_something_with(match.group(0)) > elif match := re.search(pat2, text): >? ? ?do_something_else_with(match.group(0), match.group(1)) > elif match := re.search(pat3, text): >? ? ?do_some_other_things_with(match.group(0)) >? ? ?and_also_with(match.group(1), match.group(2)) I don't think that a bunch of generic "do_something_with" functions is precisely "realistic". If I saw something like that, I'd try very hard to find a way to refactor it into code like this: for handler in handlers: ? ? if handler.match(text): ? ? ? ? handler.process() ? ? ? ? break else: ? ? # handle no-match case here where the knowledge of what to search for, where to search for it, how to search for it, and what to do when found, was encapsulated in the handler objects. Your tastes may vary. But your point is well-taken that the version with binding assignment (thanks Tim!) is nicer to read than the current procedural version: match = re.search(pat1, text) if match: ? ? do_something_with(match.group(0)) else: ? ? match = re.search(pat2, text) ? ? if match: ? ? ? ? do_something_else_with(match.group(0), match.group(1)) ? ? else: ? ? ? ? match = = re.search(pat3, text) ? ? ? ? do_some_other_things_with(match.group(0)) ? ? ? ? and_also_with(match.group(1), match.group(2)) I just don't think it counts as a motivating use-case distinct from the single match case. The version of this code found in reality is not as regular as the example quoted, and the rebuttal "but I would rewrite it with a loop" shoots a straw man. To me the if-elif-elif portion of the example is very much a separate motivation, since being able to put the assignment in the elif clause avoids runaway indentation. I've regretted not being able to use elif in this kind of situation many times, whereas in the single match case I don't find it a burden to assign the variable in a separate statement preceding the if-clause. (I guess this is a case of "flat is better than nested" -- thanks Tim! :-) -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From python-dev at mgmiller.net Sun Apr 22 18:20:49 2018 From: python-dev at mgmiller.net (Mike Miller) Date: Sun, 22 Apr 2018 15:20:49 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <9bcde4df-2772-f26e-924a-97544d6bbccc@mgmiller.net> <2611cc3d-0a7b-20a8-8a22-8732a9698f1f@mgmiller.net> Message-ID: <393f85fd-c378-4fd3-14ae-7301eeb62f07@mgmiller.net> On 2018-04-22 14:33, Chris Angelico wrote: > with open(fn) as f: > with (open(fn) as f): > > These two do the same thing, but only because a file object's > __enter__ returns self. So it's dangerous, because it WILL work... and > people will get into the habit of parenthesizing to permit a 'with' > statement to go across line breaks. And then they'll use a different > context manager, like closing(), or a PsycoPG2 database connection (I > think), where it returns something else. And it'll work, until they go > over multiple lines, and then suddenly the semantics change. Why do you think folks will be rushing to parenthesize with statements when it has always been a syntax error, there is years of code and docs that show otherwise, no use cases, and will take years for 3.8 to trickle out? Seems remote, and there are mitigations that could be done. Again it's back to "people could write bad code," but they already can with only +/* and (). -Mike From rosuav at gmail.com Sun Apr 22 18:27:42 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Apr 2018 08:27:42 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <393f85fd-c378-4fd3-14ae-7301eeb62f07@mgmiller.net> References: <874lk6bwv2.fsf@grothesque.org> <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <9bcde4df-2772-f26e-924a-97544d6bbccc@mgmiller.net> <2611cc3d-0a7b-20a8-8a22-8732a9698f1f@mgmiller.net> <393f85fd-c378-4fd3-14ae-7301eeb62f07@mgmiller.net> Message-ID: On Mon, Apr 23, 2018 at 8:20 AM, Mike Miller wrote: > > On 2018-04-22 14:33, Chris Angelico wrote: >> >> with open(fn) as f: >> with (open(fn) as f): >> >> These two do the same thing, but only because a file object's >> __enter__ returns self. So it's dangerous, because it WILL work... and >> people will get into the habit of parenthesizing to permit a 'with' >> statement to go across line breaks. And then they'll use a different >> context manager, like closing(), or a PsycoPG2 database connection (I >> think), where it returns something else. And it'll work, until they go >> over multiple lines, and then suddenly the semantics change. > > > > Why do you think folks will be rushing to parenthesize with statements when > it has always been a syntax error, there is years of code and docs that show > otherwise, no use cases, and will take years for 3.8 to trickle out? Because it's been requested a number of times as a way to allow a 'with' statement to go across lines without backslashes. If it becomes possible, it will be used. ChrisA From guido at python.org Mon Apr 23 00:55:39 2018 From: guido at python.org (Guido van Rossum) Date: Sun, 22 Apr 2018 21:55:39 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> Message-ID: On Sun, Apr 22, 2018 at 3:13 PM, Steve Dower wrote: > This example makes me want ?if expr as name:? (same semantics as ?with?, > and the name is always bound to the expression result regardless of > truthiness), but doesn?t move me on assignment expressions. > In reality there often are other conditions being applied to the match for which `if expr as name` is inadequate. The simplest would be something like if ...: elif (m := re.match('(.*):(.*)', line)) and m.group(1) == m.group(2): And the match() call may not even be the first thing to check -- e.g. we could have elif line is not None and (m := re.match('(.*):(.*)', line)) and m.group(1) == m.group(2): -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon Apr 23 00:58:16 2018 From: guido at python.org (Guido van Rossum) Date: Sun, 22 Apr 2018 21:58:16 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <1f1cce2f-c63f-9a8e-c3c2-717b25aa58f5@mgmiller.net> <5e9c96c3-54a8-3c20-2ea5-2788a27bac56@mgmiller.net> <9bcde4df-2772-f26e-924a-97544d6bbccc@mgmiller.net> <2611cc3d-0a7b-20a8-8a22-8732a9698f1f@mgmiller.net> <393f85fd-c378-4fd3-14ae-7301eeb62f07@mgmiller.net> Message-ID: Please stop debating ` as `. Nobody is being swayed by anything in this subthread. Let's move on. On Sun, Apr 22, 2018 at 3:27 PM, Chris Angelico wrote: > On Mon, Apr 23, 2018 at 8:20 AM, Mike Miller > wrote: > > > > On 2018-04-22 14:33, Chris Angelico wrote: > >> > >> with open(fn) as f: > >> with (open(fn) as f): > >> > >> These two do the same thing, but only because a file object's > >> __enter__ returns self. So it's dangerous, because it WILL work... and > >> people will get into the habit of parenthesizing to permit a 'with' > >> statement to go across line breaks. And then they'll use a different > >> context manager, like closing(), or a PsycoPG2 database connection (I > >> think), where it returns something else. And it'll work, until they go > >> over multiple lines, and then suddenly the semantics change. > > > > > > > > Why do you think folks will be rushing to parenthesize with statements > when > > it has always been a syntax error, there is years of code and docs that > show > > otherwise, no use cases, and will take years for 3.8 to trickle out? > > Because it's been requested a number of times as a way to allow a > 'with' statement to go across lines without backslashes. > > If it becomes possible, it will be used. > > ChrisA > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.peters at gmail.com Mon Apr 23 01:44:44 2018 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 23 Apr 2018 00:44:44 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> Message-ID: [Guido] > In reality there often are other conditions being applied to the match for > which `if expr as name` is inadequate. The simplest would be something like > > if ...: > > elif (m := re.match('(.*):(.*)', line)) and m.group(1) == m.group(2): > > > And the match() call may not even be the first thing to check -- e.g. we > could have > > elif line is not None and (m := re.match('(.*):(.*)', line)) and m.group(1) == m.group(2): I find myself warming more to binding expressions the more I keep them in mind while writing new code. And I think it may be helpful to continue showing real examples where they would help. Today's example: I happened to code this a few hours ago: diff = x - x_base if diff: g = gcd(diff, n) if g > 1: return g It's not really hard to follow, but two levels of nesting "feels excessive", as does using the names "diff" and "g" three times each. It's _really_ an "and" test: if the diff isn't 0 and gcd(diff, n) > 1, return the gcd. That's how I _thought_ of it from the start. Which this alternative expresses directly: if (diff := x - x_base) and (g := gcd(diff, n)) > 1: return g That's so Pythonic I could cry ;-) From solipsis at pitrou.net Mon Apr 23 03:28:06 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 23 Apr 2018 09:28:06 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> Message-ID: <20180423092806.530b5996@fsol> On Mon, 23 Apr 2018 00:44:44 -0500 Tim Peters wrote: > > Which this alternative expresses directly: > > if (diff := x - x_base) and (g := gcd(diff, n)) > 1: > return g > > That's so Pythonic I could cry ;-) It looks like C to me. That won't make me cry (I write C++ code daily these days), but it's certainly not the same language as Python. The second part, especially, where you use the result of an assignment expression as a comparison operand, looks definitely un-Pythonic. Regards Antoine. From gmarcel.plch at gmail.com Mon Apr 23 06:36:36 2018 From: gmarcel.plch at gmail.com (Marcel Plch) Date: Mon, 23 Apr 2018 12:36:36 +0200 Subject: [Python-Dev] PEP 573 -- Module State Access from C Extension Methods Message-ID: Hello, I am an intern at Red Hat mentored by Petr Viktorin. As a part of my internship, I learned the CPython internals and how to contribute to the CPython interpreter. As a result, I have prepared PEP 573, which solves some problems that PEP 489 (Multi-phase extension module initialization) has left open. Specifically, this PEP proposes a way to access per-module state from methods of built-in and extension types. Like PEP 489, it aims to make subinterpreter-friendly built-in/extension modules easier to create. A big problem found when converting many modules to PEP 489 multi-phase initialization is subinterpreter-friendly access to exception types defined in built-in/extension modules. This PEP solves this by introducing "immutable exception types". The current implementation requires one new type flag and two new pointers in the heap type structure. It should be possible to remove eiher the flag or one of the two pointers, if we agree on the other mechanics in the PEP . =================== PEP: 573 Title: Module State Access from C Extension Methods Version: $Revision$ Last-Modified: $Date$ Author: Petr Viktorin , Nick Coghlan , Eric Snow , Marcel Plch Discussions-To: import-sig at python.org Status: Active Type: Process Content-Type: text/x-rst Created: 02-Jun-2016 Python-Version: 3.8 Post-History: Abstract ======== This PEP proposes to add a way for CPython extension methods to access context such as the state of the modules they are defined in. This will allow extension methods to use direct pointer dereferences rather than PyState_FindModule for looking up module state, reducing or eliminating the performance cost of using module-scoped state over process global state. This fixes one of the remaining roadblocks for adoption of PEP 3121 (Extension module initialization and finalization) and PEP 489 (Multi-phase extension module initialization). Additionaly, support for easier creation of immutable exception classes is added. This removes the need for keeping per-module state if it would only be used for exception classes. While this PEP takes an additional step towards fully solving the problems that PEP 3121 and PEP 489 started tackling, it does not attempt to resolve *all* remaining concerns. In particular, accessing the module state from slot methods (``nb_add``, etc) remains slower than accessing that state from other extension methods. Terminology =========== Process-Global State -------------------- C-level static variables. Since this is very low-level memory storage, it must be managed carefully. Per-module State ---------------- State local to a module object, allocated dynamically as part of a module object's initialization. This isolates the state from other instances of the module (including those in other subinterpreters). Accessed by ``PyModule_GetState()``. Static Type ----------- A type object defined as a C-level static variable, i.e. a compiled-in type object. A static type needs to be shared between module instances and has no information of what module it belongs to. Static types do not have ``__dict__`` (although their instances might). Heap Type --------- A type object created at run time. Rationale ========= PEP 489 introduced a new way to initialize extension modules, which brings several advantages to extensions that implement it: * The extension modules behave more like their Python counterparts. * The extension modules can easily support loading into pre-existing module objects, which paves the way for extension module support for ``runpy`` or for systems that enable extension module reloading. * Loading multiple modules from the same extension is possible, which makes testing module isolation (a key feature for proper sub-interpreter support) possible from a single interpreter. The biggest hurdle for adoption of PEP 489 is allowing access to module state from methods of extension types. Currently, the way to access this state from extension methods is by looking up the module via ``PyState_FindModule`` (in contrast to module level functions in extension modules, which receive a module reference as an argument). However, ``PyState_FindModule`` queries the thread-local state, making it relatively costly compared to C level process global access and consequently deterring module authors from using it. Also, ``PyState_FindModule`` relies on the assumption that in each subinterpreter, there is at most one module corresponding to a given ``PyModuleDef``. This does not align well with Python's import machinery. Since PEP 489 aimed to fix that, the assumption does not hold for modules that use multi-phase initialization, so ``PyState_FindModule`` is unavailable for these modules. A faster, safer way of accessing module-level state from extension methods is needed. Immutable Exception Types ------------------------- For isolated modules to work, any class whose methods touch module state must be a heap type, so that each instance of a module can have its own type object. With the changes proposed in this PEP, heap type instances will have access to module state without global registration. But, to create instances of heap types, one will need the module state in order to get the type object corresponding to the appropriate module. In short, heap types are "viral" ? anything that ?touches? them must itself be a heap type. Curently, most exception types, apart from the ones in ``builtins``, are heap types. This is likely simply because there is a convenient way to create them: ``PyErr_NewException``. Heap types generally have a mutable ``__dict__``. In most cases, this mutability is harmful. For example, exception types from the ``sqlite`` module are mutable and shared across subinterpreters. This allows "smuggling" values to other subinterpreters via attributes of ``sqlite3.Error``. Moreover, since raising exceptions is a common operation, and heap types will be "viral", ``PyErr_NewException`` will tend to "infect" the module with "heap type-ness" ? at least if the module decides play well with subinterpreters/isolation. Many modules could go without module state entirely if the exception classes were immutable. To solve this problem, a new function for creating immutable exception types is proposed. Background =========== The implementation of a Python method may need access to one or more of the following pieces of information: * The instance it is called on (``self``) * The underlying function * The class the method was defined in * The corresponding module * The module state In Python code, the Python-level equivalents may be retrieved as:: import sys def meth(self): instance = self module_globals = globals() module_object = sys.modules[__name__] # (1) underlying_function = Foo.meth # (1) defining_class = Foo # (1) defining_class = __class__ # (2) .. note:: The defining class is not ``type(self)``, since ``type(self)`` might be a subclass of ``Foo``. The statements marked (1) implicitly rely on name-based lookup via the function's ``__globals__``: either the ``Foo`` attribute to access the defining class and Python function object, or ``__name__`` to find the module object in ``sys.modules``. In Python code, this is feasible, as ``__globals__`` is set appropriately when the function definition is executed, and even if the namespace has been manipulated to return a different object, at worst an exception will be raised. The ``__class__`` closure, (2), is a safer way to get the defining class, but it still relies on ``__closure__`` being set appropriately. By contrast, extension methods are typically implemented as normal C functions. This means that they only have access to their arguments and C level thread-local and process-global states. Traditionally, many extension modules have stored their shared state in C-level process globals, causing problems when: * running multiple initialize/finalize cycles in the same process * reloading modules (e.g. to test conditional imports) * loading extension modules in subinterpreters PEP 3121 attempted to resolve this by offering the ``PyState_FindModule`` API, but this still has significant problems when it comes to extension methods (rather than module level functions): * it is markedly slower than directly accessing C-level process-global state * there is still some inherent reliance on process global state that means it still doesn't reliably handle module reloading It's also the case that when looking up a C-level struct such as module state, supplying an unexpected object layout can crash the interpreter, so it's significantly more important to ensure that extension methods receive the kind of object they expect. Proposal ======== Currently, a bound extension method (``PyCFunction`` or ``PyCFunctionWithKeywords``) receives only ``self``, and (if applicable) the supplied positional and keyword arguments. While module-level extension functions already receive access to the defining module object via their ``self`` argument, methods of extension types don't have that luxury: they receive the bound instance via ``self``, and hence have no direct access to the defining class or the module level state. The additional module level context described above can be made available with two changes. Both additions are optional; extension authors need to opt in to start using them: * Add a pointer to the module to heap type objects. * Pass the defining class to the underlying C function. The defining class is readily available at the time built-in method object (``PyCFunctionObject``) is created, so it can be stored in a new struct that extends ``PyCFunctionObject``. The module state can then be retrieved from the module object via ``PyModule_GetState``. Note that this proposal implies that any type whose method needs to access per-module state must be a heap type, rather than a static type. This is necessary to support loading multiple module objects from a single extension: a static type, as a C-level global, has no information about which module it belongs to. Slot methods ------------ The above changes don't cover slot methods, such as ``tp_iter`` or ``nb_add``. The problem with slot methods is that their C API is fixed, so we can't simply add a new argument to pass in the defining class. Two possible solutions have been proposed to this problem: * Look up the class through walking the MRO. This is potentially expensive, but will be useful if performance is not a problem (such as when raising a module-level exception). * Storing a pointer to the defining class of each slot in a separate table, ``__typeslots__`` [#typeslots-mail]_. This is technically feasible and fast, but quite invasive. Due to the invasiveness of the latter approach, this PEP proposes adding an MRO walking helper for use in slot method implementations, deferring the more complex alternative as a potential future optimisation. Modules affected by this concern also have the option of using thread-local state or PEP 567 context variables, or else defining their own reload-friendly lookup caching scheme. Immutable Exception Types ------------------------- To faciliate creating static exception classes, a new function is proposed: ``PyErr_PrepareImmutableException``. It will work similarly to ``PyErr_NewExceptionWithDoc`` but will take a ``PyTypeObject **`` pointer, which points to a ``PyTypeObject *`` that is either ``NULL`` or an initialized ``PyTypeObject``. This pointer may be declared in process-global state. The function will then allocate the object and will keep in mind that already existing exception should not be overwritten. The extra indirection makes it possible to make ``PyErr_PrepareImmutableException`` part of the stable ABI by having the Python interpreter, rather than extension code, allocate the ``PyTypeObject``. Specification ============= Adding module references to heap types -------------------------------------- The ``PyHeapTypeObject`` struct will get a new member, ``PyObject *ht_module``, that can store a pointer to the module object for which the type was defined. It will be ``NULL`` by default, and should not be modified after the type object is created. A new factory method will be added for creating modules:: PyObject* PyType_FromModuleAndSpec(PyObject *module, PyType_Spec *spec, PyObject *bases) This acts the same as ``PyType_FromSpecWithBases``, and additionally sets ``ht_module`` to the provided module object. Additionally, an accessor, ``PyObject * PyType_GetModule(PyTypeObject *)`` will be provided. It will return the ``ht_module`` if a heap type with module pointer set is passed in, otherwise it will set a SystemError and return NULL. Usually, creating a class with ``ht_module`` set will create a reference cycle involving the class and the module. This is not a problem, as tearing down modules is not a performance-sensitive operation (and module-level functions typically also create reference cycles). The existing "set all module globals to None" code that breaks function cycles through ``f_globals`` will also break the new cycles through ``ht_module``. Passing the defining class to extension methods ----------------------------------------------- A new style of C-level functions will be added to the current selection of ``PyCFunction`` and ``PyCFunctionWithKeywords``:: PyObject *PyCMethod(PyObject *self, PyTypeObject *defining_class, PyObject *args, PyObject *kwargs) A new method object flag, ``METH_METHOD``, will be added to signal that the underlying C function is ``PyCMethod``. To hold the extra information, a new structure extending ``PyCFunctionObject`` will be added:: typedef struct { PyCFunctionObject func; PyTypeObject *mm_class; /* Passed as 'defining_class' arg to the C func */ } PyCMethodObject; To allow passing the defining class to the underlying C function, a change to private API is required, now ``_PyMethodDef_RawFastCallDict`` and ``_PyMethodDef_RawFastCallKeywords`` will receive ``PyTypeObject *cls`` as one of their arguments. A new macro ``PyCFunction_GET_CLASS(cls)`` will be added for easier access to mm_class. Method construction and calling code and will be updated to honor ``METH_METHOD``. Argument Clinic --------------- To support passing the defining class to methods using Argument Clinic, a new converter will be added to clinic.py: ``defining_class``. Each method may only have one argument using this converter, and it must appear after ``self``, or, if ``self`` is not used, as the first argument. The argument will be of type ``PyTypeObject *``. When used, Argument Clinic will select ``METH_METHOD`` as the calling convention. The argument will not appear in ``__text_signature__``. This will be compatible with ``__init__`` and ``__new__`` methods, where an MRO walker will be used to pass the defining class from clinic generated code to the user's function. Slot methods ------------ To allow access to per-module state from slot methods, an MRO walker will be implemented:: PyTypeObject *PyType_DefiningTypeFromSlotFunc(PyTypeObject *type, int slot, void *func) The walker will go through bases of heap-allocated ``type`` and search for class that defines ``func`` at its ``slot``. The ``func`` needs not to be inherited by ``type``, only requirement for the walker to find the defining class is that the defining class must be heap-allocated. On failure, exception is set and NULL is returned. Static exceptions ----------------- A new function will be added:: int PyErr_PrepareImmutableException(PyTypeObject **exc, const char *name, const char *doc, PyObject *base) Creates an immutable exception type which can be shared across multiple module objects. If the type already exists (determined by a process-global pointer, ``*exc``), skip the initialization and only ``INCREF`` it. If ``*exc`` is NULL, the function will allocate a new exception type and initialize it using given parameters the same way ``PyType_FromSpecAndBases`` would. The ``doc`` and ``base`` arguments may be ``NULL``, defaulting to a missing docstring and ``PyExc_Exception`` base class, respectively. The exception type's ``tp_flags`` will be set to values common to built-in exceptions and the ``Py_TPFLAGS_HEAP_IMMUTABLE`` flag (see below) will be set. On failure, ``PyErr_PrepareImmutableException`` will set an exception and return -1. If called with an initialized exception type (``*exc`` is non-NULL), the function will do nothing but incref ``*exc``. A new flag, ``Py_TPFLAGS_HEAP_IMMUTABLE``, will be added to prevent mutation of the type object. This makes it possible to share the object safely between multiple interpreters. This flag is checked in ``type_setattro`` and blocks setting of attributes when set, similar to built-in types. A new pointer, ``ht_moduleptr``, will be added to heap types to store ``exc``. On deinitialization of the exception type, ``*exc`` will be set to ``NULL``. This makes it safe for ``PyErr_PrepareImmutableException`` to check if the exception was already initialized. PyType_offsets -------------- Some extension types are using instances with ``__dict__`` or ``__weakref__`` allocated. Currently, there is no way of passing offsets of these through ``PyType_Spec``. To allow this, a new structure and a spec slot are proposed. A new structure, ``PyType_offsets``, will have two members containing the offsets of ``__dict__`` and ``__weakref__``:: typedef struct { Py_ssize_t dict; Py_ssize_t weaklist; } PyType_offsets; The new slot, ``Py_offsets``, will be used to pass a ``PyType_offsets *`` structure containing the mentioned data. Helpers ------- Getting to per-module state from a heap type is a very common task. To make this easier, a helper will be added:: void *PyType_GetModuleState(PyObject *type) This function takes a heap type and on success, it returns pointer to state of the module that the heap type belongs to. On failure, two scenarios may occure. When a type without a module is passed in, ``SystemError`` is set and ``NULL`` returned. If the module is found, pointer to the state, which may be ``NULL``, is returned without setting any exception. Modules Converted in the Initial Implementation ----------------------------------------------- To validate the approach, several modules will be modified during the initial implementation: The ``zipimport``, ``_io``, ``_elementtree``, and ``_csv`` modules will be ported to PEP 489 multiphase initialization. Summary of API Changes and Additions ==================================== New functions: * PyType_GetModule * PyType_DefiningTypeFromSlotFunc * PyType_GetModuleState * PyErr_PrepareImmutableException New macros: * PyCFunction_GET_CLASS New types: * PyCMethodObject New structures: * PyType_offsets Modified functions: * _PyMethodDef_RawFastCallDict now receives ``PyTypeObject *cls``. * _PyMethodDef_RawFastCallKeywords now receives ``PyTypeObject *cls``. Modified structures: * _heaptypeobject - added ht_module and ht_moduleptr Other changes: * METH_METHOD call flag * defining_class converter in clinic * Py_TPFLAGS_HEAP_IMMUTABLE flag * Py_offsets type spec slot Backwards Compatibility ======================= Two new pointers are added to all heap types. All other changes are adding new functions, structures and a type flag. The new ``PyErr_PrepareImmutableException`` function changes encourages modules to switch from using heap type Exception classes to immutable ones, and a number of modules will be switched in the initial implementation. This change will prevent adding class attributes to such types. For example, the following will raise AttributeError:: sqlite.OperationalError.foo = None Instances and subclasses of such exceptions will not be affected. Implementation ============== An initial implementation is available in a Github repository [#gh-repo]_; a patchset is at [#gh-patch]_. Possible Future Extensions ========================== Easy creation of types with module references --------------------------------------------- It would be possible to add a PEP 489 execution slot type to make creating heap types significantly easier than calling ``PyType_FromModuleAndSpec``. This is left to a future PEP. Optimization ------------ CPython optimizes calls to methods that have restricted signatures, such as not allowing keyword arguments. As proposed here, methods defined with the ``METH_METHOD`` flag do not support these optimizations. Optimized calls still have the option of accessing per-module state the same way slot methods do. References ========== .. [#typeslots-mail] [Import-SIG] On singleton modules, heap types, and subinterpreters (https://mail.python.org/pipermail/import-sig/2015-July/001035.html) .. [#gh-repo] https://github.com/Traceur759/cpython/commits/pep-c .. [#gh-patch] https://github.com/Traceur759/cpython/compare/master...Traceur759:pep-c.patch Copyright ========= This document has been placed in the public domain. From tim.peters at gmail.com Mon Apr 23 11:57:13 2018 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 23 Apr 2018 10:57:13 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <20180423092806.530b5996@fsol> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> Message-ID: [Tim] >> Which this alternative expresses directly: >> >> if (diff := x - x_base) and (g := gcd(diff, n)) > 1: >> return g >> >> That's so Pythonic I could cry ;-) [Antoine] > It looks like C to me. That won't make me cry (I write C++ code daily > these days), but it's certainly not the same language as Python. > > The second part, especially, where you use the result of an > assignment expression as a comparison operand, looks definitely > un-Pythonic. You snipped the part explaining _what's_ "Pythonic" about it: It's _really_ an "and" test: if the diff isn't 0 and gcd(diff, n) > 1, return the gcd. That's how I _thought_ of it from the start. "Expresses directly" is the Pythonic part; the syntax is minor to me. Seeing that the _intent_ is an "and test" is a pattern-matching puzzle in the original spelling (which essentially turned me into a compiler, writing low-level code for the _concepts_ I had in mind from the start): diff = x - x_base if diff: g = gcd(diff, n) if g > 1: return g But note that the part of the PEP I support is just the "binding expression" part: giving a simple name (binding an identifier) to the result of an expression. I don't want the full potential complexity of assignment statements in expressions. There's nothing "un-Pythonic" about merely giving a name to an expression result, apart from that there are few contexts that currently support that in a sanely usable way. From steve at holdenweb.com Mon Apr 23 11:59:35 2018 From: steve at holdenweb.com (Steve Holden) Date: Mon, 23 Apr 2018 16:59:35 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <20180423092806.530b5996@fsol> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> Message-ID: On Mon, Apr 23, 2018 at 8:28 AM, Antoine Pitrou wrote: > On Mon, 23 Apr 2018 00:44:44 -0500 > Tim Peters wrote: > ?[...] > > > if (diff := x - x_base) and (g := gcd(diff, n)) > 1: > > return g > > That's so Pythonic I could cry ;-) > > ?[...] > > The second part, especially, where you use the result of an > assignment expression as a comparison operand, looks definitely > un-Pythonic. > > ?Which, I suppose, underlines that Pythonicity is in the mind of the beholder.? The assignment expression seems like a vary natural way to introduce variables of limited (controlled?) scope, and the class-namespace special case doesn't seem horrible enough to put me, at least, off the idea. There will, of course, be those who abuse assignment expressions, and I'm very much looking forward to seeing what David Beazley makes of them. While Tim's expression might look (superficially) like C, the five-line alternative isn't exactly an inspiring example of Pythonicity, is it? regards Steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From srkunze at mail.de Mon Apr 23 13:13:00 2018 From: srkunze at mail.de (Sven R. Kunze) Date: Mon, 23 Apr 2018 19:13:00 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> Message-ID: On 23.04.2018 17:59, Steve Holden wrote: > > While Tim's expression might look (superficially) like C, the > five-line alternative isn't exactly an inspiring example of > Pythonicity, is it? > What about diff = x - x_base if diff and gcd(diff, n) > 1: return gcd(diff, n) # or if (x - x_base) and gcd(x - x_base, n) > 1: return gcd(x - x_base, n) and have the interpreter handle the optimization, or apply an lru_cache? ;-) Cheers, Sven -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Mon Apr 23 13:24:30 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 24 Apr 2018 03:24:30 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> Message-ID: On Tue, Apr 24, 2018 at 3:13 AM, Sven R. Kunze wrote: > On 23.04.2018 17:59, Steve Holden wrote: > > > While Tim's expression might look (superficially) like C, the five-line > alternative isn't exactly an inspiring example of Pythonicity, is it? > > > What about > > diff = x - x_base > if diff and gcd(diff, n) > 1: > return gcd(diff, n) > > # or > > if (x - x_base) and gcd(x - x_base, n) > 1: > return gcd(x - x_base, n) > > > and have the interpreter handle the optimization, or apply an lru_cache? ;-) And then you want to change something, and you have to make an edit in two places. Or, worse, you make it in only one of those places, they become desynchronized, and nobody can figure out why the program occasionally and bizarrely fails. Removing duplicate function calls isn't just about run-time performance. ChrisA From tim.peters at gmail.com Mon Apr 23 13:31:31 2018 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 23 Apr 2018 12:31:31 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> Message-ID: [Sven R. Kunze ] > What about > > diff = x - x_base > if diff and gcd(diff, n) > 1: > return gcd(diff, n) > > # or > > if (x - x_base) and gcd(x - x_base, n) > 1: > return gcd(x - x_base, n) > > > and have the interpreter handle the optimization, or apply an lru_cache? ;-) Surely you're joking. This is math.gcd(), which is expensive for multi-thousand bit integers, and the interpreter knows nothing about it. Adding a cache of _any_ kind (LRU or otherwise) would make it even slower (in the application, there's no reason to expect that x - x_base will repeat a value before O(sqrt(n)) iterations, which itself can be thousands of bits - a cache hit would be a miracle). From J.Demeyer at UGent.be Mon Apr 23 14:04:27 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Mon, 23 Apr 2018 20:04:27 +0200 Subject: [Python-Dev] PEP 573 -- Module State Access from C Extension Methods In-Reply-To: <2854d9cdad73400aa968a084f40908d7@xmail101.UGent.be> References: <2854d9cdad73400aa968a084f40908d7@xmail101.UGent.be> Message-ID: <5ADE202B.2060501@UGent.be> Hello, I just saw this PEP. There is a bit of overlap between PEP 573 and PEP 575 since these both change the calling convention for built-in methods. In particular, PEP 575 also proposes to add a "defining class" member (for different reasons). In PEP 575, this is added to the PyCFunction struct itself instead of a separate struct PyCMethod. It would be nice to justify whether you really need a new class (PyCMethod_Type) to support METH_METHOD. It looks strange to me that the class of some object depends on an implementation detail like whether METH_METHOD is specified. The current PEP 573 implies that backwards compatibility concerns would arise every time that METH_METHOD is added to an existing method. People have asked questions on PEP 575 about that: it would break code depending on "types.BuiltinFunctionType" for example. You could instead just change PyCFunctionObject to add that field (that's what I did in PEP 575). For practical reasons, it would be nice to implement PEP 573 and PEP 575 together as they affect the same code (assuming that both PEPs are accepted of course). Jeroen. From solipsis at pitrou.net Mon Apr 23 14:44:28 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 23 Apr 2018 20:44:28 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> Message-ID: <20180423204428.0aa91a39@fsol> On Mon, 23 Apr 2018 16:59:35 +0100 Steve Holden wrote: > On Mon, Apr 23, 2018 at 8:28 AM, Antoine Pitrou wrote: > > > On Mon, 23 Apr 2018 00:44:44 -0500 > > Tim Peters wrote: > > ?[...] > > > > > if (diff := x - x_base) and (g := gcd(diff, n)) > 1: > > > return g > > > That's so Pythonic I could cry ;-) > > > > ?[...] > > > > The second part, especially, where you use the result of an > > assignment expression as a comparison operand, looks definitely > > un-Pythonic. > > > ?Which, I suppose, underlines that Pythonicity is in the mind of the > beholder.? Indeed it is. What we can only say is that the proposed idiom goes against current Python syntactical rules :-) > The assignment expression seems like a vary natural way to introduce > variables of limited (controlled?) scope, [...] AFAIU, the scope isn't limited to the "if" block, it's a regular local variable. I might have misread. > While Tim's expression might look (superficially) like C, the five-line > alternative isn't exactly an inspiring example of Pythonicity, is it? I don't know. I've written my share of similar-looking code and I've never really been bothered, at least not enough that I thought we should change the language to accomodate those use cases. To be frank I don't remember the last time I was bothered by an aspect of Python's syntax. I think the language has reached "peak syntax" by now and it should basically be frozen, and any improvements targeted at other aspects (semantics, performance, stdlib, etc.). Regardless, my three questions about this are: - does it make Python more powerful? - does it make Python more readable? - does it make Python easier to learn and teach? My answer would be "no" to all three, but YMMV. Regards Antoine. From encukou at gmail.com Mon Apr 23 15:21:00 2018 From: encukou at gmail.com (Petr Viktorin) Date: Mon, 23 Apr 2018 15:21:00 -0400 Subject: [Python-Dev] PEP 573 -- Module State Access from C Extension Methods In-Reply-To: <5ADE202B.2060501@UGent.be> References: <2854d9cdad73400aa968a084f40908d7@xmail101.UGent.be> <5ADE202B.2060501@UGent.be> Message-ID: On 04/23/18 14:04, Jeroen Demeyer wrote: > Hello, > > I just saw this PEP. There is a bit of overlap between PEP 573 and PEP > 575 since these both change the calling convention for built-in methods. > In particular, PEP 575 also proposes to add a "defining class" member > (for different reasons). In PEP 575, this is added to the PyCFunction > struct itself instead of a separate struct PyCMethod. The reason you just saw the pep is because it wasn't posted to python-dev yet, and it wasn't posted yet because we first want to look for potential conflicts with 573. (There's no one forking on PEP 573 full-time, so unfortunately it's progressing slower than I'd like.) > It would be nice to justify whether you really need a new class > (PyCMethod_Type) to support METH_METHOD. It looks strange to me that the > class of some object depends on an implementation detail like whether > METH_METHOD is specified. > > The current PEP 573 implies that backwards compatibility concerns would > arise every time that METH_METHOD is added to an existing method. People > have asked questions on PEP 575 about that: it would break code > depending on "types.BuiltinFunctionType" for example. You could instead > just change PyCFunctionObject to add that field (that's what I did in > PEP 575). > > For practical reasons, it would be nice to implement PEP 573 and PEP 575 > together as they affect the same code (assuming that both PEPs are > accepted of course). I currently even think PEP 575 can go forward *before* PEP 573. Having __objclass__ on methods of extension classes does sounds like a more elegant solution! And for PEP 573 it would mean one less annoying problem to solve. Reading PEP 575, I miss an explanation of what __objclass__ *is* -- it only says the technical restrictions on it. For PEP 575 I found it pretty important that the reader gets a good mental picture of what it is, but it's a bit difficult to explain succinctly. Maybe something like this would help make it clearer in PEP 573? If set, `__objclass__` is the class that defines the method (which might be a superclass of `type(self)`). Functionally it's equivalent to the `__class__` cell [0] in Python code. [0] https://docs.python.org/3/reference/datamodel.html#creating-the-class-object From ethan at stoneleaf.us Mon Apr 23 16:01:21 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 23 Apr 2018 13:01:21 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> Message-ID: <5ADE3B91.5090804@stoneleaf.us> On 04/22/2018 10:44 PM, Tim Peters wrote: > [Guido] >> In reality there often are other conditions being applied to the match for >> which `if expr as name` is inadequate. The simplest would be something like >> >> if ...: >> >> elif (m := re.match('(.*):(.*)', line)) and m.group(1) == m.group(2): >> >> >> And the match() call may not even be the first thing to check -- e.g. we >> could have >> >> elif line is not None and (m := re.match('(.*):(.*)', line)) and m.group(1) == m.group(2): > > I find myself warming more to binding expressions the more I keep them > in mind while writing new code. And I think it may be helpful to > continue showing real examples where they would help. > > Today's example: I happened to code this a few hours ago: > > diff = x - x_base > if diff: > g = gcd(diff, n) > if g > 1: > return g > > It's not really hard to follow, but two levels of nesting "feels > excessive", as does using the names "diff" and "g" three times each. > It's _really_ an "and" test: if the diff isn't 0 and gcd(diff, n) > > 1, return the gcd. That's how I _thought_ of it from the start. > > Which this alternative expresses directly: > > if (diff := x - x_base) and (g := gcd(diff, n)) > 1: > return g So I really like being able to make the assignment in the expression, but I have a really hard time parsing it with the name first. Attempting to read just the names first: if diff scan for ending right paren found and g scan for ending right paren oops, found opening left paren scan for ending right paren found resume scanning for final right paren found > 1: return g Attempting to read expressions first: if x - x_base and gcd(diff, n) what's diff? scan backwards diff is x - x_base > 1: return g what's g? scan up and backwards g is gcd(diff, n) Attempting to read interleaved: if skip diff x - x_base back to diff as diff and skip g gcd(diff, n) back to g as g > 1: return g On the other hand, if it were using the "as" keyword: if (x - xbase as diff) and (gcd(diff, n) as g) > 1: return g I would parse as: if x - x_base as diff and gcd(diff, n) as g > 1: return g For me at least, the last is much more readable. Thinking about it some more, the problem (or maybe just my problem) is that I see an "if" or "while" and the I look for the thing that is True or False, and using the ":=" syntax the first thing I see is a placeholder for a result that doesn't exist yet, making me constantly scan backwards and forwards to put all the pieces in the correct place. With "as", it just flows forwards. -- ~Ethan~ From srkunze at mail.de Mon Apr 23 16:21:55 2018 From: srkunze at mail.de (Sven R. Kunze) Date: Mon, 23 Apr 2018 22:21:55 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> Message-ID: <6c227e21-b93a-4172-0dda-8caa1cf4d7f0@mail.de> On 23.04.2018 19:24, Chris Angelico wrote: > On Tue, Apr 24, 2018 at 3:13 AM, Sven R. Kunze wrote: >> diff = x - x_base >> if diff and gcd(diff, n) > 1: >> return gcd(diff, n) >> >> # or >> >> if (x - x_base) and gcd(x - x_base, n) > 1: >> return gcd(x - x_base, n) >> >> >> and have the interpreter handle the optimization, or apply an lru_cache? ;-) > And then you want to change something, and you have to make an edit in > two places. Or, worse, you make it in only one of those places, they > become desynchronized, and nobody can figure out why the program > occasionally and bizarrely fails. If you change any of those lines (including ones of my fore-posters) without knowing what you're doing, you'd better don't touch them at all. The SQL folks btw. are pretty okay'ish with this kind of duplication because they can resolve it. Surely, Python isn't SQL but sometimes I wish Python could handle such things as easily without me having to babysit it all the time and using Perl'ish syntax (which := looks like to me). We then have :=, = and ==. Sorry, but Python wouldn't fit my brain then. From srkunze at mail.de Mon Apr 23 16:35:39 2018 From: srkunze at mail.de (Sven R. Kunze) Date: Mon, 23 Apr 2018 22:35:39 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> Message-ID: On 23.04.2018 19:31, Tim Peters wrote: > Surely you're joking. This is math.gcd(), which is expensive for > multi-thousand bit integers, and the interpreter knows nothing about > it. Adding a cache of _any_ kind (LRU or otherwise) would make it > even slower. Alright, if that problem is just about performance, then there must be a better way to resolve it rather than inventing a new syntax. Btw. storing the result in a local var is also a cache IMHO. And if gcd is immutable, I think Python can do a great job here of optimizing. Anyway, your example is the best one I've seen so far. From rosuav at gmail.com Mon Apr 23 16:37:15 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 24 Apr 2018 06:37:15 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <6c227e21-b93a-4172-0dda-8caa1cf4d7f0@mail.de> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <6c227e21-b93a-4172-0dda-8caa1cf4d7f0@mail.de> Message-ID: On Tue, Apr 24, 2018 at 6:21 AM, Sven R. Kunze wrote: > On 23.04.2018 19:24, Chris Angelico wrote: >> >> On Tue, Apr 24, 2018 at 3:13 AM, Sven R. Kunze wrote: >>> >>> diff = x - x_base >>> if diff and gcd(diff, n) > 1: >>> return gcd(diff, n) >>> >>> # or >>> >>> if (x - x_base) and gcd(x - x_base, n) > 1: >>> return gcd(x - x_base, n) >>> >>> >>> and have the interpreter handle the optimization, or apply an lru_cache? >>> ;-) >> >> And then you want to change something, and you have to make an edit in >> two places. Or, worse, you make it in only one of those places, they >> become desynchronized, and nobody can figure out why the program >> occasionally and bizarrely fails. > > > If you change any of those lines (including ones of my fore-posters) without > knowing what you're doing, you'd better don't touch them at all. > > The SQL folks btw. are pretty okay'ish with this kind of duplication because > they can resolve it. Surely, Python isn't SQL but sometimes I wish Python > could handle such things as easily without me having to babysit it all the > time and using Perl'ish syntax (which := looks like to me). We then have :=, > = and ==. Sorry, but Python wouldn't fit my brain then. > Ah, are you one of those programmers who writes code once and it's instantly perfect? I apologize, I didn't realize I was in the presence of a unicorn. Most programmers will end up making edits to all non-trivial code. And if your code is laid out in such a way that edits are easier and safer, you're less likely to introduce bugs as you edit. Duplication works against that by forcing you to make changes in two places. I've seen code that relies on duplication and compiler optimizations. Sure, it'll run just as fast as the equivalent with actual variable names; but that's beside the point. It takes extra effort to maintain such code, and that is what matters. ChrisA From barry at python.org Mon Apr 23 17:19:36 2018 From: barry at python.org (Barry Warsaw) Date: Mon, 23 Apr 2018 14:19:36 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <5ADE3B91.5090804@stoneleaf.us> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE3B91.5090804@stoneleaf.us> Message-ID: <855A96BF-144C-4A80-9050-9323557B2D57@python.org> On Apr 23, 2018, at 13:01, Ethan Furman wrote: > > On 04/22/2018 10:44 PM, Tim Peters wrote: >> >> >> I find myself warming more to binding expressions the more I keep them >> in mind while writing new code. And I really like the term ?binding expressions? because that?s how I think about this feature. I also think it will be easier to explain because ?all it does? is bind a value to a name, and to me that?s the most powerful and valuable thing behind this feature. > So I really like being able to make the assignment in the expression, but I have a really hard time parsing it with the name first. Me too. Plus we *already* have precedence for spelling name bindings in similar constructs, such as import statements, with statements, and exceptions. It seems like a natural and Pythonic approach to extend that same spelling to binding expressions rather than introducing new, weird, symbols. I also think it effectively solves the switch-statement problem: if (get_response() as answer) == 'yes': do_it() elif answer == 'no': skip_it() elif answer == 'maybe' okay_then() That?s Pythonic enough for jazz! -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From srkunze at mail.de Mon Apr 23 17:25:42 2018 From: srkunze at mail.de (Sven R. Kunze) Date: Mon, 23 Apr 2018 23:25:42 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <6c227e21-b93a-4172-0dda-8caa1cf4d7f0@mail.de> Message-ID: <2d70c91f-ab82-7528-badd-d96291b97968@mail.de> On 23.04.2018 22:37, Chris Angelico wrote: > Ah, are you one of those programmers who writes code once and it's > instantly perfect? I apologize, I didn't realize I was in the presence > of a unicorn. Wow, constructive. Nothing is perfect, but if you don't consider your surroundings when doing changes, well, what could possibly go wrong... > Duplication works against that by forcing you to make changes in two places. ... in the very same line, a line below, few characters after/before it. > I've seen code that relies on duplication and compiler optimizations. > Sure, it'll run just as fast as the equivalent with actual variable > names; but that's beside the point. It takes extra effort to maintain > such code, and that is what matters. That's exactly my point. Readability is what counts, especially for maintaining. "gcd(diff, n)" is a great name, much better than "g", if you ask me. We aren't talking about 1000 lines of code here. The new syntax will enable one-liner optimizations. And I think Tim's example is as good as we can get realistically. Because if the expressions become longer or more complex and/or the numbers of expressions increase, I doubt that a majority want to have that in a single line even though the syntax would allow this. And if so the editor might include some line wraps, so then we are where we were before. Somewhere, you need to get your hands dirty. From rosuav at gmail.com Mon Apr 23 17:29:46 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 24 Apr 2018 07:29:46 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <2d70c91f-ab82-7528-badd-d96291b97968@mail.de> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <6c227e21-b93a-4172-0dda-8caa1cf4d7f0@mail.de> <2d70c91f-ab82-7528-badd-d96291b97968@mail.de> Message-ID: On Tue, Apr 24, 2018 at 7:25 AM, Sven R. Kunze wrote: > On 23.04.2018 22:37, Chris Angelico wrote: >> >> Ah, are you one of those programmers who writes code once and it's >> instantly perfect? I apologize, I didn't realize I was in the presence >> of a unicorn. > > > Wow, constructive. Nothing is perfect, but if you don't consider your > surroundings when doing changes, well, what could possibly go wrong... I've seen way too many programmers - myself included - think that synchronized edits are viable. If you are the one and only perfect programmer, then sure! But if you're like the rest of us, it's better to build your code in such a way that it doesn't bite you. Yes, it was a snarky way to make the point, but I have a strong point to make. ChrisA From srkunze at mail.de Mon Apr 23 17:34:22 2018 From: srkunze at mail.de (Sven R. Kunze) Date: Mon, 23 Apr 2018 23:34:22 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <855A96BF-144C-4A80-9050-9323557B2D57@python.org> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE3B91.5090804@stoneleaf.us> <855A96BF-144C-4A80-9050-9323557B2D57@python.org> Message-ID: On 23.04.2018 23:19, Barry Warsaw wrote: > I also think it effectively solves the switch-statement problem: > > if (get_response() as answer) == 'yes': > do_it() > elif answer == 'no': > skip_it() > elif answer == 'maybe' > okay_then() > > That?s Pythonic enough for jazz! Is it just me or since when has the following Python code fallen out of favor? answer = get_response() if answer == 'yes': do_it() elif answer == 'no': skip_it() elif answer == 'maybe' okay_then() I really don't see anything I would want to optimize here. Not even a single bit. But as said that might just be me. What I like about this code is: 1) symmetry: all ifs are equally structured 2) classic IPO model: first get input, then process (, then output) Sven From tim.peters at gmail.com Mon Apr 23 17:41:49 2018 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 23 Apr 2018 16:41:49 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> Message-ID: [Tim] >> Surely you're joking. This is math.gcd(), which is expensive for >> multi-thousand bit integers, and the interpreter knows nothing about >> it. Adding a cache of _any_ kind (LRU or otherwise) would make it >> even slower. [Sven R. Kunze ] > Alright, if that problem is just about performance, It's not, but others had already pointed out that it's generally considered Poor Practice (and for good reasons) to textually repeat expressions, so I didn't echo that. Even in purely functional languages, where textually equal snippets are guaranteed to evaluate to the same result every time, "give these expressions these brief names" constructs are heavily used (see, .e.g, "let" and "where" in Haskell). >:then there must be a better way to resolve it rather than > inventing a new syntax. Why? "Give the result of an expression a name" is already heavily used in Python - it's just that the _contexts_ in which it can be done are very limited now. > Btw. storing the result in a local var is also a cache IMHO. And > if gcd is immutable, I think Python can do a great job here of > optimizing. After decades, CPython still does nothing of the sort, short of having eventually made, e.g., "None" and "True" and "False" reserved words so at least it can optimize uses of those. It knows nothing at all about which library functions are pure - and there's no code in the implementation currently capable of exploiting such information even if it were known. That remains a fantasy in CPython. > Anyway, your example is the best one I've seen so far. Guido gave better ones, where binding expressions would allow to collapse arbitrarily deep levels of nesting to just one (if ... elif ... elif ... elif ...). My example only eliminated a single level of artificial indentation. But my example did have the advantage of being taken verbatim from actual, working code ;-) From tim.peters at gmail.com Mon Apr 23 18:04:55 2018 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 23 Apr 2018 17:04:55 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <5ADE3B91.5090804@stoneleaf.us> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE3B91.5090804@stoneleaf.us> Message-ID: [Ethan Furman ] > So I really like being able to make the assignment in the expression, but I > have a really hard time parsing it with the name first. > > ... > > On the other hand, if it were using the "as" keyword: > > if (x - xbase as diff) and (gcd(diff, n) as g) > 1: > return g > > I would parse as: > > if > x - x_base > as diff > and > gcd(diff, n) > as g > > 1: > return g > > For me at least, the last is much more readable. Thinking about it some > more, the problem (or maybe just my problem) is that I see an "if" or > "while" and the I look for the thing that is True or False, and using the > ":=" syntax the first thing I see is a placeholder for a result that doesn't > exist yet, making me constantly scan backwards and forwards to put all the > pieces in the correct place. > > With "as", it just flows forwards. I can read it fine either way, and don't much care. A possible advantage of an "as" operator is that its precedence could be set to bind just a tad stronger than comparisons (which include "is" and "is not" in Python), and then, e.g., if f() as result is not None: do something with result could work as intended. So long as people can't get "assignment _statements_" out of their heads, if result := f() is not None: groups instead as if result := (f() is not None): which would almost never be _intended_. Maybe spelling it "as" instead could break that. However, against "as" is that its current use in "with" statements does something quite different: with f() as name: does not bind the result of `f()` to `name`, but the result of `f().__enter__()`. Whether that "should be" fatal, I don't know, but it's at least annoying ;-) From ethan at stoneleaf.us Mon Apr 23 18:15:37 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 23 Apr 2018 15:15:37 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE3B91.5090804@stoneleaf.us> Message-ID: <5ADE5B09.80207@stoneleaf.us> On 04/23/2018 03:04 PM, Tim Peters wrote: > [Ethan Furman ] >> So I really like being able to make the assignment in the expression, but I >> have a really hard time parsing it with the name first. >> >> ... >> >> On the other hand, if it were using the "as" keyword: >> >> if (x - xbase as diff) and (gcd(diff, n) as g) > 1: >> return g >> >> I would parse as: >> >> if >> x - x_base >> as diff >> and >> gcd(diff, n) >> as g >> > 1: >> return g >> >> For me at least, the last is much more readable. Thinking about it some >> more, the problem (or maybe just my problem) is that I see an "if" or >> "while" and the I look for the thing that is True or False, and using the >> ":=" syntax the first thing I see is a placeholder for a result that doesn't >> exist yet, making me constantly scan backwards and forwards to put all the >> pieces in the correct place. >> >> With "as", it just flows forwards. > > I can read it fine either way, and don't much care. A possible > advantage of an "as" operator is that its precedence could be set to > bind just a tad stronger than comparisons (which include "is" and "is > not" in Python), and then, e.g., > > if f() as result is not None: > do something with result > > could work as intended. So long as people can't get "assignment > _statements_" out of their heads, > > if result := f() is not None: > > groups instead as > > if result := (f() is not None): > > which would almost never be _intended_. Maybe spelling it "as" > instead could break that. > > However, against "as" is that its current use in "with" statements > does something quite different: > > with f() as name: > > does not bind the result of `f()` to `name`, but the result of > `f().__enter__()`. Whether that "should be" fatal, I don't know, but > it's at least annoying ;-) "as" does something slightly different in each of its current incantations, but the one thing that they all have in common is taking some thing on the left and giving it the name on the right: - imports -> thing on left gets name on right - exceptions -> exception that matches class on left gets name on right - with -> result of left.__enter__() gets name on right I see very little harm in adding - expression-binding -> eval(thing on left) gets name on right -- ~Ethan~ From guido at python.org Mon Apr 23 18:18:39 2018 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Apr 2018 15:18:39 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <5ADE5B09.80207@stoneleaf.us> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE3B91.5090804@stoneleaf.us> <5ADE5B09.80207@stoneleaf.us> Message-ID: Whereas I find it a deal-breaker for 'as'. On Mon, Apr 23, 2018 at 3:15 PM, Ethan Furman wrote: > On 04/23/2018 03:04 PM, Tim Peters wrote: > >> [Ethan Furman ] >> >>> So I really like being able to make the assignment in the expression, >>> but I >>> have a really hard time parsing it with the name first. >>> >>> ... >>> >>> On the other hand, if it were using the "as" keyword: >>> >>> if (x - xbase as diff) and (gcd(diff, n) as g) > 1: >>> return g >>> >>> I would parse as: >>> >>> if >>> x - x_base >>> as diff >>> and >>> gcd(diff, n) >>> as g >>> > 1: >>> return g >>> >>> For me at least, the last is much more readable. Thinking about it some >>> more, the problem (or maybe just my problem) is that I see an "if" or >>> "while" and the I look for the thing that is True or False, and using the >>> ":=" syntax the first thing I see is a placeholder for a result that >>> doesn't >>> exist yet, making me constantly scan backwards and forwards to put all >>> the >>> pieces in the correct place. >>> >>> With "as", it just flows forwards. >>> >> >> I can read it fine either way, and don't much care. A possible >> advantage of an "as" operator is that its precedence could be set to >> bind just a tad stronger than comparisons (which include "is" and "is >> not" in Python), and then, e.g., >> >> if f() as result is not None: >> do something with result >> >> could work as intended. So long as people can't get "assignment >> _statements_" out of their heads, >> >> if result := f() is not None: >> >> groups instead as >> >> if result := (f() is not None): >> >> which would almost never be _intended_. Maybe spelling it "as" >> instead could break that. >> >> However, against "as" is that its current use in "with" statements >> does something quite different: >> >> with f() as name: >> >> does not bind the result of `f()` to `name`, but the result of >> `f().__enter__()`. Whether that "should be" fatal, I don't know, but >> it's at least annoying ;-) >> > > "as" does something slightly different in each of its current > incantations, but the one thing that they all have in common is taking some > thing on the left and giving it the name on the right: > > - imports -> thing on left gets name on right > - exceptions -> exception that matches class on left gets name on right > - with -> result of left.__enter__() gets name on right > > I see very little harm in adding > > - expression-binding -> eval(thing on left) gets name on right > > -- > ~Ethan~ > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido% > 40python.org > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From nad at python.org Mon Apr 23 18:19:39 2018 From: nad at python.org (Ned Deily) Date: Mon, 23 Apr 2018 18:19:39 -0400 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE3B91.5090804@stoneleaf.us> Message-ID: <12E5DDE7-30E7-4870-83BC-AD772661962B@python.org> On Apr 23, 2018, at 18:04, Tim Peters wrote: > However, against "as" is that its current use in "with" statements > does something quite different: > > with f() as name: > > does not bind the result of `f()` to `name`, but the result of > `f().__enter__()`. Whether that "should be" fatal, I don't know, but > it's at least annoying ;-) Prior art: COBOL uses "GIVING", as in: ADD x, y GIVING z No need to re-invent the wheel ;) -- Ned Deily nad at python.org -- [] From paul at ganssle.io Mon Apr 23 18:36:18 2018 From: paul at ganssle.io (Paul G) Date: Mon, 23 Apr 2018 18:36:18 -0400 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE3B91.5090804@stoneleaf.us> Message-ID: <8cb0003c-613b-906b-82f9-a65c4df5ba49@ganssle.io> On 04/23/2018 06:04 PM, Tim Peters wrote: > However, against "as" is that its current use in "with" statements > does something quite different: > > with f() as name: > > does not bind the result of `f()` to `name`, but the result of > `f().__enter__()`. Whether that "should be" fatal, I don't know, but > it's at least annoying ;-) This could be read a different way, though, since `with f()` calls `f().__enter__()`, you could read it as `(with f()) as name:`, in which case it does the same thing as an `as`-based binding expression would. Viewing it that way *also* helps alleviate the cognitive problem that `with f() as name` and `with (f() as name)` do two different things - the parentheses there are changing the precedence in the same way that `2 + 4 * 3` and `(2 + 4) * 3` do two different things. This sorta also works for `except`, if you read it as `(except SomeException) as e:`, but I think this fiction falls apart when you look at `import`, since `import foo` *already* binds "foo" to a name, and `import foo as bar` not only binds `foo` to `bar`, but also *doesn't* bind `foo` to `foo`. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From guido at python.org Mon Apr 23 18:36:10 2018 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Apr 2018 15:36:10 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <12E5DDE7-30E7-4870-83BC-AD772661962B@python.org> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE3B91.5090804@stoneleaf.us> <12E5DDE7-30E7-4870-83BC-AD772661962B@python.org> Message-ID: Using 'as' was debated extensively on python-ideas. I don't like it for several reasons: - the semantics are subtly different from all other uses of 'as' in Python; I'd like to reserve 'as' for "not a plain assignment" - a short word is easier to miss when skimming code than punctuation - most other languages (Java*, C*) borrow from assignment (name = expr) On Mon, Apr 23, 2018 at 3:19 PM, Ned Deily wrote: > On Apr 23, 2018, at 18:04, Tim Peters wrote: > > However, against "as" is that its current use in "with" statements > > does something quite different: > > > > with f() as name: > > > > does not bind the result of `f()` to `name`, but the result of > > `f().__enter__()`. Whether that "should be" fatal, I don't know, but > > it's at least annoying ;-) > > Prior art: COBOL uses "GIVING", as in: > > ADD x, y GIVING z > > No need to re-invent the wheel ;) > > -- > Ned Deily > nad at python.org -- [] > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From mcepl at cepl.eu Mon Apr 23 17:59:02 2018 From: mcepl at cepl.eu (=?UTF-8?Q?Mat=C4=9Bj?= Cepl) Date: Mon, 23 Apr 2018 23:59:02 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE3B91.5090804@stoneleaf.us> <855A96BF-144C-4A80-9050-9323557B2D57@python.org> Message-ID: On 2018-04-23, 21:34 GMT, Sven R. Kunze wrote: > Is it just me or since when has the following Python code > fallen out of favor? It isn?t just you. Mat?j -- https://matej.ceplovi.cz/blog/, Jabber: mcepl at ceplovi.cz GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 ? one of the main causes of the fall of the Roman Empire was that, lacking zero, they had no way to indicate successful termination of their C programs. -- Robert Firth From v+python at g.nevcal.com Mon Apr 23 18:45:59 2018 From: v+python at g.nevcal.com (Glenn Linderman) Date: Mon, 23 Apr 2018 15:45:59 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <5ADE3B91.5090804@stoneleaf.us> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE3B91.5090804@stoneleaf.us> Message-ID: On 4/23/2018 1:01 PM, Ethan Furman wrote: > On 04/22/2018 10:44 PM, Tim Peters wrote: >> [Guido] >>> In reality there often are other conditions being applied to the >>> match for >>> which `if expr as name` is inadequate. The simplest would be >>> something like >>> >>> ?? if ...: >>> ?????? >>> ?? elif (m := re.match('(.*):(.*)', line)) and m.group(1) == >>> m.group(2): >>> ???? >>> >>> And the match() call may not even be the first thing to check -- >>> e.g. we >>> could have >>> >>> ???? elif line is not None and (m := re.match('(.*):(.*)', line)) >>> and m.group(1) == m.group(2): >> >> I find myself warming more to binding expressions the more I keep them >> in mind while writing new code.? And I think it may be helpful to >> continue showing real examples where they would help. >> >> Today's example:? I happened to code this a few hours ago: >> >> diff = x - x_base >> if diff: >> ???? g = gcd(diff, n) >> ???? if g > 1: >> ???????? return g >> >> It's not really hard to follow, but two levels of nesting "feels >> excessive", as does using the names "diff" and "g" three times each. >> It's _really_ an "and" test:? if the diff isn't 0 and gcd(diff, n) > >> 1, return the gcd.? That's how I _thought_ of it from the start. >> >> Which this alternative expresses directly: >> >> if (diff := x - x_base) and (g := gcd(diff, n)) > 1: >> ???? return g > > So I really like being able to make the assignment in the expression, > but I have a really hard time parsing it with the name first. > > Attempting to read just the names first: > > ? if > ??? diff > ????? scan for ending right paren > ????? found > ? and > ???? g > ????? scan for ending right paren > ??????? oops, found opening left paren > ??????? scan for ending right paren > ??????? found > ????? resume scanning for final right paren > ????? found > ???? > 1: > ???????? return g > > > Attempting to read expressions first: > > ? if > ??? x - x_base > ? and > ??? gcd(diff, n) > ????? what's diff? > ??????? scan backwards > ????????? diff is x - x_base > ??? > 1: > ??????? return g > ????????? what's g? > ??????????? scan up and backwards > ??????????? g is gcd(diff, n) > > > Attempting to read interleaved: > > ? if > ??? skip diff > ????? x - x_base > ????? back to diff > ??? as diff > ? and > ??? skip g > ????? gcd(diff, n) > ????? back to g > ??? as g > ? > 1: > ????? return g > > > On the other hand, if it were using the "as" keyword: > > if (x - xbase as diff) and (gcd(diff, n) as g) > 1: > ??? return g > > I would parse as: > > ? if > ??? x - x_base > ??? as diff > ? and > ??? gcd(diff, n) > ??? as g > ? > 1: > ????? return g > > For me at least, the last is much more readable.? Thinking about it > some more, the problem (or maybe just my problem) is that I see an > "if" or "while" and the I look for the thing that is True or False, > and using the ":=" syntax the first thing I see is a placeholder for a > result that doesn't exist yet, making me constantly scan backwards and > forwards to put all the pieces in the correct place. > > With "as", it just flows forwards. You need to borrow the time machine, and get with those early mathematicians that first said: Let x be the sum of y and z and convince them that what they should have said was: Let the sum of y and z be called x. :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Mon Apr 23 19:54:13 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 24 Apr 2018 11:54:13 +1200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> Message-ID: <5ADE7225.5080307@canterbury.ac.nz> Tim Peters wrote: > if (diff := x - x_base) and (g := gcd(diff, n)) > 1: > return g My problem with this is -- how do you read such code out loud? From my Pascal days I'm used to reading ":=" as "becomes". So this says: "If diff becomes x - base and g becomes gcd(diff, n) is greater than or equal to 1 then return g." But "diff becomes x - base" is not what we're testing! That makes it sound like the result of x - base may or may not get assigned to diff, which is not what's happening at all. The "as" variant makes more sense when you read it as an English sentence: if ((x - x_base) as diff) and ... "If x - x_base (and by the way, I'm going to call that diff so I can refer to it later) is not zero ..." -- Greg From greg.ewing at canterbury.ac.nz Mon Apr 23 19:56:54 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 24 Apr 2018 11:56:54 +1200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> Message-ID: <5ADE72C6.6080106@canterbury.ac.nz> Sven R. Kunze wrote: > if (x - x_base) and gcd(x - x_base, n) > 1: > return gcd(x - x_base, n) > > and have the interpreter handle the optimization, or apply an lru_cache? ;-) Problem is, there's no way to automatically tell whether the expressions involved have side effects that are being relied on. -- Greg From tim.peters at gmail.com Mon Apr 23 20:23:49 2018 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 23 Apr 2018 19:23:49 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <5ADE7225.5080307@canterbury.ac.nz> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE7225.5080307@canterbury.ac.nz> Message-ID: [Tim] >> if (diff := x - x_base) and (g := gcd(diff, n)) > 1: >> return g [Greg Ewing ] > My problem with this is -- how do you read such code out loud? In the message in which I first gave that example: if the diff isn't 0 and gcd(diff, n) > 1, return the gcd. That's how I _thought_ of it from the start. In my mind, `x - x_base` doesn't even exist except as a low-level definition of what "diff" means. It's different for the other test: _there_ `g` doesn't exist except as a shorthand for "the gcd". In one case it's the name that's important to me, and in the other case the expression. The entire function from which this came is doing all arithmetic modulo `n`, so `n` isn't in my mind either - it's a ubiquitous part of the background in this specific function. But you did ask how_I_ would read that code ;-) Anyone else is free to read it however they like. I naturally read it in the way that makes most sense to me in its context. > From my Pascal days I'm used to reading ":=" as "becomes". So > this says: > > "If diff becomes x - base and g becomes gcd(diff, n) is > greater than or equal to 1 then return g." > > But "diff becomes x - base" is not what we're testing! I don't really follow that. In Python, if f() and g > 1: first tests whether `f()` "is truthy", regardless of whether it does or doesn't appear in a binding expression. Because this code is working with integers, there's an _implied_ "!= 0" comparison. > That makes it sound like the result of x - base may or may not > get assigned to diff, which is not what's happening at all. Then I suggest the problem you're having doesn't stem from the binding expression, but from that you're omitting to fill in the != 0 part: if you're not thrown by "greater than 1", I can't see how you can be thrown by "not zero". > The "as" variant makes more sense when you read it as an > English sentence: > > if ((x - x_base) as diff) and ... > > "If x - x_base (and by the way, I'm going to call that > diff so I can refer to it later) is not zero ..." So read the original as "if diff (which is x - x_base) is not zero ...". Regardless, Guido has already said "as" is DOA (Dead On Arrival) (illustrating that it's also common enough in English to give a short name before its long-winded meaning ;-) ). From steve at pearwood.info Mon Apr 23 20:31:36 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 24 Apr 2018 10:31:36 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE3B91.5090804@stoneleaf.us> <12E5DDE7-30E7-4870-83BC-AD772661962B@python.org> Message-ID: <20180424003136.GD11616@ando.pearwood.info> On Mon, Apr 23, 2018 at 03:36:10PM -0700, Guido van Rossum wrote: > Using 'as' was debated extensively on python-ideas. I don't like it for > several reasons: [...] For what it is worth, I was one of the original proponents of the "as" syntax, but at this point I am no longer going to argue for it. I'm satisfied that using "as" has disadvantages, and that := is an acceptable alternative. While I haven't changed my mind that putting the expression first and the target second is better, I'm satisfied that it is not so much better that it is worth extending this enormous discussion even further. In other words, as far as I am concerned, I am happy to take the syntax as decided: "expr as name" is rejected and "name := expr" is acceptable. I'm also satisfied that *at least for now* we should stick to only allowing simple names. We can always loosen the restriction later and allow arbitrarily complex targets at a later date, but we can't easily change our mind and prohibit them. -- Steve From chris.jerdonek at gmail.com Mon Apr 23 21:42:17 2018 From: chris.jerdonek at gmail.com (Chris Jerdonek) Date: Mon, 23 Apr 2018 18:42:17 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <5ADE7225.5080307@canterbury.ac.nz> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE7225.5080307@canterbury.ac.nz> Message-ID: On Mon, Apr 23, 2018 at 4:54 PM, Greg Ewing wrote: > Tim Peters wrote: > >> if (diff := x - x_base) and (g := gcd(diff, n)) > 1: >> return g > > > My problem with this is -- how do you read such code out loud? It could be-- "if diff, which we let equal x - x_base, and g, which ..." or "if diff, which we set equal to x - x_base, and g, which ...." or "if diff, which we define to be x - x_base, and g, which ...." or "if diff, which we define as x - x_base, and g, which ....." etc. --Chris > > From my Pascal days I'm used to reading ":=" as "becomes". So > this says: > > "If diff becomes x - base and g becomes gcd(diff, n) is > greater than or equal to 1 then return g." > > But "diff becomes x - base" is not what we're testing! That > makes it sound like the result of x - base may or may not > get assigned to diff, which is not what's happening at all. > > The "as" variant makes more sense when you read it as an > English sentence: > > if ((x - x_base) as diff) and ... > > "If x - x_base (and by the way, I'm going to call that > diff so I can refer to it later) is not zero ..." > > -- > Greg > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/chris.jerdonek%40gmail.com From turnbull.stephen.fw at u.tsukuba.ac.jp Tue Apr 24 01:18:35 2018 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Tue, 24 Apr 2018 14:18:35 +0900 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE7225.5080307@canterbury.ac.nz> Message-ID: <23262.48683.257457.448898@turnbull.sk.tsukuba.ac.jp> Tim Peters writes: > Regardless, Guido has already said "as" is DOA (Dead On Arrival) > (illustrating that it's also common enough in English to give a short > name before its long-winded meaning ;-) ). The parens above are a gloss on a well-known acronym for those who have not encountered it before. Neologisms are usually written in the other order: "dead on arrival (DOA, for short)." ;-) From tim.peters at gmail.com Tue Apr 24 02:06:30 2018 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 24 Apr 2018 01:06:30 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <20180423204428.0aa91a39@fsol> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <20180423204428.0aa91a39@fsol> Message-ID: [Steve Holden ] >> ... >> The assignment expression seems like a vary natural way to introduce >> variables of limited (controlled?) scope, [...] [Antoine Pitrou ] > AFAIU, the scope isn't limited to the "if" block, it's a regular local > variable. I might have misread. You're right about the current version of the PEP. No new scoping rules are introduced. The PEP does suggest some changes to corner case scoping semantics, though. > ... > Regardless, my three questions about this are: > - does it make Python more powerful? Goodness no. > - does it make Python more readable? There are cases where it would, and cases where it wouldn't. People shouldn't use it in the latter cases ;-) I very recently wrote this block of code: outside = p2units[p1][tgt_kind] - units[2] if outside: if not all(self.crossout(q, n, undo) for q in outside): return False The opening pair is a very common minor annoyance; it's marginally more readable like so: if outside := p2units[p1][tgt_kind] - units[2]: Saving an essentially useless line with a duplicated name is worth something to me, because it comes up so very often. But that's indeed "minor". In my diff/gcd example, it reduced 5 lines of code to 2; saved a level of annoying (semantically misleading) indentation; and cut the number of instances of both "diff" and "g" from 3 each to 2 each (ideal: one each to bind the name, and then one each to use the name later). That's relatively substantial by any measure. In Guido's if/elif/elif/elif/elif ... complex text processing example template, it can save an unbounded number of semantically needless indentation levels. So the readability benefits can range from highly negative to highly positive. > - does it make Python easier to learn and teach? By whom? Almost no addition has ever made a language easier to learn for raw beginners: every addition is something they eventually need to learn. We could make Python easier to learn for beginners by throwing out virtually everything added since version 0.9.6 ;-) But people coming _from_ most other very widely used languages (C, C++, Java, Javascript, Perl, ...) are already familiar with assignment expressions. The limited (to a plain identifier target) "binding expression" PEP simplification I favor would be nothing new to them at all (whereas the full complexity of Python's assignment statements is indeed beyond what they're used to, but needs to be taught & learned regardless of this PEP's fate). At least when restricted to binding expressions, the syntax is simple and the semantics are the very simplest case of what people (convert or raw beginner) need to learn for Python's assignment statements regardless. > My answer would be "no" to all three, but YMMV. And it did ;-) From solipsis at pitrou.net Tue Apr 24 02:27:51 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 24 Apr 2018 08:27:51 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <20180423204428.0aa91a39@fsol> Message-ID: <20180424082751.39708d4f@fsol> On Tue, 24 Apr 2018 01:06:30 -0500 Tim Peters wrote: > > > - does it make Python easier to learn and teach? > > By whom? Almost no addition has ever made a language easier to learn > for raw beginners: every addition is something they eventually need > to learn. We could make Python easier to learn for beginners by > throwing out virtually everything added since version 0.9.6 ;-) Constructs like "with ..." or "try / except / finally" make the language easier to learn compared to the dances they are meant to replace. "await" is a more readable and less confusing improvement over "yield from". Format strings dispense from the older, more convoluted formulations. Iteration is much simpler than the longer forms we would have to write if generalized iterators didn't exist. Regards Antoine. From rosuav at gmail.com Tue Apr 24 02:38:39 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 24 Apr 2018 16:38:39 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <20180424082751.39708d4f@fsol> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <20180423204428.0aa91a39@fsol> <20180424082751.39708d4f@fsol> Message-ID: On Tue, Apr 24, 2018 at 4:27 PM, Antoine Pitrou wrote: > On Tue, 24 Apr 2018 01:06:30 -0500 > Tim Peters wrote: >> >> > - does it make Python easier to learn and teach? >> >> By whom? Almost no addition has ever made a language easier to learn >> for raw beginners: every addition is something they eventually need >> to learn. We could make Python easier to learn for beginners by >> throwing out virtually everything added since version 0.9.6 ;-) > > Constructs like "with ..." or "try / except / finally" make the > language easier to learn compared to the dances they are meant to > replace. "await" is a more readable and less confusing improvement > over "yield from". Format strings dispense from the older, more > convoluted formulations. Iteration is much simpler than the longer > forms we would have to write if generalized iterators didn't exist. And assignment expressions are far simpler than breaking things out over multiple lines (particularly when you consider the infinite while loop alternative). But that doesn't change the fact that, as features, they do make the language harder to understand. The difference is that they make *your program* easier to understand. If you really want the *language* to be as easy as possible to fit into your head, you want a Turing tarpit: https://esolangs.org/wiki/Ook! Only three lexical tokens, and only eight operations. But any program written in Ook is going to be almost completely unreadable. That's the tradeoff. ChrisA From tim.peters at gmail.com Tue Apr 24 02:55:13 2018 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 24 Apr 2018 01:55:13 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <20180424082751.39708d4f@fsol> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <20180423204428.0aa91a39@fsol> <20180424082751.39708d4f@fsol> Message-ID: [Antoine] >>> - does it make Python easier to learn and teach? [Tim] >> By whom? Almost no addition has ever made a language easier to learn >> for raw beginners: every addition is something they eventually need >> to learn. We could make Python easier to learn for beginners by >> throwing out virtually everything added since version 0.9.6 ;-) [Antoine] > Constructs like "with ..." or "try / except / finally" make the > language easier to learn compared to the dances they are meant to > replace. They nevertheless need to be taught & learned (and try/except/finally was essentially always in the language), You snipped the parts pointing out that binding expressions are already familiar to people coming from most other languages, and even for raw beginners the semantics are the tiniest part of what they need to learn anyway about Python's assignment expressions. So that was my point: they don't make Python any harder to learn or teach. To the contrary, for people coming from other languages, it's one less thing they're used to they wouldn't have to _un_learn. > "await" is a more readable and less confusing improvement > over "yield from". Heh. Not to me. I have literally have no idea what to with "await" (I use generators heavily, but have had no use yet for coroutines), but use yield from an_iterable routinely. That use is perfectly clear, to the point that it _can't_ be improved on: it already does exactly what I want from it, with no effort at all. It's simply impossible that, whatever "await" does, it could be more readable or less confusing than what I use "yield from" for. > Format strings dispense from the older, more convoluted formulations. But they didn't _replace_ them. They're Yet Another Way to Format Strings everyone has to learn. That made teaching/learning harder, not easier, but you can make a case they make Python easier to _use_ for people who eschew the older forms. > Iteration is much simpler than the longer forms we would have to write > if generalized iterators didn't exist. I'll buy that one. Now go through the HISTORY file and count all the changes you didn't name ;-) From solipsis at pitrou.net Tue Apr 24 02:57:13 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 24 Apr 2018 08:57:13 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <20180423204428.0aa91a39@fsol> <20180424082751.39708d4f@fsol> Message-ID: <20180424085713.1cfa02be@fsol> On Tue, 24 Apr 2018 16:38:39 +1000 Chris Angelico wrote: > On Tue, Apr 24, 2018 at 4:27 PM, Antoine Pitrou wrote: > > On Tue, 24 Apr 2018 01:06:30 -0500 > > Tim Peters wrote: > >> > >> > - does it make Python easier to learn and teach? > >> > >> By whom? Almost no addition has ever made a language easier to learn > >> for raw beginners: every addition is something they eventually need > >> to learn. We could make Python easier to learn for beginners by > >> throwing out virtually everything added since version 0.9.6 ;-) > > > > Constructs like "with ..." or "try / except / finally" make the > > language easier to learn compared to the dances they are meant to > > replace. "await" is a more readable and less confusing improvement > > over "yield from". Format strings dispense from the older, more > > convoluted formulations. Iteration is much simpler than the longer > > forms we would have to write if generalized iterators didn't exist. > > And assignment expressions are far simpler than breaking things out > over multiple lines (particularly when you consider the infinite while > loop alternative). But that doesn't change the fact that, as features, > they do make the language harder to understand. I did specifically say "easier to learn and teach" instead of using generic phrases such as "simpler" or "harder to understand". You won't make Python easier to learn and teach by adding a new assignment syntax, since people will have to learn the old form as well. Having to break things out over multiple lines is a fact of life, if only for readability when implementing (and maintaining!) non-trivial processing routines. It's a good thing to be used to it, and to learn to choose good names for intermediate variables. Regards Antoine. From greg.ewing at canterbury.ac.nz Tue Apr 24 03:12:52 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 24 Apr 2018 19:12:52 +1200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE7225.5080307@canterbury.ac.nz> Message-ID: <5ADED8F4.1070109@canterbury.ac.nz> Chris Jerdonek wrote: >>>if (diff := x - x_base) and (g := gcd(diff, n)) > 1: > "if diff, which we let equal x - x_base, and g, which ..." or > "if diff, which we set equal to x - x_base, and g, which ...." or > "if diff, which we define to be x - x_base, and g, which ...." or > "if diff, which we define as x - x_base, and g, which ....." etc. How about "being" as a keyword: if (diff being x - x_base) and (g being gcd(diff, n)) > 1: return g An advantage is that you're not likely to be tempted to write diff being x - x_base on its own as a statement. -- Greg From greg.ewing at canterbury.ac.nz Tue Apr 24 03:23:43 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 24 Apr 2018 19:23:43 +1200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <23262.48683.257457.448898@turnbull.sk.tsukuba.ac.jp> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE7225.5080307@canterbury.ac.nz> <23262.48683.257457.448898@turnbull.sk.tsukuba.ac.jp> Message-ID: <5ADEDB7F.2060701@canterbury.ac.nz> Stephen J. Turnbull wrote: > Neologisms are usually written in the > other order: "dead on arrival (DOA, for short)." ;-) Maybe we can make use of that? if (x - x_base) (diff) and gcd(diff, n) (g) > 1: That doesn't work, because the (...) look like function calls. But what if we used a different set of bracketing characters: if (x - x_base) {diff} and gcd(diff, n) {g} > 1: I think that's unambiguous, because you can't currently put {...} straight after an expression. To make it look even more like a neologism definition, we could require the bound names to be all-uppercase. :-) if (x - x_base) {DIFF} and gcd(DIFF, n) {G} > 1: return G -- Greg From rosuav at gmail.com Tue Apr 24 03:24:04 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 24 Apr 2018 17:24:04 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <5ADED8F4.1070109@canterbury.ac.nz> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE7225.5080307@canterbury.ac.nz> <5ADED8F4.1070109@canterbury.ac.nz> Message-ID: On Tue, Apr 24, 2018 at 5:12 PM, Greg Ewing wrote: > Chris Jerdonek wrote: > >>>> if (diff := x - x_base) and (g := gcd(diff, n)) > 1: > > >> "if diff, which we let equal x - x_base, and g, which ..." or >> "if diff, which we set equal to x - x_base, and g, which ...." or >> "if diff, which we define to be x - x_base, and g, which ...." or >> "if diff, which we define as x - x_base, and g, which ....." etc. > > > How about "being" as a keyword: > > if (diff being x - x_base) and (g being gcd(diff, n)) > 1: > return g > > An advantage is that you're not likely to be tempted to write > > diff being x - x_base > > on its own as a statement. Considering that we have ':=', 'as', and 'from', I very much doubt that *any* proposal requiring a new keyword is going to fly. The bar for creating new keywords is a lot higher than that. We hashed out a lot of this on python-ideas; it's almost certainly a waste of time to go through it all again now. I have no intention of editing the PEP to recommend a brand new keyword. ChrisA From tim.peters at gmail.com Tue Apr 24 03:26:50 2018 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 24 Apr 2018 02:26:50 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <20180424085713.1cfa02be@fsol> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <20180423204428.0aa91a39@fsol> <20180424082751.39708d4f@fsol> <20180424085713.1cfa02be@fsol> Message-ID: [Antoine Pitrou ] > ... > Having to break things out over multiple lines is a fact of life, if > only for readability when implementing (and maintaining!) non-trivial > processing routines. It's a good thing to be used to it, and to learn to > choose good names for intermediate variables. Well, the last part is overselling: by its very nature, a binding expression does not relieve the programmer one whit from needing to pick good names. The name is part of the binding expression. The sheer number of names needed is the same with or without binding expressions, although the latter allow for less repetitive typing (& reading) of those names. For the rest, _needing_ to split a simple bind-and-test across two lines doesn't really build character, or have any other virtue (besides familiarity to old-time Python programmers) I can see. Neither does falling into indentation hell have any virtue in the rarer cases where binding expressions really shine. Simple things _should_ be simple to do; indeed, when they are, that's an incentive to keep things simple. There will still be plenty of code where splitting multiple bindings across multiple lines is obviously better. From rosuav at gmail.com Tue Apr 24 03:32:19 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 24 Apr 2018 17:32:19 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <5ADEDB7F.2060701@canterbury.ac.nz> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE7225.5080307@canterbury.ac.nz> <23262.48683.257457.448898@turnbull.sk.tsukuba.ac.jp> <5ADEDB7F.2060701@canterbury.ac.nz> Message-ID: On Tue, Apr 24, 2018 at 5:23 PM, Greg Ewing wrote: > Stephen J. Turnbull wrote: >> >> Neologisms are usually written in the >> other order: "dead on arrival (DOA, for short)." ;-) > > > Maybe we can make use of that? > > if (x - x_base) (diff) and gcd(diff, n) (g) > 1: > > That doesn't work, because the (...) look like function > calls. But what if we used a different set of bracketing > characters: > > if (x - x_base) {diff} and gcd(diff, n) {g} > 1: > > I think that's unambiguous, because you can't currently > put {...} straight after an expression. > > To make it look even more like a neologism definition, > we could require the bound names to be all-uppercase. :-) > > if (x - x_base) {DIFF} and gcd(DIFF, n) {G} > 1: > return G > Great! And to further enhance the neologism parallel, we could allow them to be defined at the bottom of the program. if {DIFF} and {G} > 1: return G with glossary: DIFF: x - x_base G: gcd(DIFF, n) Definite improvement over all the current proposals!! ChrisA From tim.peters at gmail.com Tue Apr 24 03:34:11 2018 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 24 Apr 2018 02:34:11 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <5ADEDB7F.2060701@canterbury.ac.nz> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE7225.5080307@canterbury.ac.nz> <23262.48683.257457.448898@turnbull.sk.tsukuba.ac.jp> <5ADEDB7F.2060701@canterbury.ac.nz> Message-ID: [Stephen J. Turnbull[ >> Neologisms are usually written in the other order: >> "dead on arrival (DOA, for short)." ;-) [Greg Ewing ] > Maybe we can make use of that? > > if (x - x_base) (diff) and gcd(diff, n) (g) > 1: > > That doesn't work, because the (...) look like function > calls. But what if we used a different set of bracketing > characters: > > if (x - x_base) {diff} and gcd(diff, n) {g} > 1: > > I think that's unambiguous, because you can't currently > put {...} straight after an expression. As Guido noted more than once when this was still on python-ideas, this isn't a "a puzzle" to be solved by any technical tricks conceivable. He's not going to accept anything in his language that isn't at least plausibly evident. There's a long & distinguished history of other languages using ":=" for binding, which is why that one gained traction before this moved to python-dev. > To make it look even more like a neologism definition, > we could require the bound names to be all-uppercase. :-) > > if (x - x_base) {DIFF} and gcd(DIFF, n) {G} > 1: > return G Yes - now you're on the right track ;-) From levkivskyi at gmail.com Tue Apr 24 03:27:13 2018 From: levkivskyi at gmail.com (Ivan Levkivskyi) Date: Tue, 24 Apr 2018 08:27:13 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <5ADED8F4.1070109@canterbury.ac.nz> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE7225.5080307@canterbury.ac.nz> <5ADED8F4.1070109@canterbury.ac.nz> Message-ID: On 24 April 2018 at 08:12, Greg Ewing wrote: > Chris Jerdonek wrote: > > if (diff := x - x_base) and (g := gcd(diff, n)) > 1: >>>> >>> > "if diff, which we let equal x - x_base, and g, which ..." or >> "if diff, which we set equal to x - x_base, and g, which ...." or >> "if diff, which we define to be x - x_base, and g, which ...." or >> "if diff, which we define as x - x_base, and g, which ....." etc. >> > > How about "being" as a keyword: > > if (diff being x - x_base) and (g being gcd(diff, n)) > 1: > return g > > An advantage is that you're not likely to be tempted to write > > diff being x - x_base > > on its own as a statement. > > I like this term, but I don't like having more reserved words. Or PEP can make it an official way to read := (z := x + y) is _called_ a binding expression. (z := x + y) _reads_ as "z being x + y" -- Ivan -------------- next part -------------- An HTML attachment was scrubbed... URL: From J.Demeyer at UGent.be Tue Apr 24 04:17:42 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Tue, 24 Apr 2018 10:17:42 +0200 Subject: [Python-Dev] PEP 573 -- Module State Access from C Extension Methods In-Reply-To: <43a667611aa04e60a66372d75c759fa6@xmail101.UGent.be> References: <2854d9cdad73400aa968a084f40908d7@xmail101.UGent.be> <5ADE202B.2060501@UGent.be> <43a667611aa04e60a66372d75c759fa6@xmail101.UGent.be> Message-ID: <5ADEE826.9070806@UGent.be> In PEP 573, instead of passing the defining class to the C function, why not pass the function object itself? That is far more general: once you have the function object, you can still access the defining class using your PyCMethod_CLASS. It's also more future-proof: if we ever decide to add even more attributes to the function object, those could be accessed the same way. In PEP 575, I'm already proposing a flag (METH_ARG0_FUNCTION) to pass the function *instead* of self. Unless PEP 573 is rejected, maybe that should change to passing the function *in addition* to self. Of course, this doesn't quite work with your current version of PEP 573 since METH_METHOD really does two things: it changes the class of the function object (which is not a good idea anyway) and it changes the calling convention. It could work if you add mm_class to PyCFunctionObject instead of creating a new class. Jeroen. From ja.py at farowl.co.uk Tue Apr 24 03:14:12 2018 From: ja.py at farowl.co.uk (Jeff Allen) Date: Tue, 24 Apr 2018 08:14:12 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE7225.5080307@canterbury.ac.nz> Message-ID: <07e63f60-9308-1c25-1f4b-63a84fa0b18f@farowl.co.uk> On 24/04/2018 02:42, Chris Jerdonek wrote: > On Mon, Apr 23, 2018 at 4:54 PM, Greg Ewing wrote: >> Tim Peters wrote: >>> if (diff := x - x_base) and (g := gcd(diff, n)) > 1: >>> return g >> My problem with this is -- how do you read such code out loud? > It could be... > > "if diff, which we define as x - x_base, and g, which ....." etc. > That's good. It also makes it natural to expect only a simple name. One can "define" a name, but assignment to a complex left-side expression is not definition (binding). Jeff Allen -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Tue Apr 24 06:07:39 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 24 Apr 2018 12:07:39 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <20180423204428.0aa91a39@fsol> <20180424082751.39708d4f@fsol> Message-ID: <20180424120739.43ac2d2e@fsol> On Tue, 24 Apr 2018 01:55:13 -0500 Tim Peters wrote: > [Antoine] > > Constructs like "with ..." or "try / except / finally" make the > > language easier to learn compared to the dances they are meant to > > replace. > > They nevertheless need to be taught & learned (and try/except/finally > was essentially always in the language), You snipped the parts > pointing out that binding expressions are already familiar to people > coming from most other languages Yes... I think most will agree that Python is generally easy to take up for people coming from C++ etc., so my "easier to learn and teach" was mostly about non-programmers. > even for raw beginners the > semantics are the tiniest part of what they need to learn anyway about > Python's assignment expressions. I'm not sure what you mean by that. If it's the tiniest part, what's the overwhelming part? Is the new assigment expression that delicate to use that it requires reading a long and intimidating design document ? I didn't get that impression, so it seems you may be making a stronger point than me for rejeting the PEP :-) > > "await" is a more readable and less confusing improvement > > over "yield from". > > Heh. Not to me. I have literally have no idea what to with "await" > (I use generators heavily, but have had no use yet for coroutines), > but use > > yield from an_iterable > > routinely. Yeah... "yield from" is fine for that, except that it was explicitly meant for the coroutine use case as well (I'm not sure what the timeline is, but probably Guido was already thinking/dreaming about tulip/asyncio back then). And trying to shoehorn both in a single construct made it confusing and inadequate. When you want to express two abstractly different concepts (generating a stream of values, or suspending a task until some asynchronous subtask finishes), it makes things easier if those two concepts have two different concrete expressions. Hence "await" making the language easier to learn for those whose use cases benefit from it. To bring another example: the fact that Python has separate syntax to declare classes and functions makes it easier to learn "Python with classes" than "Javascript with classes", even though the raw _grammar_ is made slightly more complex by it. In Javascript you have to learn the weird (ab)use of functional notation for the purpose of declaring object behaviour, and learn to recognize it when you read it. In Python there's the "class" notation which, despite adding a keyword to remember, makes class declaration easier to learn and master. (^^ note this example is about a potentially obsolete dialect of Javascript; perhaps it has class notation nowadays? ^^) > It's simply impossible that, whatever "await" does, it > could be more readable or less confusing than what I use "yield from" > for. Probably because "await" wouldn't work at all for you, then :-) > > Format strings dispense from the older, more convoluted formulations. > > But they didn't _replace_ them. That made teaching/learning harder, > not easier, Intuitively, it sounds easier to teach f'some {value}' rather than either the .format() or %-formatting alternatives. The whole goal of f-strings, after all, is to make string formatting more approachable. Learning a language is not learning the whole spec. When you learn C, you don't need to learn the oddities of pre-ANSI function declarations :-) However, assignment a special case in this regard, since traditional assignment is so omnipresent in online resources, that people _will_ encounter it even if they make a very focused use of Python. > > Iteration is much simpler than the longer forms we would have to write > > if generalized iterators didn't exist. > > I'll buy that one. Now go through the HISTORY file and count all the > changes you didn't name ;-) You claimed that """almost no addition has ever made a language easier to learn for raw beginners""". I claim that several additions did (for Python alone), but I don't need to prove that most of them did ;-) Regards Antoine. From steve at holdenweb.com Tue Apr 24 07:37:06 2018 From: steve at holdenweb.com (Steve Holden) Date: Tue, 24 Apr 2018 12:37:06 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> Message-ID: What facilities does the interpreter currently have for extracting common subexpressions, and how would it verify in such a dynamic environment that such extractions wouldn't alter the semantics of the program? Explicit (assignment using :=) is better than implicit (by optimizations hidden to the programmer). regards Steve Steve Holden On Mon, Apr 23, 2018 at 6:13 PM, Sven R. Kunze wrote: > On 23.04.2018 17:59, Steve Holden wrote: > > > While Tim's expression might look (superficially) like C, the five-line > alternative isn't exactly an inspiring example of Pythonicity, is it? > > > What about > > diff = x - x_base > if diff and gcd(diff, n) > 1: > return gcd(diff, n) > > # or > > if (x - x_base) and gcd(x - x_base, n) > 1: > return gcd(x - x_base, n) > > > > and have the interpreter handle the optimization, or apply an lru_cache? > ;-) > > Cheers, > Sven > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > steve%40holdenweb.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mertz at gnosis.cx Tue Apr 24 08:30:23 2018 From: mertz at gnosis.cx (David Mertz) Date: Tue, 24 Apr 2018 08:30:23 -0400 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE7225.5080307@canterbury.ac.nz> Message-ID: I do think the pronunciation issue that Greg notices is important. I teach Python for most of my living, and reading and discussing code segments is an important part of that. When focussing on how Python actually *spells* something, you can't always jump to the higher-level meaning of a construct. For some complex expression?whether or not "binding expressions" are added?sometimes it makes sense to give a characterization of the *meaning* of the expression, but other times you want to say aloud the entire spelling of the expression. Although feelings are mixed about this, I like the "dunder" contraction for this purpose. It's less of a mouthful to say "dunder-init" than "underscore-underscore-init-underscore-underscore" aloud. And once you learn that shorthand, it's unambiguous. I think I'd pronounce: if (diff := x - x_base) and (g := gcd(diff, n)) > 1: return g As: "If diff bound to x minus x_base (is non-zero), and g bound to gcd of diff comma n is greater than 1, return g" But having a convention for pronouncing this would be nice, rather than it being my idiosyncrasy. On Mon, Apr 23, 2018 at 8:23 PM, Tim Peters wrote: > [Tim] > >> if (diff := x - x_base) and (g := gcd(diff, n)) > 1: > >> return g > > [Greg Ewing ] > > My problem with this is -- how do you read such code out loud? > > In the message in which I first gave that example: > > if the diff isn't 0 and gcd(diff, n) > 1, return the gcd. > That's how I _thought_ of it from the start. > > In my mind, `x - x_base` doesn't even exist except as a low-level > definition of what "diff" means. It's different for the other test: > _there_ `g` doesn't exist except as a shorthand for "the gcd". In one > case it's the name that's important to me, and in the other case the > expression. The entire function from which this came is doing all > arithmetic modulo `n`, so `n` isn't in my mind either - it's a > ubiquitous part of the background in this specific function. > > But you did ask how_I_ would read that code ;-) Anyone else is free > to read it however they like. I naturally read it in the way that > makes most sense to me in its context. > > > > From my Pascal days I'm used to reading ":=" as "becomes". So > > this says: > > > > "If diff becomes x - base and g becomes gcd(diff, n) is > > greater than or equal to 1 then return g." > > > > But "diff becomes x - base" is not what we're testing! > > I don't really follow that. In Python, > > if f() and g > 1: > > first tests whether `f()` "is truthy", regardless of whether it does > or doesn't appear in a binding expression. Because this code is > working with integers, there's an _implied_ "!= 0" comparison. > > > > That makes it sound like the result of x - base may or may not > > get assigned to diff, which is not what's happening at all. > > Then I suggest the problem you're having doesn't stem from the binding > expression, but from that you're omitting to fill in the != 0 part: > if you're not thrown by "greater than 1", I can't see how you can be > thrown by "not zero". > > > > The "as" variant makes more sense when you read it as an > > English sentence: > > > > if ((x - x_base) as diff) and ... > > > > "If x - x_base (and by the way, I'm going to call that > > diff so I can refer to it later) is not zero ..." > > So read the original as "if diff (which is x - x_base) is not zero ...". > > Regardless, Guido has already said "as" is DOA (Dead On Arrival) > (illustrating that it's also common enough in English to give a short > name before its long-winded meaning ;-) ). > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > mertz%40gnosis.cx > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Apr 24 08:53:26 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 24 Apr 2018 22:53:26 +1000 Subject: [Python-Dev] PEP 573 -- Module State Access from C Extension Methods In-Reply-To: <5ADEE826.9070806@UGent.be> References: <2854d9cdad73400aa968a084f40908d7@xmail101.UGent.be> <5ADE202B.2060501@UGent.be> <43a667611aa04e60a66372d75c759fa6@xmail101.UGent.be> <5ADEE826.9070806@UGent.be> Message-ID: On 24 April 2018 at 18:17, Jeroen Demeyer wrote: > In PEP 573, instead of passing the defining class to the C function, why not > pass the function object itself? That is far more general: once you have the > function object, you can still access the defining class using your > PyCMethod_CLASS. It's also more future-proof: if we ever decide to add even > more attributes to the function object, those could be accessed the same > way. > > In PEP 575, I'm already proposing a flag (METH_ARG0_FUNCTION) to pass the > function *instead* of self. Unless PEP 573 is rejected, maybe that should > change to passing the function *in addition* to self. That would definitely be an elegant way of addressing both use cases. > Of course, this doesn't quite work with your current version of PEP 573 > since METH_METHOD really does two things: it changes the class of the > function object (which is not a good idea anyway) and it changes the calling > convention. It could work if you add mm_class to PyCFunctionObject instead > of creating a new class. Creating a new type in the module state access aimed to reduce the potential for unintended side effects, but assuming we go ahead with PEP 575's restructuring of the native function class heirarchy, I agree it would make more sense to add it as a new capability within that heirarchy, rather than adding a dedicated type for it. Regarding the module state access PEP overall - I'm +1 on the proposed changes (the PEP's been through a number of earlier rounds of discussion on import-sig, and my feedback from those has already been incorporated into the current version). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Tue Apr 24 09:37:42 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 24 Apr 2018 23:37:42 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE7225.5080307@canterbury.ac.nz> Message-ID: On 24 April 2018 at 22:30, David Mertz wrote: > I do think the pronunciation issue that Greg notices is important. I teach > Python for most of my living, and reading and discussing code segments is an > important part of that. When focussing on how Python actually *spells* > something, you can't always jump to the higher-level meaning of a construct. > For some complex expression?whether or not "binding expressions" are > added?sometimes it makes sense to give a characterization of the *meaning* > of the expression, but other times you want to say aloud the entire spelling > of the expression. > > Although feelings are mixed about this, I like the "dunder" contraction for > this purpose. It's less of a mouthful to say "dunder-init" than > "underscore-underscore-init-underscore-underscore" aloud. And once you > learn that shorthand, it's unambiguous. > > I think I'd pronounce: > > if (diff := x - x_base) and (g := gcd(diff, n)) > 1: > return g > > As: > > "If diff bound to x minus x_base (is non-zero), and g bound to gcd of diff > comma n is greater than 1, return g" Pronouncing it as "name bound to expr" would also fit well with calling the overall construct a binding expression. You could also postpone the definitions to the end when speaking aloud: "if diff is true and g is greater than 1, then return g, given that diff is bound to ex minus ex-base and g is bound to the gcd of diff and n" However, that long form sounded awkward to me, though, so I ended up wanting to rephrase it as just: "if diff is true and g is greater than 1, then return g, given that diff is ex minus ex-base and g is the gcd of diff and n" (The only change is to replace both occurrences of "is bound to" with a simple "is") And writing that out actually gave me an idea that I don't believe has come up before (or if it did, it got lost somewhere in the depths of a long python-ideas thread): if (diff is= x - x_base) and (g is= gcd(diff, n)) > 1: return g With the mnemonic for what the binding expression means being the following invariant: _rhs = expr assert (name is= _rhs) is _rhs and name is _rhs In a very real sense, that's *exactly* recreating the C pointer semantics for "==" ("check if two pointers reference the same object") vs "=" ("make two pointers reference the same object"), we'd just be spelling it as "is" vs "is=". Given that spelling, a reasonable inline pronunciation of "is=" would still be Davids suggestion of "is bound to": "if diff is bound to ex minux ex-base and is true and g is bound to the gcd of diff and n and is greater than 1, then return g" Simplifying "is bound to" to "is" in the post-definition form would just be a verbal shorthand. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From yselivanov.ml at gmail.com Tue Apr 24 09:38:33 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Tue, 24 Apr 2018 09:38:33 -0400 Subject: [Python-Dev] assignment expressions: an alternative proposal Message-ID: I propose to use the following syntax for assignment expressions: ( NAME = expr ) I know that it was proposed before and this idea was rejected, because accidentally using '=' in place of '==' is a pain point in C/C++/JavaScript. That said, I believe we can still use this syntax as long as we impose the following three restrictions on it: 1. Only NAME token is allowed as a single target. 2. Parenthesis are required. 3. Most importantly: it is *not* allowed to mask names in the current local scope. Let's see how each restriction affects the syntax in detail: (1) NAME tokens only: if (a[1] = value) # SyntaxError if (a.attr = value) # SyntaxError (2) Required parens disambiguate the new syntax from keyword-arguments and prevent using '=' in place of '==': if a = value # SyntaxError if expr and a = value # SyntaxError (3) No masking of existing names in local scope makes using '=' in place of '==' by mistake even less probable: flag = get_flag() ... if (flag = 'win') # SyntaxError # or def foo(value): if (value = 1) # SyntaxError # or py> (c = 1) and (c = 2) # SyntaxError # etc The following code snippets are perfectly valid though: py> a = (b = (c = 3)) py> a, b, c (3, 3, 3) # and py> f = lambda x: x * 10 py> [[(y = f(x)), x/y] for x in range(1,5)] [[10, 0.1], [20, 0.1], [30, 0.1], [40, 0.1]] # and def read(): while (command = input("> ")) != "quit": print('you entered', command) # and py> if (match = re.search(r'wor\w+', 'hello world')): py. print(match) # and if (diff = x - x_base) and (g = gcd(diff, n)) > 1: return g Enabling '=' for assignment expressions introduces a limited form of the more general assignment statement. It is designed to be useful in expressions and is deliberately simple to make it hard for users to shoot in the foot. The required Python grammar changes are simple and unambiguous. Although it is still possible to accidentally mask a global name or a name from an outer scope, the risk of that is significantly lower than masking a local name. IDEs and linters can improve the usability further by highlighting invalid or suspicious assignment expressions. I believe that this syntax is the best of both worlds: it allows to write succinct code just like PEP 572, but without introducing a new ':=' operator. An implementation of this proposal is available here: https://github.com/1st1/cpython/tree/assign. If this idea is deemed viable I will write a PEP detailing the grammar/compiler changes and syntax restrictions. Thanks, Yury From ncoghlan at gmail.com Tue Apr 24 09:46:34 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 24 Apr 2018 23:46:34 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: On 24 April 2018 at 23:38, Yury Selivanov wrote: > I propose to use the following syntax for assignment expressions: > > ( NAME = expr ) > > I know that it was proposed before and this idea was rejected, because > accidentally using '=' in place of '==' is a pain point in > C/C++/JavaScript. > > That said, I believe we can still use this syntax as long as we impose > the following three restrictions on it: > > 1. Only NAME token is allowed as a single target. > > 2. Parenthesis are required. > > 3. Most importantly: it is *not* allowed to mask names in the current > local scope. While I agree this would be unambiguous to a computer, I think for most humans it would be experienced as a confusing set of arcane and arbitrary rules about what "=" means in Python. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From yselivanov.ml at gmail.com Tue Apr 24 09:50:34 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Tue, 24 Apr 2018 09:50:34 -0400 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: On Tue, Apr 24, 2018 at 9:46 AM, Nick Coghlan wrote: > On 24 April 2018 at 23:38, Yury Selivanov wrote: >> I propose to use the following syntax for assignment expressions: >> >> ( NAME = expr ) >> >> I know that it was proposed before and this idea was rejected, because >> accidentally using '=' in place of '==' is a pain point in >> C/C++/JavaScript. >> >> That said, I believe we can still use this syntax as long as we impose >> the following three restrictions on it: >> >> 1. Only NAME token is allowed as a single target. >> >> 2. Parenthesis are required. >> >> 3. Most importantly: it is *not* allowed to mask names in the current >> local scope. > > While I agree this would be unambiguous to a computer, I think for > most humans it would be experienced as a confusing set of arcane and > arbitrary rules about what "=" means in Python. I respectfully disagree. There are no "arcane and confusing rules" about "=", it's rather simple: "=" is always an assignment. "==" is always an equality check. Having two assignment operators feels way more arcane to me. Especially in Python guided by "there should be one way" Zen. Yury From solipsis at pitrou.net Tue Apr 24 09:55:31 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 24 Apr 2018 15:55:31 +0200 Subject: [Python-Dev] assignment expressions: an alternative proposal References: Message-ID: <20180424155531.0edd962a@fsol> On Tue, 24 Apr 2018 23:46:34 +1000 Nick Coghlan wrote: > On 24 April 2018 at 23:38, Yury Selivanov wrote: > > I propose to use the following syntax for assignment expressions: > > > > ( NAME = expr ) > > > > I know that it was proposed before and this idea was rejected, because > > accidentally using '=' in place of '==' is a pain point in > > C/C++/JavaScript. > > > > That said, I believe we can still use this syntax as long as we impose > > the following three restrictions on it: > > > > 1. Only NAME token is allowed as a single target. > > > > 2. Parenthesis are required. > > > > 3. Most importantly: it is *not* allowed to mask names in the current > > local scope. > > While I agree this would be unambiguous to a computer, I think for > most humans it would be experienced as a confusing set of arcane and > arbitrary rules about what "=" means in Python. If the ambition is to find a piece of syntax that reads as "binds", then we can use a variation on the FLUFL operator: "<->". Regards Antoine. From ncoghlan at gmail.com Tue Apr 24 10:07:43 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 25 Apr 2018 00:07:43 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: On 24 April 2018 at 23:50, Yury Selivanov wrote: > On Tue, Apr 24, 2018 at 9:46 AM, Nick Coghlan wrote: >> On 24 April 2018 at 23:38, Yury Selivanov wrote: >>> I propose to use the following syntax for assignment expressions: >>> >>> ( NAME = expr ) >>> >>> I know that it was proposed before and this idea was rejected, because >>> accidentally using '=' in place of '==' is a pain point in >>> C/C++/JavaScript. >>> >>> That said, I believe we can still use this syntax as long as we impose >>> the following three restrictions on it: >>> >>> 1. Only NAME token is allowed as a single target. >>> >>> 2. Parenthesis are required. >>> >>> 3. Most importantly: it is *not* allowed to mask names in the current >>> local scope. >> >> While I agree this would be unambiguous to a computer, I think for >> most humans it would be experienced as a confusing set of arcane and >> arbitrary rules about what "=" means in Python. > > I respectfully disagree. There are no "arcane and confusing rules" > about "=", it's rather simple: > > "=" is always an assignment. > "==" is always an equality check. That's not the distinction I meant, I meant the difficulty of explaining the discrepancies in this list: a = 1 # Assignment (a = 1) # Also assignment a, b = 1, 2 # Tuple assignment (a, b = 1, 2) # SyntaxError. Why? a.b = 1 # Attribute assignment (a.b = 1) # SyntaxError. Why? a[b] = 1 # Subscript assignment (a[b] = 1) # SyntaxError. Why? (a=1), (b=2) # Two assignments a=1, b=2 # SyntaxError. Why? f(a=1, b=2) # Function call with keyword args (a=1, b=2) # SyntaxError. Why? if (a=1): pass # Assignment if a=1: pass # SyntaxError. Why? Whereas if binding expressions use a different symbol, the question is far less likely to arise, and if it does come up, then the answer is the same as the one for def statements vs lambda expressions: because one is a statement, and the other is an expression. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ethan at stoneleaf.us Tue Apr 24 10:24:12 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 24 Apr 2018 07:24:12 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE7225.5080307@canterbury.ac.nz> Message-ID: <5ADF3E0C.9080903@stoneleaf.us> On 04/23/2018 06:42 PM, Chris Jerdonek wrote: > On Mon, Apr 23, 2018 at 4:54 PM, Greg Ewing wrote: >> Tim Peters wrote: >>> if (diff := x - x_base) and (g := gcd(diff, n)) > 1: >>> return g >> >> >> My problem with this is -- how do you read such code out loud? > > It could be-- > > "if diff, which we let equal x - x_base, and g, which ..." or > "if diff, which we set equal to x - x_base, and g, which ...." or > "if diff, which we define to be x - x_base, and g, which ...." or > "if diff, which we define as x - x_base, and g, which ....." etc. Thanks, Chris J. For myself, I can read that as "if diff, which is x - x_base, and g, which is ..." That works for me. Changing my vote to +1 (but only for simple name bindings) -- ~Ethan~ From yselivanov.ml at gmail.com Tue Apr 24 10:23:50 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Tue, 24 Apr 2018 10:23:50 -0400 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: On Tue, Apr 24, 2018 at 10:07 AM, Nick Coghlan wrote: >> "=" is always an assignment. >> "==" is always an equality check. > > That's not the distinction I meant, I meant the difficulty of > explaining the discrepancies in this list: > > a = 1 # Assignment > (a = 1) # Also assignment > > a, b = 1, 2 # Tuple assignment > (a, b = 1, 2) # SyntaxError. Why? > > ... > Whereas if binding expressions use a different symbol, the question is > far less likely to arise, and if it does come up, then the answer is > the same as the one for def statements vs lambda expressions: because > one is a statement, and the other is an expression. A lot of other questions arise though. PEP 572 proposes: a = 1 # assignment a := 1 # also assignment (a := 1) # also assignment (a = 1) # error, why? It's also difficult to explain which one to use when. The net result is that code will be littered with both at random places. That will decrease the readability of Python code at least for some users who have similar taste to myself. With '=' in expressions, the code will look uniform. There will be a simple rule to put parens around assignments in expression and use simple names. After one or two descriptive SyntaxError users will learn how this syntax works (like people learn everything in coding). This all is very subjective. Yury From J.Demeyer at UGent.be Tue Apr 24 10:34:04 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Tue, 24 Apr 2018 16:34:04 +0200 Subject: [Python-Dev] PEP 573 -- Module State Access from C Extension Methods In-Reply-To: <9333b45af89e47b4a0b349f284894435@xmail101.UGent.be> References: <2854d9cdad73400aa968a084f40908d7@xmail101.UGent.be> <5ADE202B.2060501@UGent.be> <43a667611aa04e60a66372d75c759fa6@xmail101.UGent.be> <5ADEE826.9070806@UGent.be> <9333b45af89e47b4a0b349f284894435@xmail101.UGent.be> Message-ID: <5ADF405C.4060607@UGent.be> On 2018-04-24 14:53, Nick Coghlan wrote: >> In PEP 575, I'm already proposing a flag (METH_ARG0_FUNCTION) to pass the >> function *instead* of self. Unless PEP 573 is rejected, maybe that should >> change to passing the function *in addition* to self. > > That would definitely be an elegant way of addressing both use cases. On the other hand, if you are passing the function object, then you can get __self__ from it (unless it's an unbound method: in that case __self__ is NULL and self is really args[0]). So there wouldn't be a need for passing "self". I'm not saying that this is better than passing "self" explicitly... I haven't yet decided what is best. In any case, these things would be handled by Argument Clinic anyway, so it only matters if you are parsing arguments "by hand". Jeroen. From p.f.moore at gmail.com Tue Apr 24 10:49:55 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 24 Apr 2018 15:49:55 +0100 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: On 24 April 2018 at 14:46, Nick Coghlan wrote: > On 24 April 2018 at 23:38, Yury Selivanov wrote: >> I propose to use the following syntax for assignment expressions: >> >> ( NAME = expr ) >> >> I know that it was proposed before and this idea was rejected, because >> accidentally using '=' in place of '==' is a pain point in >> C/C++/JavaScript. >> >> That said, I believe we can still use this syntax as long as we impose >> the following three restrictions on it: >> >> 1. Only NAME token is allowed as a single target. >> >> 2. Parenthesis are required. >> >> 3. Most importantly: it is *not* allowed to mask names in the current >> local scope. > > While I agree this would be unambiguous to a computer, I think for > most humans it would be experienced as a confusing set of arcane and > arbitrary rules about what "=" means in Python. Also, there's the ambiguity and potential for misreading in the opposite direction (accidentally *reading* = as == even though it isn't): if (diff = x - x_base) and (g = gcd(diff, n)) > 1: return g My immediate reading of this is as an equality comparison between diff and x - x_base (which would send me futilely looking for a definition of diff) and an equality comparison of g and gcd(diff, n)... wait, why am I doing (equality comparison) > 1? Oh, hang on... At this point, any hope of me quickly understanding what this code does is lost. Paul From ncoghlan at gmail.com Tue Apr 24 10:51:15 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 25 Apr 2018 00:51:15 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: On 25 April 2018 at 00:23, Yury Selivanov wrote: > A lot of other questions arise though. PEP 572 proposes: > > a = 1 # assignment > a := 1 # also assignment > (a := 1) # also assignment > (a = 1) # error, why? That's just the typical assignment/expression dichotomy, though, which is genuinely confusing for learners (since expression-level Python and statement-level Python allow different constructs), but also not a new problem. All the other keywords that have both statement level and expression level use cases are structured as prefix operators in statement form, and some kind of infix operator in expression form, whereas this would be the first case where we offered a construct that used infix syntax for both its statement form and its expression form. > It's also difficult to explain which one to use when. The net result > is that code will be littered with both at random places. That will > decrease the readability of Python code at least for some users who > have similar taste to myself. That's a legitimate concern with PEP 572 (and part of why I'm somewhere between -1 and -0 on the ":=" spelling, although I'd be +0 on an "is=" spelling that riffs off the "is" comparison operator - using the "name is= expr" spelling in place of a regular assignment looks sufficiently odd that I'd expect the temptation to write it in place of "name = expr" when the latter is permitted would be low) > With '=' in expressions, the code will look uniform. There will be a > simple rule to put parens around assignments in expression and use > simple names. After one or two descriptive SyntaxError users will > learn how this syntax works (like people learn everything in coding). Except that they'll also find other discrepancies like: a = 1 a = 2 being OK, while: a = 1 (a = 2) fails with SyntaxError on the second line. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From anthony.flury at btinternet.com Tue Apr 24 10:54:03 2018 From: anthony.flury at btinternet.com (Anthony Flury) Date: Tue, 24 Apr 2018 15:54:03 +0100 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: On 24/04/18 14:50, Yury Selivanov wrote: > On Tue, Apr 24, 2018 at 9:46 AM, Nick Coghlan wrote: >> On 24 April 2018 at 23:38, Yury Selivanov wrote: >>> I propose to use the following syntax for assignment expressions: >>> >>> ( NAME = expr ) >>> >>> I know that it was proposed before and this idea was rejected, because >>> accidentally using '=' in place of '==' is a pain point in >>> C/C++/JavaScript. >>> >>> That said, I believe we can still use this syntax as long as we impose >>> the following three restrictions on it: >>> >>> 1. Only NAME token is allowed as a single target. >>> >>> 2. Parenthesis are required. >>> >>> 3. Most importantly: it is *not* allowed to mask names in the current >>> local scope. >> While I agree this would be unambiguous to a computer, I think for >> most humans it would be experienced as a confusing set of arcane and >> arbitrary rules about what "=" means in Python. > I respectfully disagree. There are no "arcane and confusing rules" > about "=", it's rather simple: > > "=" is always an assignment. But it isn't - in your proposed syntax : * * = * is an assignment with no return value * *( = )* is an assignment with a returned value ? So now '=' is always an assignment, it is an assignment with extra semantics depending on surrounding syntax. As discussed previously by others on this exact proposals, you now have the issue of? confusion when using keyword arguments : *my_func(a = b)* : clearly that is a call to `my_func' where argument a has the value of b, but if you want to do an assigment expression when calling the function you now have to do *my_func((a=b)) -* which frankly looks messy in my opinion; you get the same issue when you are wanting to do assignment expressions in tuples. Using a different operator for assignments which return values avoids the messy potentially multiple level brackets, and means that the semantics of an operator depends only on that operator and not on syntax elements before and after it. -- -- Anthony Flury email : *Anthony.flury at btinternet.com* Twitter : *@TonyFlury * From ericsnowcurrently at gmail.com Tue Apr 24 10:54:52 2018 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Tue, 24 Apr 2018 08:54:52 -0600 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: Thanks for thinking this through, Yury. :) FWIW, I'm still unconvinced that an assignment expression is worth it. It's hard to say, though, without seeing how much folks would actually use it (and I don't have my own time machine unfortunately). IIRC, in the past several proposed syntax (e.g. decorators) were in the same boat and in retrospect turned out to be a strongly positive addition to the language. :) Comments in-line below. -eric On Tue, Apr 24, 2018 at 7:38 AM, Yury Selivanov wrote: > I propose to use the following syntax for assignment expressions: > > ( NAME = expr ) > > [snip] > > 1. Only NAME token is allowed as a single target. This seems reasonable and does keep assignment expressions easier for the reader to find. At the same time, there have been some arguments elsewhere in favor of tuple unpacking as the other obvious use case. Presumably that would not be supported under this rule. > 2. Parenthesis are required. Similar to rule #1, this would make assignment expressions more obvious to readers, which is a good thing. > > 3. Most importantly: it is *not* allowed to mask names in the current > local scope. I was about to leave this counter example for accidentally-typed-one-equal-sign-instead-of-two bugs: if (y == 3): print(y) # vs. if (y = 3): print(y) Then it dawned on me that your rule #3 solves this. :) > [snip] > > py> f = lambda x: x * 10 > py> [[(y = f(x)), x/y] for x in range(1,5)] > [[10, 0.1], [20, 0.1], [30, 0.1], [40, 0.1]] > > [snip] > > I believe that this syntax is the best of both worlds: it allows to > write succinct code just like PEP 572, but without introducing a new > ':=' operator. This is the main point of this alternate proposal, right? It certainly seems reasonable that we not add another assignment syntax. On the other hand, having ":=" as a distinct syntax for assignment expressions is close enough to the existing syntax that it doesn't really add any real extra burden to readers, while being more searchable and visually distinct. If we were to add assignment expressions I'd probably favor ":=". Regardless, your 3 rules would benefit either syntax. Nick may have a point that the rules might be an excessive burden, but I don't think it's too big a deal since the restrictions are few (and align with the most likely usage) and are limited to syntax so the compiler will be quick to point mistakes. From rosuav at gmail.com Tue Apr 24 10:56:31 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Apr 2018 00:56:31 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: On Wed, Apr 25, 2018 at 12:23 AM, Yury Selivanov wrote: > On Tue, Apr 24, 2018 at 10:07 AM, Nick Coghlan wrote: > >>> "=" is always an assignment. >>> "==" is always an equality check. >> >> That's not the distinction I meant, I meant the difficulty of >> explaining the discrepancies in this list: >> >> a = 1 # Assignment >> (a = 1) # Also assignment >> >> a, b = 1, 2 # Tuple assignment >> (a, b = 1, 2) # SyntaxError. Why? >> >> ... >> Whereas if binding expressions use a different symbol, the question is >> far less likely to arise, and if it does come up, then the answer is >> the same as the one for def statements vs lambda expressions: because >> one is a statement, and the other is an expression. > > A lot of other questions arise though. PEP 572 proposes: > > a = 1 # assignment > a := 1 # also assignment > (a := 1) # also assignment > (a = 1) # error, why? Your third example is just the same as the second, with parentheses around it. In most of Python, parentheses (if legal) have no effect other than grouping; "a + b * c" is the same thing as "(a + b) * c", just done in the other order. The last one is a clear demonstration that "=" is a statement, not an expression. Are people confused by this sort of thing: if x > 1: print("x is more than 1") (if x > 1:) print("SyntaxError") ? Yes, the word 'if' does have meaning in an expression context, and yes, it has a similar meaning to the 'if' statement, but people don't parenthesize entire statements. You try that with assignment, you get an error, and bam, it's obvious that you ran into this particular case. ChrisA From gmarcel.plch at gmail.com Tue Apr 24 10:58:14 2018 From: gmarcel.plch at gmail.com (Marcel Plch) Date: Tue, 24 Apr 2018 16:58:14 +0200 Subject: [Python-Dev] PEP 573 -- Module State Access from C Extension Methods In-Reply-To: <5ADF405C.4060607@UGent.be> References: <2854d9cdad73400aa968a084f40908d7@xmail101.UGent.be> <5ADE202B.2060501@UGent.be> <43a667611aa04e60a66372d75c759fa6@xmail101.UGent.be> <5ADEE826.9070806@UGent.be> <9333b45af89e47b4a0b349f284894435@xmail101.UGent.be> <5ADF405C.4060607@UGent.be> Message-ID: If PEP 575's new call doesn't have any surprising restrictions, I think that completely dropping METH_METHOD would be the best way of resolving this. I suggest we push PEP 575 first and if it gets accepted, I will rebase PEP 573 to these changes. On Tue, Apr 24, 2018 at 4:34 PM, Jeroen Demeyer wrote: > On 2018-04-24 14:53, Nick Coghlan wrote: >>> >>> In PEP 575, I'm already proposing a flag (METH_ARG0_FUNCTION) to pass the >>> function *instead* of self. Unless PEP 573 is rejected, maybe that should >>> change to passing the function *in addition* to self. >> >> >> That would definitely be an elegant way of addressing both use cases. > > > On the other hand, if you are passing the function object, then you can get > __self__ from it (unless it's an unbound method: in that case __self__ is > NULL and self is really args[0]). So there wouldn't be a need for passing > "self". I'm not saying that this is better than passing "self" explicitly... > I haven't yet decided what is best. > > In any case, these things would be handled by Argument Clinic anyway, so it > only matters if you are parsing arguments "by hand". > > > > Jeroen. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/gmarcel.plch%40gmail.com From yselivanov.ml at gmail.com Tue Apr 24 10:58:24 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Tue, 24 Apr 2018 10:58:24 -0400 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: On Tue, Apr 24, 2018 at 10:49 AM, Paul Moore wrote: [..] >>> 3. Most importantly: it is *not* allowed to mask names in the current >>> local scope. >> >> While I agree this would be unambiguous to a computer, I think for >> most humans it would be experienced as a confusing set of arcane and >> arbitrary rules about what "=" means in Python. > > Also, there's the ambiguity and potential for misreading in the > opposite direction (accidentally *reading* = as == even though it > isn't): > > if (diff = x - x_base) and (g = gcd(diff, n)) > 1: > return g Since 'diff' and 'g' must be new names according to rule (3), those who read the code will notice that both were not previously bound. Therefore both are new variables so it can't be a comparison. Yury From yselivanov.ml at gmail.com Tue Apr 24 11:03:35 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Tue, 24 Apr 2018 11:03:35 -0400 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: On Tue, Apr 24, 2018 at 10:56 AM, Chris Angelico wrote: [..] >> A lot of other questions arise though. PEP 572 proposes: >> >> a = 1 # assignment >> a := 1 # also assignment >> (a := 1) # also assignment >> (a = 1) # error, why? > > Your third example is just the same as the second, with parentheses > around it. In most of Python, parentheses (if legal) have no effect > other than grouping; "a + b * c" is the same thing as "(a + b) * c", > just done in the other order. The last one is a clear demonstration > that "=" is a statement, not an expression. Are people confused by > this sort of thing: > > if x > 1: > print("x is more than 1") > (if x > 1:) > print("SyntaxError") This is a very far-fetched example :) My point was that when you see lots of '=' and ':=' used at the statement level, one might try to write "if x = 1" instead of "if x := 1" -- boom, we have an unexpected SyntaxError for some users. In my opinion adding *any* assignment expression syntax to Python *will* create this sort of issues. PEP 572 isn't free of them, my proposal isn't free of them. My proposal doesn't add a new ':=' operator at the cost of slightly complicating rules around '='. PEP 572 avoids complicating '=', but adds an entirely new form of assignment. Yury From yselivanov.ml at gmail.com Tue Apr 24 11:05:57 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Tue, 24 Apr 2018 11:05:57 -0400 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: On Tue, Apr 24, 2018 at 10:54 AM, Anthony Flury via Python-Dev wrote: [..] > As discussed previously by others on this exact proposals, you now have the > issue of confusion when using keyword arguments : *my_func(a = b)* : > clearly that is a call to `my_func' where argument a has the value of b, but > if you want to do an assigment expression when calling the function you now > have to do *my_func((a=b)) -* which frankly looks messy in my opinion; you > get the same issue when you are wanting to do assignment expressions in > tuples. Well, `my_func(a=(b:=foo))` or `my_func(b:=foo)` are also barely readable to my eye. My expectation is that users won't use any form of assignment expressions in function calls, it's painful with both proposals. Yury From p.f.moore at gmail.com Tue Apr 24 11:07:10 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 24 Apr 2018 16:07:10 +0100 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: On 24 April 2018 at 15:58, Yury Selivanov wrote: > On Tue, Apr 24, 2018 at 10:49 AM, Paul Moore wrote: > [..] >>>> 3. Most importantly: it is *not* allowed to mask names in the current >>>> local scope. >>> >>> While I agree this would be unambiguous to a computer, I think for >>> most humans it would be experienced as a confusing set of arcane and >>> arbitrary rules about what "=" means in Python. >> >> Also, there's the ambiguity and potential for misreading in the >> opposite direction (accidentally *reading* = as == even though it >> isn't): >> >> if (diff = x - x_base) and (g = gcd(diff, n)) > 1: >> return g > > Since 'diff' and 'g' must be new names according to rule (3), those > who read the code will notice that both were not previously bound. > Therefore both are new variables so it can't be a comparison. That was essentially my point, though - I can no longer read that line of code in isolation from the surrounding context. Consider something like a github PR review screen, where surrounding unchanged code is frequently hidden. Anyway, we can agree to differ on this - I don't like this idea and I'd personally find it hard to read, but as you've already pointed out, this is all extremely subjective. Paul From rosuav at gmail.com Tue Apr 24 11:07:47 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Apr 2018 01:07:47 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: On Wed, Apr 25, 2018 at 12:54 AM, Anthony Flury via Python-Dev wrote: > As discussed previously by others on this exact proposals, you now have the > issue of confusion when using keyword arguments : *my_func(a = b)* : > clearly that is a call to `my_func' where argument a has the value of b, but > if you want to do an assigment expression when calling the function you now > have to do *my_func((a=b)) -* which frankly looks messy in my opinion; you > get the same issue when you are wanting to do assignment expressions in > tuples. To be fair, function arguments already follow "practicality beats purity" in many ways. Let's look at tuples: x = 1, 2 # fine x = (1, 2) # fine x = 1, # fine, though not advisable x = (1,) # fine But if you're going to use a tuple literal as a function parameter, you have to give it extra parens: f((1, 2)) # one arg, a tuple f(1, 2) # two args The comma has multiple meanings, and it has to be disambiguated. The equals sign would be the same. I'm still strongly -1 on any proposal to have "=" mean assignment in any expression context, though. It is WAY too easy for a comparison to sneakily become an assignment, or to get bizarre syntax errors: x = 1 if (x = 2): ... This, according to your proposal, raises SyntaxError - not because a comparison was wanted and an assignment was made, but because the name already had a value. And, even worse, this is NOT an error: x = 1 def f(): if (x = 2): ... That's a bizarre distinction. ChrisA From rosuav at gmail.com Tue Apr 24 11:10:52 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Apr 2018 01:10:52 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: On Wed, Apr 25, 2018 at 12:58 AM, Yury Selivanov wrote: > On Tue, Apr 24, 2018 at 10:49 AM, Paul Moore wrote: > [..] >>>> 3. Most importantly: it is *not* allowed to mask names in the current >>>> local scope. >>> >>> While I agree this would be unambiguous to a computer, I think for >>> most humans it would be experienced as a confusing set of arcane and >>> arbitrary rules about what "=" means in Python. >> >> Also, there's the ambiguity and potential for misreading in the >> opposite direction (accidentally *reading* = as == even though it >> isn't): >> >> if (diff = x - x_base) and (g = gcd(diff, n)) > 1: >> return g > > Since 'diff' and 'g' must be new names according to rule (3), those > who read the code will notice that both were not previously bound. > Therefore both are new variables so it can't be a comparison. That would not be true if this code were in a loop. Or do you have a different definition of "not previously bound" that is actually a syntactic feature? For instance: if (x = 1): x = 2 Legal? Not legal? ChrisA From rosuav at gmail.com Tue Apr 24 11:15:22 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Apr 2018 01:15:22 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: On Wed, Apr 25, 2018 at 1:03 AM, Yury Selivanov wrote: > On Tue, Apr 24, 2018 at 10:56 AM, Chris Angelico wrote: > [..] >>> A lot of other questions arise though. PEP 572 proposes: >>> >>> a = 1 # assignment >>> a := 1 # also assignment >>> (a := 1) # also assignment >>> (a = 1) # error, why? >> >> Your third example is just the same as the second, with parentheses >> around it. In most of Python, parentheses (if legal) have no effect >> other than grouping; "a + b * c" is the same thing as "(a + b) * c", >> just done in the other order. The last one is a clear demonstration >> that "=" is a statement, not an expression. Are people confused by >> this sort of thing: >> >> if x > 1: >> print("x is more than 1") >> (if x > 1:) >> print("SyntaxError") > > This is a very far-fetched example :) Heh, yes it is. But my point is that the parens are not creating a weird situation here. They're just showcasing a distinction: one of these is a statement, the other an expression. Which is the entire point of the different operator - one is a syntactic feature of a statement that creates one or more name bindings, the other is a binary operator which results in a name binding as well as a value. ChrisA From steve at pearwood.info Tue Apr 24 11:15:28 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 25 Apr 2018 01:15:28 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: <20180424151528.GF11616@ando.pearwood.info> On Tue, Apr 24, 2018 at 09:50:34AM -0400, Yury Selivanov wrote: > On Tue, Apr 24, 2018 at 9:46 AM, Nick Coghlan wrote: > > On 24 April 2018 at 23:38, Yury Selivanov wrote: > >> I propose to use the following syntax for assignment expressions: > >> > >> ( NAME = expr ) > >> > >> I know that it was proposed before and this idea was rejected, because > >> accidentally using '=' in place of '==' is a pain point in > >> C/C++/JavaScript. > >> > >> That said, I believe we can still use this syntax as long as we impose > >> the following three restrictions on it: > >> > >> 1. Only NAME token is allowed as a single target. > >> > >> 2. Parenthesis are required. There are many places where I would use parentheses even if they are not required, but being forced to use them when they're not and I don't want them is ugly. I also question why you think this will help prevent accidentally writing = when you meant == (equality). Have you never written something like this? if (x == y) or (a > b): ... Yes, I know the parens are not strictly needed, since the precedence of `or` is lower than the comparison operators. But still, especially for complex comparisons, a few extra (round) brackets can improve readability. So now we have: if (x = y) or (a > b): ... # oops But the biggest problem with this is that it is ambiguous to the human reader. At a glance, I'm likely to read x=y in an expression as equality. If I notice that it is a single = sign, I'm never going to be sure whether it was a mistake or intentional until I study the rest of the function minutely. The benefit of := is that if I see it, I can be pretty sure it was not a typo. It is hard to mistype == as := by accident, and they are visually distinct enough that I am not going to misread := as == . > >> 3. Most importantly: it is *not* allowed to mask names in the current > >> local scope. That means you can't rebind existing variables. That means you can't rebind to the same variable in a loop. I believe that one of the most important use-cases for binding- expression syntax is while loops, like this modified example taken from PEP 572 version 3: while (data = sock.read()): print("Received data:", data) If you prohibit re-binding data, that prohibits cases like this, or even using it inside a loop: for value in sequence: process(arg, (item = expression), item+1) Your suggestion to prohibit rebinding variables effectively makes them so crippled as to be useless to me. > > While I agree this would be unambiguous to a computer, I think for > > most humans it would be experienced as a confusing set of arcane and > > arbitrary rules about what "=" means in Python. > > I respectfully disagree. There are no "arcane and confusing rules" > about "=", it's rather simple: > > "=" is always an assignment. Why is this allowed? x = 1 # both are statement forms x = 2 but this is prohibited? x = 1 (x = 2) # no rebinding is allowed and even more confusing, this is allowed! (x = 1) # x doesn't exist yet, so it is allowed x = 2 # statement assignment is allowed to rebind By the way, the check for existing variables cannot help to be either incomplete or incorrect if you try to do it at compile time: from module import * (x = 2) # allowed or not allowed? If you don't like wild-card imports, how about this: if random.random() > 0.5: spam = 1 else: eggs = 1 (spam = 2) # allowed or not? no way to tell at compile time But doing the rebinding/shadowing check at runtime will slow down binding expressions, and lead to even more arcane and confusing results: it = iter("abc") while (obj = next(it)): print(obj) will print "a" on the first loop, but then raise an exception on the second time loop as obj now exists. -- Steve From christoph at grothesque.org Tue Apr 24 11:16:54 2018 From: christoph at grothesque.org (Christoph Groth) Date: Tue, 24 Apr 2018 17:16:54 +0200 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: Message-ID: <87po2o3a8p.fsf@grothesque.org> Yury Selivanov wrote: > I propose to use the following syntax for assignment expressions: > > ( NAME = expr ) Yury, did you notice the subthread [1] that discusses the merits and problems of the same idea (except for your point 3)? [1] https://mail.python.org/pipermail/python-dev/2018-April/152868.html From yselivanov.ml at gmail.com Tue Apr 24 11:19:27 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Tue, 24 Apr 2018 11:19:27 -0400 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: On Tue, Apr 24, 2018 at 11:07 AM, Chris Angelico wrote: [..] > x = 1 > if (x = 2): ... > > This, according to your proposal, raises SyntaxError - not because a > comparison was wanted and an assignment was made, but because the name > already had a value. And, even worse, this is NOT an error: Yes, because I'm trying to think about this from a pragmatic side of things. My question to myself: "what syntax could I use that would prevent me from making '=' vs '==' mistake when I code?" To me, the answer is that I usually want to compare local variables. When I compare to variables from outer scopes they *usually* are on the *right* side of '=='. > > x = 1 > def f(): > if (x = 2): > ... > > That's a bizarre distinction. Chris, FWIW I'm trying to avoid using 'bizarre', 'arcane' etc with regards to PEP 572 or any proposal, really. For example, I, personally, find ':=' bizarre, but it's subjective and it's unproductive to say that. Yury From yselivanov.ml at gmail.com Tue Apr 24 11:25:58 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Tue, 24 Apr 2018 11:25:58 -0400 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: <20180424151528.GF11616@ando.pearwood.info> References: <20180424151528.GF11616@ando.pearwood.info> Message-ID: On Tue, Apr 24, 2018 at 11:15 AM, Steven D'Aprano wrote: [..] >> >> 3. Most importantly: it is *not* allowed to mask names in the current >> >> local scope. > > That means you can't rebind existing variables. That means you can't > rebind to the same variable in a loop. No, it doesn't. The check is performed during compile phase, and Python does not unroll loops. Anyways, read below. > I believe that one of the most important use-cases for binding- > expression syntax is while loops, like this modified example taken from > PEP 572 version 3: > > while (data = sock.read()): > print("Received data:", data) > > If you prohibit re-binding data, that prohibits cases like this, or even > using it inside a loop: > > for value in sequence: > process(arg, (item = expression), item+1) No it doesn't. symtable in Python works differently. I encourage you to test my reference implementation: py> for val in [1, 2, 3]: ... print((item=val), item+1) ... 1 2 2 3 3 4 > Why is this allowed? > > x = 1 # both are statement forms > x = 2 > > but this is prohibited? > > x = 1 > (x = 2) # no rebinding is allowed > > and even more confusing, this is allowed! > > (x = 1) # x doesn't exist yet, so it is allowed > x = 2 # statement assignment is allowed to rebind These all are very limited code snippets that you're unlikely to see in real code. I can write (and I did in this thread) a bunch of examples of where PEP 572 is also inconsistent. Yury From steve at pearwood.info Tue Apr 24 11:27:59 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 25 Apr 2018 01:27:59 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: <20180424152759.GG11616@ando.pearwood.info> On Tue, Apr 24, 2018 at 11:03:35AM -0400, Yury Selivanov wrote: > My point was that when you see lots of '=' and ':=' used at the > statement level, one might try to write "if x = 1" instead of "if x := > 1" -- boom, we have an unexpected SyntaxError for some users. That's a *good* thing. They will then learn not to write x = 1 as an expression. Also, if I write lots of x := 1 binding-expressions as statements, my code is bad and deserves to fail code-review. But why would I write the extra colon (one character, two key-presses) to use x := 1 as a statement, when x = 1 will work? That's a sure sign that I don't know what I'm doing. (Or that I desperately wish I was writing Pascal.) I don't think we need worry about this. The sort of code that is filled with binding-expressions used as statements will almost certainly be so un-Pythonic and ugly in many other ways, that this won't make any difference. > In my opinion adding *any* assignment expression syntax to Python > *will* create this sort of issues. PEP 572 isn't free of them, my > proposal isn't free of them. My proposal doesn't add a new ':=' > operator at the cost of slightly complicating rules around '='. PEP > 572 avoids complicating '=', but adds an entirely new form of > assignment. Indeed. That is true: either way, we introduce complexity into the language. (But that will allow us to reduce complexity in *our* code.) Given that increasing complexity is inevitable regardless of whether we choose PEP 572 or your suggestion, it is better to choose the option which keeps binding-expressions and assignment statements separate, since they are two different concepts. -- Steve From rosuav at gmail.com Tue Apr 24 11:28:32 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Apr 2018 01:28:32 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: <20180424151528.GF11616@ando.pearwood.info> References: <20180424151528.GF11616@ando.pearwood.info> Message-ID: On Wed, Apr 25, 2018 at 1:15 AM, Steven D'Aprano wrote: > By the way, the check for existing variables cannot help to be either > incomplete or incorrect if you try to do it at compile time: > > > from module import * > (x = 2) # allowed or not allowed? > > > If you don't like wild-card imports, how about this: > > if random.random() > 0.5: > spam = 1 > else: > eggs = 1 > (spam = 2) # allowed or not? no way to tell at compile time > > > But doing the rebinding/shadowing check at runtime will slow down > binding expressions, and lead to even more arcane and confusing results: > > it = iter("abc") > while (obj = next(it)): > print(obj) > > > will print "a" on the first loop, but then raise an exception on > the second time loop as obj now exists. On re-thinking this, I think the distinction IS possible, but (a) only in function/class scope, not at global; and (b) would be defined in terms of lexical position, not run-time. For instance: def f(): (a = 1) # Legal; 'a' has not been used yet a = 2 # doesn't change that def f(a): (a = 1) # Invalid - 'a' has been used already def f(): while (a = get_next()): # Legal ... This could be handled in the symbol collection pass; if the name already exists in the function's locals, it's disallowed. But I still stand by my statement that this creates bizarre cases, and yes, I know that that word is subjective (just like "readable", "intuitive", and "sensible"). The rules as given sound like they would make great linter rules and terrible syntax rules. They are closely aligned with the OP's experience and usage patterns - which means that, as a personal linter, they could marvellously assist in catching bugs. You could have personal (or organization-wide) linter rules disallowing "class foo:" with a lower-case name, and disallowing the rebinding of any name in ALL_CAPS, but I would not want either rule codified into language syntax. ChrisA From ncoghlan at gmail.com Tue Apr 24 11:31:25 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 25 Apr 2018 01:31:25 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: On 25 April 2018 at 00:54, Eric Snow wrote: > Regardless, your 3 rules would benefit either syntax. Nick may have a > point that the rules might be an excessive burden, but I don't think > it's too big a deal since the restrictions are few (and align with the > most likely usage) and are limited to syntax so the compiler will be > quick to point mistakes. I think the "single name target only" rule should be in place no matter the syntax for the name binding operator itself. I don't mind too much either way on the mandatory parentheses question (it's certainly an easy option to actively discourage use of binding expressions as a direct alternative to assignment statements, but as with the single-name-only rule, it's independent of the choice of syntax) I *do* think the "no name rebinding except in a while loop header" restriction would be annoying for the if/elif use case and the while use case: while (item = get_item()) is not first_delimiter: # First processing loop while (item = get_item()) is not second_delimiter: # Second processing loop # etc... if (target = get_first_candidate()) is not None: ... elif (target = get_second_candidate()) is not None: ... elif (target = get_third_candidate()) is not None: ... And *that* rule is unique to the "=" spelling, since for other proposals "lhs = rhs" in an expression is *always* a syntax error, and you have to resolve the ambiguity in intent explicitly by either adding a second "=" (to request equality comparison), or else some other leading symbol (to request a binding expression). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From rosuav at gmail.com Tue Apr 24 11:31:54 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Apr 2018 01:31:54 +1000 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! Message-ID: The most notable change since last posting is that the assignment target is no longer as flexible as with the statement form of assignment, but is restricted to a simple name. Note that the reference implementation has not been updated. ChrisA PEP: 572 Title: Assignment Expressions Author: Chris Angelico Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 28-Feb-2018 Python-Version: 3.8 Post-History: 28-Feb-2018, 02-Mar-2018, 23-Mar-2018, 04-Apr-2018, 17-Apr-2018, 25-Apr-2018 Abstract ======== This is a proposal for creating a way to assign to variables within an expression. Additionally, the precise scope of comprehensions is adjusted, to maintain consistency and follow expectations. Rationale ========= Naming the result of an expression is an important part of programming, allowing a descriptive name to be used in place of a longer expression, and permitting reuse. Currently, this feature is available only in statement form, making it unavailable in list comprehensions and other expression contexts. Merely introducing a way to assign as an expression would create bizarre edge cases around comprehensions, though, and to avoid the worst of the confusions, we change the definition of comprehensions, causing some edge cases to be interpreted differently, but maintaining the existing behaviour in the majority of situations. Syntax and semantics ==================== In any context where arbitrary Python expressions can be used, a **named expression** can appear. This is of the form ``name := expr`` where ``expr`` is any valid Python expression, and ``name`` is an identifier. The value of such a named expression is the same as the incorporated expression, with the additional side-effect that the target is assigned that value:: # Handle a matched regex if (match := pattern.search(data)) is not None: ... # A more explicit alternative to the 2-arg form of iter() invocation while (value := read_next_item()) is not None: ... # Share a subexpression between a comprehension filter clause and its output filtered_data = [y for x in data if (y := f(x)) is not None] Differences from regular assignment statements ---------------------------------------------- Most importantly, since ``:=`` is an expression, it can be used in contexts where statements are illegal, including lambda functions and comprehensions. An assignment statement can assign to multiple targets, left-to-right:: x = y = z = 0 The equivalent assignment expression is parsed as separate binary operators, and is therefore processed right-to-left, as if it were spelled thus:: assert 0 == (x := (y := (z := 0))) Statement assignment can include annotations. This would be syntactically noisy in expressions, and is of minor importance. An annotation can be given separately from the assignment if needed:: x:str = "" # works (x:str := "") # SyntaxError x:str # possibly before a loop (x := "") # fine Augmented assignment is not supported in expression form:: >>> x +:= 1 File "", line 1 x +:= 1 ^ SyntaxError: invalid syntax Statement assignment is able to set attributes and subscripts, but expression assignment is restricted to names. (This restriction may be relaxed in a future version of Python.) Otherwise, the semantics of assignment are identical in statement and expression forms. Alterations to comprehensions ----------------------------- The current behaviour of list/set/dict comprehensions and generator expressions has some edge cases that would behave strangely if an assignment expression were to be used. Therefore the proposed semantics are changed, removing the current edge cases, and instead altering their behaviour *only* in a class scope. As of Python 3.7, the outermost iterable of any comprehension is evaluated in the surrounding context, and then passed as an argument to the implicit function that evaluates the comprehension. Under this proposal, the entire body of the comprehension is evaluated in its implicit function. Names not assigned to within the comprehension are located in the surrounding scopes, as with normal lookups. As one special case, a comprehension at class scope will **eagerly bind** any name which is already defined in the class scope. A list comprehension can be unrolled into an equivalent function. With Python 3.7 semantics:: numbers = [x + y for x in range(3) for y in range(4)] # Is approximately equivalent to def (iterator): result = [] for x in iterator: for y in range(4): result.append(x + y) return result numbers = (iter(range(3))) Under the new semantics, this would instead be equivalent to:: def (): result = [] for x in range(3): for y in range(4): result.append(x + y) return result numbers = () When a class scope is involved, a naive transformation into a function would prevent name lookups (as the function would behave like a method):: class X: names = ["Fred", "Barney", "Joe"] prefix = "> " prefixed_names = [prefix + name for name in names] With Python 3.7 semantics, this will evaluate the outermost iterable at class scope, which will succeed; but it will evaluate everything else in a function:: class X: names = ["Fred", "Barney", "Joe"] prefix = "> " def (iterator): result = [] for name in iterator: result.append(prefix + name) return result prefixed_names = (iter(names)) The name ``prefix`` is thus searched for at global scope, ignoring the class name. Under the proposed semantics, this name will be eagerly bound; and the same early binding then handles the outermost iterable as well. The list comprehension is thus approximately equivalent to:: class X: names = ["Fred", "Barney", "Joe"] prefix = "> " def (names=names, prefix=prefix): result = [] for name in names: result.append(prefix + name) return result prefixed_names = () With list comprehensions, this is unlikely to cause any confusion. With generator expressions, this has the potential to affect behaviour, as the eager binding means that the name could be rebound between the creation of the genexp and the first call to ``next()``. It is, however, more closely aligned to normal expectations. The effect is ONLY seen with names that are looked up from class scope; global names (eg ``range()``) will still be late-bound as usual. One consequence of this change is that certain bugs in genexps will not be detected until the first call to ``next()``, where today they would be caught upon creation of the generator. Recommended use-cases ===================== Simplifying list comprehensions ------------------------------- A list comprehension can map and filter efficiently by capturing the condition:: results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0] Similarly, a subexpression can be reused within the main expression, by giving it a name on first use:: stuff = [[y := f(x), x/y] for x in range(5)] # There are a number of less obvious ways to spell this in current # versions of Python, such as: # Inline helper function stuff = [(lambda y: [y,x/y])(f(x)) for x in range(5)] # Extra 'for' loop - potentially could be optimized internally stuff = [[y, x/y] for x in range(5) for y in [f(x)]] # Using a mutable cache object (various forms possible) c = {} stuff = [[c.update(y=f(x)) or c['y'], x/c['y']] for x in range(5)] In all cases, the name is local to the comprehension; like iteration variables, it cannot leak out into the surrounding context. Capturing condition values -------------------------- Assignment expressions can be used to good effect in the header of an ``if`` or ``while`` statement:: # Proposed syntax while (command := input("> ")) != "quit": print("You entered:", command) # Capturing regular expression match objects # See, for instance, Lib/pydoc.py, which uses a multiline spelling # of this effect if match := re.search(pat, text): print("Found:", match.group(0)) # Reading socket data until an empty string is returned while data := sock.read(): print("Received data:", data) # Equivalent in current Python, not caring about function return value while input("> ") != "quit": print("You entered a command.") # To capture the return value in current Python demands a four-line # loop header. while True: command = input("> "); if command == "quit": break print("You entered:", command) Particularly with the ``while`` loop, this can remove the need to have an infinite loop, an assignment, and a condition. It also creates a smooth parallel between a loop which simply uses a function call as its condition, and one which uses that as its condition but also uses the actual value. Rejected alternative proposals ============================== Proposals broadly similar to this one have come up frequently on python-ideas. Below are a number of alternative syntaxes, some of them specific to comprehensions, which have been rejected in favour of the one given above. Alternative spellings --------------------- Broadly the same semantics as the current proposal, but spelled differently. 1. ``EXPR as NAME``:: stuff = [[f(x) as y, x/y] for x in range(5)] Since ``EXPR as NAME`` already has meaning in ``except`` and ``with`` statements (with different semantics), this would create unnecessary confusion or require special-casing (eg to forbid assignment within the headers of these statements). 2. ``EXPR -> NAME``:: stuff = [[f(x) -> y, x/y] for x in range(5)] This syntax is inspired by languages such as R and Haskell, and some programmable calculators. (Note that a left-facing arrow ``y <- f(x)`` is not possible in Python, as it would be interpreted as less-than and unary minus.) This syntax has a slight advantage over 'as' in that it does not conflict with ``with`` and ``except`` statements, but otherwise is equivalent. 3. Adorning statement-local names with a leading dot:: stuff = [[(f(x) as .y), x/.y] for x in range(5)] # with "as" stuff = [[(.y := f(x)), x/.y] for x in range(5)] # with ":=" This has the advantage that leaked usage can be readily detected, removing some forms of syntactic ambiguity. However, this would be the only place in Python where a variable's scope is encoded into its name, making refactoring harder. 4. Adding a ``where:`` to any statement to create local name bindings:: value = x**2 + 2*x where: x = spam(1, 4, 7, q) Execution order is inverted (the indented body is performed first, followed by the "header"). This requires a new keyword, unless an existing keyword is repurposed (most likely ``with:``). See PEP 3150 for prior discussion on this subject (with the proposed keyword being ``given:``). 5. ``TARGET from EXPR``:: stuff = [[y from f(x), x/y] for x in range(5)] This syntax has fewer conflicts than ``as`` does (conflicting only with the ``raise Exc from Exc`` notation), but is otherwise comparable to it. Instead of paralleling ``with expr as target:`` (which can be useful but can also be confusing), this has no parallels, but is evocative. Special-casing conditional statements ------------------------------------- One of the most popular use-cases is ``if`` and ``while`` statements. Instead of a more general solution, this proposal enhances the syntax of these two statements to add a means of capturing the compared value:: if re.search(pat, text) as match: print("Found:", match.group(0)) This works beautifully if and ONLY if the desired condition is based on the truthiness of the captured value. It is thus effective for specific use-cases (regex matches, socket reads that return `''` when done), and completely useless in more complicated cases (eg where the condition is ``f(x) < 0`` and you want to capture the value of ``f(x)``). It also has no benefit to list comprehensions. Advantages: No syntactic ambiguities. Disadvantages: Answers only a fraction of possible use-cases, even in ``if``/``while`` statements. Special-casing comprehensions ----------------------------- Another common use-case is comprehensions (list/set/dict, and genexps). As above, proposals have been made for comprehension-specific solutions. 1. ``where``, ``let``, or ``given``:: stuff = [(y, x/y) where y = f(x) for x in range(5)] stuff = [(y, x/y) let y = f(x) for x in range(5)] stuff = [(y, x/y) given y = f(x) for x in range(5)] This brings the subexpression to a location in between the 'for' loop and the expression. It introduces an additional language keyword, which creates conflicts. Of the three, ``where`` reads the most cleanly, but also has the greatest potential for conflict (eg SQLAlchemy and numpy have ``where`` methods, as does ``tkinter.dnd.Icon`` in the standard library). 2. ``with NAME = EXPR``:: stuff = [(y, x/y) with y = f(x) for x in range(5)] As above, but reusing the `with` keyword. Doesn't read too badly, and needs no additional language keyword. Is restricted to comprehensions, though, and cannot as easily be transformed into "longhand" for-loop syntax. Has the C problem that an equals sign in an expression can now create a name binding, rather than performing a comparison. Would raise the question of why "with NAME = EXPR:" cannot be used as a statement on its own. 3. ``with EXPR as NAME``:: stuff = [(y, x/y) with f(x) as y for x in range(5)] As per option 2, but using ``as`` rather than an equals sign. Aligns syntactically with other uses of ``as`` for name binding, but a simple transformation to for-loop longhand would create drastically different semantics; the meaning of ``with`` inside a comprehension would be completely different from the meaning as a stand-alone statement, while retaining identical syntax. Regardless of the spelling chosen, this introduces a stark difference between comprehensions and the equivalent unrolled long-hand form of the loop. It is no longer possible to unwrap the loop into statement form without reworking any name bindings. The only keyword that can be repurposed to this task is ``with``, thus giving it sneakily different semantics in a comprehension than in a statement; alternatively, a new keyword is needed, with all the costs therein. Lowering operator precedence ---------------------------- There are two logical precedences for the ``:=`` operator. Either it should bind as loosely as possible, as does statement-assignment; or it should bind more tightly than comparison operators. Placing its precedence between the comparison and arithmetic operators (to be precise: just lower than bitwise OR) allows most uses inside ``while`` and ``if`` conditions to be spelled without parentheses, as it is most likely that you wish to capture the value of something, then perform a comparison on it:: pos = -1 while pos := buffer.find(search_term, pos + 1) >= 0: ... Once find() returns -1, the loop terminates. If ``:=`` binds as loosely as ``=`` does, this would capture the result of the comparison (generally either ``True`` or ``False``), which is less useful. While this behaviour would be convenient in many situations, it is also harder to explain than "the := operator behaves just like the assignment statement", and as such, the precedence for ``:=`` has been made as close as possible to that of ``=``. Migration path ============== The semantic changes to list/set/dict comprehensions, and more so to generator expressions, may potentially require migration of code. In many cases, the changes simply make legal what used to raise an exception, but there are some edge cases that were previously legal and now are not, and a few corner cases with altered semantics. The Outermost Iterable ---------------------- As of Python 3.7, the outermost iterable in a comprehension is special: it is evaluated in the surrounding context, instead of inside the comprehension. Thus it is permitted to contain a ``yield`` expression, to use a name also used elsewhere, and to reference names from class scope. Also, in a genexp, the outermost iterable is pre-evaluated, but the rest of the code is not touched until the genexp is first iterated over. Class scope is now handled more generally (see above), but if other changes require the old behaviour, the iterable must be explicitly elevated from the comprehension:: # Python 3.7 def f(x): return [x for x in x if x] def g(): return [x for x in [(yield 1)]] # With PEP 572 def f(x): return [y for y in x if y] def g(): sent_item = (yield 1) return [x for x in [sent_item]] This more clearly shows that it is g(), not the comprehension, which is able to yield values (and is thus a generator function). The entire comprehension is consistently in a single scope. The following expressions would, in Python 3.7, raise exceptions immediately. With the removal of the outermost iterable's special casing, they are now equivalent to the most obvious longhand form:: gen = (x for x in rage(10)) # NameError gen = (x for x in 10) # TypeError (not iterable) gen = (x for x in range(1/0)) # ZeroDivisionError def (): for x in rage(10): yield x gen = () # No exception yet tng = next(gen) # NameError Open questions ============== Importing names into comprehensions ----------------------------------- A list comprehension can use and update local names, and they will retain their values from one iteration to another. It would be convenient to use this feature to create rolling or self-effecting data streams:: progressive_sums = [total := total + value for value in data] This will fail with UnboundLocalError due to ``total`` not being initalized. Simply initializing it outside of the comprehension is insufficient - unless the comprehension is in class scope:: class X: total = 0 progressive_sums = [total := total + value for value in data] At other scopes, it may be beneficial to have a way to fetch a value from the surrounding scope. Should this be automatic? Should it be controlled with a keyword? Hypothetically (and using no new keywords), this could be written:: total = 0 progressive_sums = [total := total + value import nonlocal total for value in data] Translated into longhand, this would become:: total = 0 def (total=total): result = [] for value in data: result.append(total := total + value) return result progressive_sums = () ie utilizing the same early-binding technique that is used at class scope. Frequently Raised Objections ============================ Why not just turn existing assignment into an expression? --------------------------------------------------------- C and its derivatives define the ``=`` operator as an expression, rather than a statement as is Python's way. This allows assignments in more contexts, including contexts where comparisons are more common. The syntactic similarity between ``if (x == y)`` and ``if (x = y)`` belies their drastically different semantics. Thus this proposal uses ``:=`` to clarify the distinction. This could be used to create ugly code! --------------------------------------- So can anything else. This is a tool, and it is up to the programmer to use it where it makes sense, and not use it where superior constructs can be used. With assignment expressions, why bother with assignment statements? ------------------------------------------------------------------- The two forms have different flexibilities. The ``:=`` operator can be used inside a larger expression; the ``=`` statement can be augmented to ``+=`` and its friends, can be chained, and can assign to attributes and subscripts. Why not use a sublocal scope and prevent namespace pollution? ------------------------------------------------------------- Previous revisions of this proposal involved sublocal scope (restricted to a single statement), preventing name leakage and namespace pollution. While a definite advantage in a number of situations, this increases complexity in many others, and the costs are not justified by the benefits. In the interests of language simplicity, the name bindings created here are exactly equivalent to any other name bindings, including that usage at class or module scope will create externally-visible names. This is no different from ``for`` loops or other constructs, and can be solved the same way: ``del`` the name once it is no longer needed, or prefix it with an underscore. Names bound within a comprehension are local to that comprehension, even in the outermost iterable, and can thus be used freely without polluting the surrounding namespace. (The author wishes to thank Guido van Rossum and Christoph Groth for their suggestions to move the proposal in this direction. [2]_) Style guide recommendations =========================== As expression assignments can sometimes be used equivalently to statement assignments, the question of which should be preferred will arise. For the benefit of style guides such as PEP 8, two recommendations are suggested. 1. If either assignment statements or assignment expressions can be used, prefer statements; they are a clear declaration of intent. 2. If using assignment expressions would lead to ambiguity about execution order, restructure it to use statements instead. Acknowledgements ================ The author wishes to thank Guido van Rossum and Nick Coghlan for their considerable contributions to this proposal, and to members of the core-mentorship mailing list for assistance with implementation. References ========== .. [1] Proof of concept / reference implementation (https://github.com/Rosuav/cpython/tree/assignment-expressions) .. [2] Pivotal post regarding inline assignment semantics (https://mail.python.org/pipermail/python-ideas/2018-March/049409.html) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From steve at pearwood.info Tue Apr 24 11:34:43 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 25 Apr 2018 01:34:43 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: <20180424153443.GH11616@ando.pearwood.info> On Tue, Apr 24, 2018 at 11:05:57AM -0400, Yury Selivanov wrote: > Well, `my_func(a=(b:=foo))` or `my_func(b:=foo)` are also barely > readable to my eye. There's no advantage to using binding-expressions unless you're going to re-use the name you just defined, and that re-use will give you a hint as to what is happening: my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf)) > My expectation is that users won't use any form > of assignment expressions in function calls, it's painful with both > proposals. If binding-expressions are accepted into the language, I will certainly use them in function calls, *if and when appropriate*. I don't expect it will be common, but I'm sure it will happen. -- Steve From yselivanov.ml at gmail.com Tue Apr 24 11:35:20 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Tue, 24 Apr 2018 11:35:20 -0400 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: On Tue, Apr 24, 2018 at 11:31 AM, Nick Coghlan wrote: > On 25 April 2018 at 00:54, Eric Snow wrote: >> Regardless, your 3 rules would benefit either syntax. Nick may have a >> point that the rules might be an excessive burden, but I don't think >> it's too big a deal since the restrictions are few (and align with the >> most likely usage) and are limited to syntax so the compiler will be >> quick to point mistakes. > > I think the "single name target only" rule should be in place no > matter the syntax for the name binding operator itself. > > I don't mind too much either way on the mandatory parentheses question > (it's certainly an easy option to actively discourage use of binding > expressions as a direct alternative to assignment statements, but as > with the single-name-only rule, it's independent of the choice of > syntax) Mandatory parenthesis around `(name := expr)` would at least solve the problem of users mixing up '=' and ':=' in statements. > > I *do* think the "no name rebinding except in a while loop header" > restriction would be annoying for the if/elif use case and the while > use case: > > while (item = get_item()) is not first_delimiter: > # First processing loop > while (item = get_item()) is not second_delimiter: > # Second processing loop > # etc... > > if (target = get_first_candidate()) is not None: > ... > elif (target = get_second_candidate()) is not None: > ... > elif (target = get_third_candidate()) is not None: > ... Yes, it would force users to come up with better names *iff* they want to use this new sugar: if (first_target = get_first_candidate()) ... elif (second_target = get_second_candidate()) ... Yury From yselivanov.ml at gmail.com Tue Apr 24 11:45:06 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Tue, 24 Apr 2018 11:45:06 -0400 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: <20180424152759.GG11616@ando.pearwood.info> References: <20180424152759.GG11616@ando.pearwood.info> Message-ID: On Tue, Apr 24, 2018 at 11:27 AM, Steven D'Aprano wrote: > On Tue, Apr 24, 2018 at 11:03:35AM -0400, Yury Selivanov wrote: > >> My point was that when you see lots of '=' and ':=' used at the >> statement level, one might try to write "if x = 1" instead of "if x := >> 1" -- boom, we have an unexpected SyntaxError for some users. > > That's a *good* thing. They will then learn not to write x = 1 as an > expression. > > Also, if I write lots of x := 1 binding-expressions as statements, my > code is bad and deserves to fail code-review. But why would I write the > extra colon (one character, two key-presses) to use > > x := 1 > > as a statement, when x = 1 will work? That's a sure sign that I don't > know what I'm doing. (Or that I desperately wish I was writing Pascal.) In JavaScript there's a new backticks syntax for string?their variant of f-strings. I'm seeing a lot of JS coders that use backticks everywhere, regardless if there's formatting in them or not. The result is that some JS code in popular libraries has now *three* different string literal syntaxes separated by one line of code. It looks weird. I expect to see something similar in Python code if we adapt ':='. I don't think the language will benefit from this. FWIW I'm fine with keeping the status quo and not adding new syntax at all. Yury From yselivanov.ml at gmail.com Tue Apr 24 11:46:57 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Tue, 24 Apr 2018 11:46:57 -0400 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: <20180424151528.GF11616@ando.pearwood.info> Message-ID: On Tue, Apr 24, 2018 at 11:28 AM, Chris Angelico wrote: > On re-thinking this, I think the distinction IS possible, but (a) only > in function/class scope, not at global; and (b) would be defined in > terms of lexical position, not run-time. For instance: > > def f(): > (a = 1) # Legal; 'a' has not been used yet > a = 2 # doesn't change that > > def f(a): > (a = 1) # Invalid - 'a' has been used already > > def f(): > while (a = get_next()): # Legal > ... Now *this* is a weird rule. Moving functions around files would become impossible. Please experiment with my reference implementation, it already implements my proposal in full. Loops and inline assignments work as expected in it. Yury From ethan at stoneleaf.us Tue Apr 24 11:51:22 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 24 Apr 2018 08:51:22 -0700 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: <5ADF527A.8000402@stoneleaf.us> On 04/24/2018 08:19 AM, Yury Selivanov wrote: > Yes, because I'm trying to think about this from a pragmatic side of > things. My question to myself: "what syntax could I use that would > prevent me from making '=' vs '==' mistake when I code?" To me, the > answer is that I usually want to compare local variables. I think we need to disambiguate between typo-typos and thinko-typos. I suspect the vast majority of the '=' bugs are not due to the programmer /thinking/ the wrong operation, but of their hands/keyboards not /entering/ the right symbols; having a legal operator ("==") degrade into another legal operator ("=") that looks similar but means incredibly different things is a trap that we should not add to Python. You might say that we have the same problems with ">=", "<=", and "!=". We don't with "!=" because neither "!" nor "=" can stand alone and would fail. We only have it partially with "<=" and ">=" because missing the angle bracket results in failure, but missing the "=" results in a working statement -- but that statement is still the same type of operation and is easier to debug when boundary cases fail. > When I compare to variables from outer scopes they *usually* are on > the *right* side of '=='. You mean something like if 2 == x: ? I never write code like that, and I haven't seen it, either. -- ~Ethan~ From yselivanov.ml at gmail.com Tue Apr 24 11:49:28 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Tue, 24 Apr 2018 11:49:28 -0400 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: <20180424153443.GH11616@ando.pearwood.info> References: <20180424153443.GH11616@ando.pearwood.info> Message-ID: On Tue, Apr 24, 2018 at 11:34 AM, Steven D'Aprano wrote: > On Tue, Apr 24, 2018 at 11:05:57AM -0400, Yury Selivanov wrote: > >> Well, `my_func(a=(b:=foo))` or `my_func(b:=foo)` are also barely >> readable to my eye. > > There's no advantage to using binding-expressions unless you're going to > re-use the name you just defined, and that re-use will give you a hint > as to what is happening: > > my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf)) Again, this is very subjective, but this code would fail my code review :) Don't you find buf = [None] * get_size() my_func(arg, buffer=buf, size=len(buf)) to be more readable? IMHO this example is why we shouldn't implement any form of assignment expressions in Python :) Yury From ethan at stoneleaf.us Tue Apr 24 11:57:02 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 24 Apr 2018 08:57:02 -0700 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: <5ADF53CE.7020501@stoneleaf.us> On 04/24/2018 08:35 AM, Yury Selivanov wrote: > Yes, it would force users to come up with better names *iff* they want > to use this new sugar: > > if (first_target = get_first_candidate()) ... > elif (second_target = get_second_candidate()) ... And then the common code below that only cares about a target being found now has to do a item = first_target or second_target or ... I don't see that as an improvement. -- ~Ethan~ From ncoghlan at gmail.com Tue Apr 24 11:55:36 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 25 Apr 2018 01:55:36 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: On 25 April 2018 at 01:35, Yury Selivanov wrote: > On Tue, Apr 24, 2018 at 11:31 AM, Nick Coghlan wrote: >> I *do* think the "no name rebinding except in a while loop header" >> restriction would be annoying for the if/elif use case and the while >> use case: >> >> while (item = get_item()) is not first_delimiter: >> # First processing loop >> while (item = get_item()) is not second_delimiter: >> # Second processing loop >> # etc... >> >> if (target = get_first_candidate()) is not None: >> ... >> elif (target = get_second_candidate()) is not None: >> ... >> elif (target = get_third_candidate()) is not None: >> ... > > Yes, it would force users to come up with better names *iff* they want > to use this new sugar: > > if (first_target = get_first_candidate()) ... > elif (second_target = get_second_candidate()) ... Sorry, I didn't make the intended nature of that example clear: if (target = get_first_candidate()) is not None: ... # Any setup code specific to this kind of target elif (target = get_second_candidate()) is not None: ... # Any setup code specific to this kind of target elif (target = get_third_candidate()) is not None: ... # Any setup code specific to this kind of target else: raise RuntimeError("No valid candidate found") # Common code using target goes here Using a separate name in each branch wouldn't solve the problem of binding the *same* name for the common code to use later - you'd have to put a "target = candidate_n" bit of boilerplate in each branch. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From yselivanov.ml at gmail.com Tue Apr 24 11:56:17 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Tue, 24 Apr 2018 11:56:17 -0400 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: <5ADF527A.8000402@stoneleaf.us> References: <5ADF527A.8000402@stoneleaf.us> Message-ID: On Tue, Apr 24, 2018 at 11:51 AM, Ethan Furman wrote: >> When I compare to variables from outer scopes they *usually* are on >> the *right* side of '=='. > > > You mean something like > > if 2 == x: > > ? I never write code like that, and I haven't seen it, either. Hm. I mean this: const = 'something' def foo(arg): if arg == const: do something Note that "const" is on the right side of "==". Would you write this as def foo(arg): if const == arg: ? ;) Yury From rosuav at gmail.com Tue Apr 24 11:58:55 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Apr 2018 01:58:55 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: <20180424153443.GH11616@ando.pearwood.info> Message-ID: On Wed, Apr 25, 2018 at 1:49 AM, Yury Selivanov wrote: > On Tue, Apr 24, 2018 at 11:34 AM, Steven D'Aprano wrote: >> On Tue, Apr 24, 2018 at 11:05:57AM -0400, Yury Selivanov wrote: >> >>> Well, `my_func(a=(b:=foo))` or `my_func(b:=foo)` are also barely >>> readable to my eye. >> >> There's no advantage to using binding-expressions unless you're going to >> re-use the name you just defined, and that re-use will give you a hint >> as to what is happening: >> >> my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf)) > > Again, this is very subjective, but this code would fail my code review :) > > Don't you find > > buf = [None] * get_size() > my_func(arg, buffer=buf, size=len(buf)) > > to be more readable? Only if 'buf' is going to be used elsewhere. I'd be looking down below for some other use of 'buf'. Technically the same could be true of the inline assignment, but it makes more sense for a "this statement only" name binding to be within that statement, not broken out and placed above it as another operation at equal importance. ChrisA From ethan at stoneleaf.us Tue Apr 24 12:03:06 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 24 Apr 2018 09:03:06 -0700 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: <5ADF527A.8000402@stoneleaf.us> Message-ID: <5ADF553A.60402@stoneleaf.us> On 04/24/2018 08:56 AM, Yury Selivanov wrote: > On Tue, Apr 24, 2018 at 11:51 AM, Ethan Furman wrote: > >>> When I compare to variables from outer scopes they *usually* are on >>> the *right* side of '=='. >> >> >> You mean something like >> >> if 2 == x: >> >> ? I never write code like that, and I haven't seen it, either. > > Hm. I mean this: > > const = 'something' > > def foo(arg): > if arg == const: > do something > > Note that "const" is on the right side of "==". > > Would you write this as > > def foo(arg): > if const == arg: > > ? ;) Heh, no. But I do write this: def wrapper(func, some_value): value_I_want = process(some_value) def wrapped(*args, **kwds): if value_I_want == 42: ... -- ~Ethan~ From rosuav at gmail.com Tue Apr 24 12:07:24 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Apr 2018 02:07:24 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: <5ADF527A.8000402@stoneleaf.us> Message-ID: On Wed, Apr 25, 2018 at 1:56 AM, Yury Selivanov wrote: > On Tue, Apr 24, 2018 at 11:51 AM, Ethan Furman wrote: > >>> When I compare to variables from outer scopes they *usually* are on >>> the *right* side of '=='. >> >> >> You mean something like >> >> if 2 == x: >> >> ? I never write code like that, and I haven't seen it, either. > > Hm. I mean this: > > const = 'something' > > def foo(arg): > if arg == const: > do something > > Note that "const" is on the right side of "==". > > Would you write this as > > def foo(arg): > if const == arg: > > ? ;) That's assuming the global is a constant. What if it's a mode-setting marker? def foo(arg): ... if output == "verbose" or (output != "quiet" and error_count): print("Arg foo'd", arg) print("Errors found:", error_count) Then I would definitely put the variable first. And I know a lot of people who would parenthesize the first condition in this. ChrisA From yselivanov.ml at gmail.com Tue Apr 24 12:07:37 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Tue, 24 Apr 2018 12:07:37 -0400 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: <20180424153443.GH11616@ando.pearwood.info> Message-ID: On Tue, Apr 24, 2018 at 11:58 AM, Chris Angelico wrote: > On Wed, Apr 25, 2018 at 1:49 AM, Yury Selivanov wrote: >> On Tue, Apr 24, 2018 at 11:34 AM, Steven D'Aprano wrote: [..] >>> There's no advantage to using binding-expressions unless you're going to >>> re-use the name you just defined, and that re-use will give you a hint >>> as to what is happening: >>> >>> my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf)) >> >> Again, this is very subjective, but this code would fail my code review :) >> >> Don't you find >> >> buf = [None] * get_size() >> my_func(arg, buffer=buf, size=len(buf)) >> >> to be more readable? > > Only if 'buf' is going to be used elsewhere. I'd be looking down below > for some other use of 'buf'. Technically the same could be true of the > inline assignment, but it makes more sense for a "this statement only" > name binding to be within that statement, not broken out and placed > above it as another operation at equal importance. Well, you can use empty lines to visually indicate that 'buf' is related to the call. Moreover, 'buf' is still available to the code below that code and sometimes be used there. You can't tell for sure until you glance over the entire file/function. PEP 572 does not implement any sort of sub-scoping. Yury From yselivanov.ml at gmail.com Tue Apr 24 12:11:50 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Tue, 24 Apr 2018 12:11:50 -0400 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: <5ADF553A.60402@stoneleaf.us> References: <5ADF527A.8000402@stoneleaf.us> <5ADF553A.60402@stoneleaf.us> Message-ID: On Tue, Apr 24, 2018 at 12:03 PM, Ethan Furman wrote: [..] > But I do write this: > > def wrapper(func, some_value): > value_I_want = process(some_value) > def wrapped(*args, **kwds): > if value_I_want == 42: > ... But this pattern is more rare than comparing local variables. That's the point I'm trying to use. Besides, to make it an assignment expression under my proposal you would need to use parens. Which makes it even less likely that you confuse '=' and '=='. Yury From steve at pearwood.info Tue Apr 24 12:28:11 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 25 Apr 2018 02:28:11 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: <20180424162811.GJ11616@ando.pearwood.info> On Tue, Apr 24, 2018 at 10:58:24AM -0400, Yury Selivanov wrote: > Since 'diff' and 'g' must be new names according to rule (3), those > who read the code will notice that both were not previously bound. How am I supposed to notice that they've never been bound without carefully reading through the rest of the function in detail, checking every single expression and statement? And besides, you have already established that there are exceptions to the rule "names must be new names". For example, in loops. What other exceptions are there? -- Steve From steve at pearwood.info Tue Apr 24 12:23:46 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 25 Apr 2018 02:23:46 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: <20180424151528.GF11616@ando.pearwood.info> Message-ID: <20180424162346.GI11616@ando.pearwood.info> On Tue, Apr 24, 2018 at 11:25:58AM -0400, Yury Selivanov wrote: > No, it doesn't. The check is performed during compile phase, and > Python does not unroll loops. Anyways, read below. What does unrolling loops have to do with anything? And besides, loop unrolling is an implementation detail -- maybe Python will unroll loops, maybe it won't. If you insist that the check is only done at compile time, then your description is wrong and your rule that "it is *not* allowed to mask names in the current local scope" is false. It *is* allowed to shadow names in the local scope, but only names that cannot be determined at compile-time. from math import * process(arg, (pi = 1), pi+1) # allowed That's more and worse complexity. And what about masking names in the class, nonlocal, global and builtin scopes? Even more complexity and inconsistent behaviour! def function(): global a a = b = 1 process(arg, (a = 2), a+1) # allowed process(arg, (b = 2), b+1) # not allowed > > I believe that one of the most important use-cases for binding- > > expression syntax is while loops, like this modified example taken from > > PEP 572 version 3: > > > > while (data = sock.read()): > > print("Received data:", data) > > > > If you prohibit re-binding data, that prohibits cases like this, or even > > using it inside a loop: > > > > for value in sequence: > > process(arg, (item = expression), item+1) > > No it doesn't. symtable in Python works differently. I encourage you > to test my reference implementation: > > py> for val in [1, 2, 3]: > ... print((item=val), item+1) > ... > 1 2 > 2 3 > 3 4 Then your description is false: the assignment in the second time around the loop is masking the value that was set the first time around the loop. I should be able to unroll the loop by hand, and the code should still work: val = 1 print((item=val), item+1) val = 2 print((item=val), item+1) val = 3 print((item=val), item+1) Does your reference implementation allow that? If not, then you have added yet another inconsistency and obscure rule to be learned: using assignment expressions will break loop unrolling even if you do it by hand. If it *does* allow that, then so much for your claim that you cannot mask existing variables. It can. > > Why is this allowed? > > > > x = 1 # both are statement forms > > x = 2 > > > > but this is prohibited? > > > > x = 1 > > (x = 2) # no rebinding is allowed > > > > and even more confusing, this is allowed! > > > > (x = 1) # x doesn't exist yet, so it is allowed > > x = 2 # statement assignment is allowed to rebind > > These all are very limited code snippets that you're unlikely to see > in real code. Oh come on now Yury, please be reasonable. They're only *sketches* of more realistic code. Of course I'm not actually going to write something like x = 1 (x = 2) but do you really need me to take the time and effort to come up with a more realistic (and therefore complex) example? Okay. # allowed mo = re.match(HEADER_PATTERN, string) if mo: process_header(mo) ... # much later mo = re.match(FOOTER_PATTERN, string) if mo: process_footer(no) # not allowed mo = re.match(HEADER_PATTERN, string) if mo: process_header(mo) ... # much later if (mo = re.match(FOOTER_PATTERN, string)): # SyntaxError process_footer(no) You stated that 'There are no "arcane and confusing rules" about "=", it's rather simple' but every time we look closely at it, the rules seem to get more arcane and confusing. - why is it okay to mask nonlocal, global, class and builtin names, but not local? - for module-level code, how is the compiler supposed to determine the local names in the face of wildcard imports? - why is it a syntax error to assign to a name which is not actually used? # not allowed if "a".upper() == "XYZ"[-1].lower(): spam = "this is dead code and will never happen" process(arg, (spam=expression), spam+1) # syntax error # but this is allowed if "a".upper() == "XYZ"[-1].lower(): spam = "this is dead code and will never happen" spam = expression process(arg, spam, spam+1) - why can you *sometimes* mask existing local variables, if they are used in a loop, but not *other* local variables? - how does this stop *me*, the human reader, from misreading (name=expression) as an equality test? -- Steve From rosuav at gmail.com Tue Apr 24 12:35:57 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Apr 2018 02:35:57 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: <20180424162811.GJ11616@ando.pearwood.info> References: <20180424162811.GJ11616@ando.pearwood.info> Message-ID: On Wed, Apr 25, 2018 at 2:28 AM, Steven D'Aprano wrote: > On Tue, Apr 24, 2018 at 10:58:24AM -0400, Yury Selivanov wrote: > >> Since 'diff' and 'g' must be new names according to rule (3), those >> who read the code will notice that both were not previously bound. > > How am I supposed to notice that they've never been bound without > carefully reading through the rest of the function in detail, checking > every single expression and statement? > > And besides, you have already established that there are exceptions to > the rule "names must be new names". For example, in loops. > > What other exceptions are there? > Yuri is talking about "new" in the syntactic sense. A new name is one which, reading lexically through the code, has not yet been assigned to in the function. (I don't know what happens with global/nonlocal declarations.) Loops have a single point at which the name is assigned to. This has a single point where the name is assigned, too, even though you'll never hit it: def f(x): if x is not x: y = 1 print(y) # UnboundLocalError While I disagree with the proposal, it is at least sane from the compiler's POV. I don't think it makes sense from a human's POV, but it's internally consistent. ChrisA From steve at pearwood.info Tue Apr 24 12:38:03 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 25 Apr 2018 02:38:03 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: <20180424163803.GK11616@ando.pearwood.info> On Tue, Apr 24, 2018 at 11:35:20AM -0400, Yury Selivanov wrote: > Yes, it would force users to come up with better names *iff* they want > to use this new sugar: > > if (first_target = get_first_candidate()) ... > elif (second_target = get_second_candidate()) ... They're not better names. Adding "first_" and "second_" prefixes are just meaningless waffle added to the name "target" to satisfy the compiler so it doesn't complain about reusing the name. And it is a clear inconsistency with = as a statement and = as an expression: # allowed target = get_first_candidate() if target: ... else: target = get_second_candidate() if target: ... # refactor, and we get a syntax error if (target = get_first_candidate()): ... elif (target = get_second_candidate()): ... And I cannot even begin to guess whether this will be allowed or not: if (target = get_first_candidate()): ... while (target = get_second_candidate()): ... -- Steve From tim.peters at gmail.com Tue Apr 24 12:40:42 2018 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 24 Apr 2018 11:40:42 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <20180424120739.43ac2d2e@fsol> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <20180423204428.0aa91a39@fsol> <20180424082751.39708d4f@fsol> <20180424120739.43ac2d2e@fsol> Message-ID: [Antoine] > ... > Yes... I think most will agree that Python is generally easy to take up > for people coming from C++ etc., so my "easier to learn and teach" was > mostly about non-programmers. [Tim] >> even for raw beginners the semantics are the tiniest part of what >> they need to learn anyway about Python's assignment expressions. > I'm not sure what you mean by that. If it's the tiniest part, what's > the overwhelming part? I was hoping it was clear from context that I was talking about "binding expressions", not the PEP's wholly general "assignment expressions". > Is the new assigment expression that delicate to use that it requires > reading a long and intimidating design document ? I didn't get that > impression, so it seems you may be making a stronger point than me > for rejecting the PEP :-) I'm -1 myself on the PEP's assignment expressions, because there are no compelling use cases yet for any but the simplest ("binding expressions") cases. And, yes, understanding Python's assignment statements is challenging. Just understanding their grammar is challenging: assignment_stmt ::= (target_list "=")+ (starred_expression | yield_expression) target_list ::= target ("," target)* [","] target ::= identifier | "(" [target_list] ")" | "[" [target_list] "]" | attributeref | subscription | slicing | "*" target Followed by pages of dense text explaining what all those possibilities mean. A binding expression is more like: binding_expression ::= identifier ":=" expression and the only part of the assignment statement docs needed to explain the meaning is the brief "If the target is an identifier (name)" section, augmented with "and the value of `expression` is the value of the binding expression". If someone has learned what i = 1 means, they already learned almost all of what binding expressions mean too. The target in a binding expression can't be more complicated than the `i` in that example. >>> "await" is a more readable and less confusing improvement >>> over "yield from". >> Heh. Not to me. I have literally have no idea what to with "await" >> (I use generators heavily, but have had no use yet for coroutines), >> but use >> >> yield from an_iterable >> >> routinely. > Yeah... "yield from" is fine for that, except that it was explicitly > meant for the coroutine use case as well (I'm not sure what the > timeline is, but probably Guido was already thinking/dreaming about > tulip/asyncio back then). And trying to shoehorn both in a single > construct made it confusing and inadequate. > > When you want to express two abstractly different concepts (generating > a stream of values, or suspending a task until some asynchronous subtask > finishes), it makes things easier if those two concepts have two > different concrete expressions. Hence "await" making the language > easier to learn for those whose use cases benefit from it. All of which I remain blissfully unaware of :-) ... >> It's simply impossible that, whatever "await" does, it >> could be more readable or less confusing than what I use "yield from" >> for. > Probably because "await" wouldn't work at all for you, then :-) I'm glad people who need "await" got it - they'd have to pry _my_ uses of "yield from" from my cold, dead fingers ;-) Despite that all my uses could be trivially replaced by for _ in an_iterable: yield _ "yield from" saves typing, indentation, and conceptual noise for me. It's the "binding expressions" of nested generators ;-) >>> Format strings dispense from the older, more convoluted formulations. >> But they didn't _replace_ them. That made teaching/learning harder, >> not easier, > Intuitively, it sounds easier to teach f'some {value}' rather than > either the .format() or %-formatting alternatives. The whole goal of > f-strings, after all, is to make string formatting more approachable. > > Learning a language is not learning the whole spec. When you learn C, > you don't need to learn the oddities of pre-ANSI function > declarations :-) A difference is that there still are mountains of code using earlier string formatting methods, and my guess is that there always will be. f-strings aren't always "better". For example, any number of generators (including the combinatoric generators from itertools) yield a sequence of tuples, and format_string % a_tuple is often the simplest way to format the tuple components. Breaking the tuple apart first, whether via explicit indexing in an f-string, or via unpacking into a tuple of names for use in an f-string, is often needless complication. So % formatting needs to be learned by anyone who wants to read _other_ peoples' code. Then again, that's fine by me, because I don't really care whether something new needs to be learned. What I do care about is whether the benefits exceed the costs of learning. > However, assignment a special case in this regard, since traditional > assignment is so omnipresent in online resources, that people _will_ > encounter it even if they make a very focused use of Python. Which means absolutely everyone already understands almost everything about the semantics of binding expressions ;-) >>> Iteration is much simpler than the longer forms we would have to write >> > if generalized iterators didn't exist. >> I'll buy that one. Now go through the HISTORY file and count all the >> changes you didn't name ;-) > You claimed that """almost no addition has ever made a language easier > to learn for raw beginners""". I claim that several additions did > (for Python alone), but I don't need to prove that most of them did ;-) Nor do I need to prove that "almost no" is substantially smaller than "several" ;-) From rosuav at gmail.com Tue Apr 24 12:42:08 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Apr 2018 02:42:08 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: <20180424162346.GI11616@ando.pearwood.info> References: <20180424151528.GF11616@ando.pearwood.info> <20180424162346.GI11616@ando.pearwood.info> Message-ID: On Wed, Apr 25, 2018 at 2:23 AM, Steven D'Aprano wrote: > On Tue, Apr 24, 2018 at 11:25:58AM -0400, Yury Selivanov wrote: > >> No, it doesn't. The check is performed during compile phase, and >> Python does not unroll loops. Anyways, read below. > > What does unrolling loops have to do with anything? And besides, loop > unrolling is an implementation detail -- maybe Python will unroll loops, > maybe it won't. > > If you insist that the check is only done at compile time, then your > description is wrong and your rule that "it is *not* allowed to mask > names in the current local scope" is false. It *is* allowed to shadow > names in the local scope, but only names that cannot be determined at > compile-time. > > from math import * > process(arg, (pi = 1), pi+1) # allowed > > That's more and worse complexity. That's not allowed at local scope (at least, it's not allowed at function scope - is there any other "local scope" at which it is allowed?). Not sure if you can craft equivalent shenanigans with 'exec'. ChrisA From rosuav at gmail.com Tue Apr 24 12:46:39 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Apr 2018 02:46:39 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <20180423204428.0aa91a39@fsol> <20180424082751.39708d4f@fsol> <20180424120739.43ac2d2e@fsol> Message-ID: On Wed, Apr 25, 2018 at 2:40 AM, Tim Peters wrote: > [Antoine] >> ... >> Yes... I think most will agree that Python is generally easy to take up >> for people coming from C++ etc., so my "easier to learn and teach" was >> mostly about non-programmers. > > [Tim] >>> even for raw beginners the semantics are the tiniest part of what >>> they need to learn anyway about Python's assignment expressions. > >> I'm not sure what you mean by that. If it's the tiniest part, what's >> the overwhelming part? > > I was hoping it was clear from context that I was talking about > "binding expressions", not the PEP's wholly general "assignment > expressions". Hopefully you have seen, or soon will see, the latest posting of the PEP, in which assignment targets are restricted to simple names. :) Though I still talk about "assignment expressions". I don't see a problem with calling them that, but I also don't see a problem with calling them "binding expressions" if you prefer. ChrisA From steve at pearwood.info Tue Apr 24 12:57:45 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 25 Apr 2018 02:57:45 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: <20180424151528.GF11616@ando.pearwood.info> <20180424162346.GI11616@ando.pearwood.info> Message-ID: <20180424165745.GL11616@ando.pearwood.info> On Wed, Apr 25, 2018 at 02:42:08AM +1000, Chris Angelico wrote: > > from math import * > > process(arg, (pi = 1), pi+1) # allowed > That's not allowed at local scope (at least, it's not allowed at > function scope - is there any other "local scope" at which it is > allowed?). Of course: local just means the current scope, wherever you happen to be. Inside a function, local is the current function scope. Inside a class body, local is the class body scope. At the top level of the module, local means the module scope (and locals() returns the same dict as globals()). If Yury means for this "cannot mask existing variables" to only operate inside functions, that means that you can mask existing variables if you use assignment expressions in class bodies or top-level module code. -- Steve From tim.peters at gmail.com Tue Apr 24 13:01:18 2018 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 24 Apr 2018 12:01:18 -0500 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <20180423204428.0aa91a39@fsol> <20180424082751.39708d4f@fsol> <20180424120739.43ac2d2e@fsol> Message-ID: [Chris Angelico ] > Hopefully you have seen, or soon will see, the latest posting of the > PEP, in which assignment targets are restricted to simple names. :) I haven't yet, but look forward to it! You have the patience of a saint to endure all this - I would have given up 6 years ago ;-) > Though I still talk about "assignment expressions". I don't see a > problem with calling them that, but I also don't see a problem with > calling them "binding expressions" if you prefer. It's psychology ;-) So long as the PEP calls them assignment expressions, people are going to imagine facing the horrors of things like the current *b, c = a[c] = a assignment statement buried deep inside expressions. But in conventional use, "binding" is restricted to identifiers, which vastly simplifies the mental model for "the worst" that can happen. Since fear is the most potent motivator, "don't scare people" is rule #1 ;-) But, in the absence of Guido chiming in, it's really up to you. A few people have expressed positive feelings about changing the name to "binding expressions", and none opposed it (that I saw), but the sample size is too small to claim that "proves" anything. From vstinner at redhat.com Tue Apr 24 05:21:34 2018 From: vstinner at redhat.com (Victor Stinner) Date: Tue, 24 Apr 2018 11:21:34 +0200 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow Message-ID: Hi, I have been asked to express myself on the PEP 572. I'm not sure that it's useful, but here is my personal opinion on the proposed "assignment expressions". PEP 572 -- Assignment Expressions: https://www.python.org/dev/peps/pep-0572/ First of all, I concur with others: Chris Angelico did a great job to design a good and full PEP, and a working implementation which is also useful to play with it! WARNING! I was (strongly) opposed to PEP 448 Unpacking Generalizations (ex: [1, 2, *list]) and PEP 498 f-string (f"Hello {name}"), whereas I am now a happy user of these new syntaxes. So I'm not sure that I have good tastes :-) Tim Peter gaves the following example. "LONG" version: diff = x - x_base if diff: g = gcd(diff, n) if g > 1: return g versus the "SHORT" version: if (diff := x - x_base) and (g := gcd(diff, n)) > 1: return g == Write == If your job is to write code: the SHORT version can be preferred since it's closer to what you have in mind and the code is shorter. When you read your own code, it seems straightforward and you like to see everything on the same line. The LONG version looks like your expressiveness is limited by the computer. It's like having to use simple words when you talk to a child, because a child is unable to understand more subtle and advanced sentences. You want to write beautiful code for adults, right? == Read and Understand == In my professional experience, I spent most of my time on reading code, rather than writing code. By reading, I mean: try to understand why this specific bug that cannot occur... is always reproduced by the customer, whereas we fail to reproduce it in our test lab :-) This bug is impossible, you know it, right? So let's say that you never read the example before, and it has a bug. By "reading the code", I really mean understanding here. In your opinion, which version is easier to *understand*, without actually running the code? IMHO the LONG version is simpler to understand, since the code is straightforward, it's easy to "guess" the *control flow* (guess in which order instructions will be executed). Print the code on paper and try to draw lines to follow the control flow. It may be easier to understand how SHORT is more complex to understand than LONG. == Debug == Now let's imagine that you can run the code (someone succeeded to reproduce the bug in the test lab!). Since it has a bug, you now likely want to try to understand why the bug occurs using a debugger. Sadly, most debugger are designed as if a single line of code can only execute a single instruction. I tried pdb: you cannot only run (diff := x - x_base) and then get "diff" value, before running the second assingment, you can only execute the *full line* at once. I would say that the LONG version is easier to debug, at least using pdb. I'm using regularly gdb which implements the "step" command as I expect (don't execute the full line, execute sub expressions one by one), but it's still harder to follow the control flow when a single line contains multiple instructions, than debugging lines with a single instruction. You can see it as a limitation of pdb, but many tools only have the granularity of whole line. Think about tracebacks. If you get an exception at "line 1" in the SHORT example (the long "if" expression), what can you deduce from the line number? What happened? If you get an exception in the LONG example, the line number gives you a little bit more information... maybe just enough to understand the bug? Example showing the pdb limitation: >>> def f(): ... breakpoint() ... if (x:=1) and (y:=2): pass ... >>> f() > (3)f() (Pdb) p x *** NameError: name 'x' is not defined (Pdb) p y *** NameError: name 'y' is not defined (Pdb) step --Return-- > (3)f()->None (Pdb) p x 1 (Pdb) p y 2 ... oh, pdb gone too far. I expected a break after "x := 1" and before "y := 2" :-( == Write code for babies! == Please don't write code for yourself, but write code for babies! :-) These babies are going to maintain your code for the next 5 years, while you moved to a different team or project in the meanwhile. Be kind with your coworkers and juniors! I'm trying to write a single instruction per line whenever possible, even if the used language allows me much more complex expressions. Even if the C language allows assignments in if, I avoid them, because I regularly have to debug my own code in gdb ;-) Now the question is which Python are allowed for babies. I recall that a colleague was surprised and confused by context managers. Does it mean that try/finally should be preferred? What about f'Hello {name.title()}' which calls a method into a "string" (formatting)? Or metaclasses? I guess that the limit should depend on your team, and may be explained in the coding style designed by your whole team? Victor From J.Demeyer at UGent.be Tue Apr 24 13:12:34 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Tue, 24 Apr 2018 19:12:34 +0200 Subject: [Python-Dev] PEP 573 -- Module State Access from C Extension Methods In-Reply-To: <5ADF405C.4060607@UGent.be> References: <2854d9cdad73400aa968a084f40908d7@xmail101.UGent.be> <5ADE202B.2060501@UGent.be> <43a667611aa04e60a66372d75c759fa6@xmail101.UGent.be> <5ADEE826.9070806@UGent.be> <9333b45af89e47b4a0b349f284894435@xmail101.UGent.be> <5ADF405C.4060607@UGent.be> Message-ID: <5ADF6582.5080301@UGent.be> On 2018-04-24 16:34, Jeroen Demeyer wrote: > On the other hand, if you are passing the function object, then you can > get __self__ from it (unless it's an unbound method: in that case > __self__ is NULL and self is really args[0]). So there wouldn't be a > need for passing "self". I'm not saying that this is better than passing > "self" explicitly... I haven't yet decided what is best. One thing I realized from PEP 573: the fact that __self__ for built-in functions is set to the module is considered a feature. I never understood the reason for it (and I don't know if the original reason was the same as the reason in PEP 573). If we want to continue supporting that and we also want to support __get__ for built-in functions (to make them act as methods), then there are really two "selfs": there is the "self" from the method (the object that it's bound to) and the "self" from the built-in function (the module). To support that, passing *both* the function and "self" seems like the best way. Jeroen. From J.Demeyer at UGent.be Tue Apr 24 13:19:35 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Tue, 24 Apr 2018 19:19:35 +0200 Subject: [Python-Dev] PEP 575: Unifying function/method classes In-Reply-To: <5AD9BAC4.7000204@UGent.be> References: <5ACF8552.6020607@UGent.be> <5AD1C725.5090603@UGent.be> <3b065fca4bbd416b9d71afab455a4c2c@xmail101.UGent.be> <5AD9BAC4.7000204@UGent.be> Message-ID: <5ADF6727.4040105@UGent.be> On 2018-04-20 12:02, Jeroen Demeyer wrote: > One solution to improve backwards compatibility would be to duplicate > some classes. For example, make a separate class for bound methods in > extension types, which would be literally a duplicate of the existing > types.MethodType class (possibly with a different name). In other words, > a bound method of an extension type would work exactly the same way as > an existing bound method but it would artificially be a different class > for the benefit of non-duck-typing. I elaborated on this: https://www.python.org/dev/peps/pep-0575/#two-phase-implementation From rosuav at gmail.com Tue Apr 24 13:24:10 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Apr 2018 03:24:10 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: <20180424165745.GL11616@ando.pearwood.info> References: <20180424151528.GF11616@ando.pearwood.info> <20180424162346.GI11616@ando.pearwood.info> <20180424165745.GL11616@ando.pearwood.info> Message-ID: On Wed, Apr 25, 2018 at 2:57 AM, Steven D'Aprano wrote: > On Wed, Apr 25, 2018 at 02:42:08AM +1000, Chris Angelico wrote: > >> > from math import * >> > process(arg, (pi = 1), pi+1) # allowed > > >> That's not allowed at local scope (at least, it's not allowed at >> function scope - is there any other "local scope" at which it is >> allowed?). > > Of course: local just means the current scope, wherever you happen to > be. Inside a function, local is the current function scope. Inside a > class body, local is the class body scope. At the top level of the > module, local means the module scope (and locals() returns the same dict > as globals()). > > If Yury means for this "cannot mask existing variables" to only operate > inside functions, that means that you can mask existing variables if you > use assignment expressions in class bodies or top-level module code. I don't have a quote for it, but I was under the impression that this shielding was indeed function-scope-only. Actually, now that I think about it, I'm not sure whether Yuri's plan for assignment expressions even included module scope. ChrisA From python at mrabarnett.plus.com Tue Apr 24 14:10:36 2018 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 24 Apr 2018 19:10:36 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> <87h8o5x36p.fsf@grothesque.org> <70f57020-7d99-a401-d273-00587422256e@mrabarnett.plus.com> Message-ID: <2e543bd2-ba07-f7a4-ee46-735e28ff0ca7@mrabarnett.plus.com> On 2018-04-21 03:15, Tim Peters wrote: > [Tim] > >> And I'll take this opportunity to repeat the key point for me: I > >> tried hard, but never found a single case based on staring at real > >> code where allowing _fancier_ (than "plain name") targets would be a > >> real improvement. In every case I thought it _might_ help, it turned > >> out that it really didn't unless Python _also_ grew an analog to C's > >> "comma operator" (take only the last result from a sequence of > >> expressions). I'll also note that I asked if anyone else had a > >> real-life example, and got no responses. > > [MRAB ] > > Could a semicolon in a parenthesised expression be an equivalent to C's > > "comma operator"? > > I expect it could, but I it's been many years since I tried hacking > Python's grammar, and I wouldn't want a comma operator anyway ;-) [snip] Just reading this: https://www.bfilipek.com/2018/04/refactoring-with-c17-stdoptional.html about C++17, and what did I see? An example with a semicolon in parentheses! From barry at python.org Tue Apr 24 14:39:13 2018 From: barry at python.org (Barry Warsaw) Date: Tue, 24 Apr 2018 11:39:13 -0700 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: <20180424155531.0edd962a@fsol> References: <20180424155531.0edd962a@fsol> Message-ID: <702AFE4A-6326-40AE-B54D-C0C8FAA8AB6E@python.org> On Apr 24, 2018, at 06:55, Antoine Pitrou wrote: > > If the ambition is to find a piece of syntax that reads as "binds", > then we can use a variation on the FLUFL operator: "<->". FLUFL Override Approved! -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From skip.montanaro at gmail.com Tue Apr 24 15:00:43 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 24 Apr 2018 19:00:43 +0000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <2e543bd2-ba07-f7a4-ee46-735e28ff0ca7@mrabarnett.plus.com> References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> <87h8o5x36p.fsf@grothesque.org> <70f57020-7d99-a401-d273-00587422256e@mrabarnett.plus.com> <2e543bd2-ba07-f7a4-ee46-735e28ff0ca7@mrabarnett.plus.com> Message-ID: > > Just reading this: > > https://www.bfilipek.com/2018/04/refactoring-with-c17-stdoptional.html > > about C++17, and what did I see? An example with a semicolon in > parentheses! > Isn't that kind of the C++ motto? "Leave no syntactic stone unturned." Whereas in Python, the syntax motto might better be stated, "We will ship no syntax before its time." (With apologies to Ernest and Julio Gallo.) Skip -------------- next part -------------- An HTML attachment was scrubbed... URL: From python-dev at mgmiller.net Tue Apr 24 15:54:14 2018 From: python-dev at mgmiller.net (Mike Miller) Date: Tue, 24 Apr 2018 12:54:14 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE3B91.5090804@stoneleaf.us> <12E5DDE7-30E7-4870-83BC-AD772661962B@python.org> Message-ID: <14b34909-02ba-433e-ccfb-8657b058eca6@mgmiller.net> On 2018-04-23 15:36, Guido van Rossum wrote: > - the semantics are subtly different from all other uses of 'as' in Python; I'd > like to reserve 'as' for "not a plain assignment" It's conceptually similar to "import as", no? The keyword "as" is a fuzzy yet intuitive concept already. Also, these "binding expressions" are not exactly plain assignment, as it has been known. -Mike From rosuav at gmail.com Tue Apr 24 15:59:08 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Apr 2018 05:59:08 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <14b34909-02ba-433e-ccfb-8657b058eca6@mgmiller.net> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE3B91.5090804@stoneleaf.us> <12E5DDE7-30E7-4870-83BC-AD772661962B@python.org> <14b34909-02ba-433e-ccfb-8657b058eca6@mgmiller.net> Message-ID: On Wed, Apr 25, 2018 at 5:54 AM, Mike Miller wrote: > Also, these "binding expressions" are not exactly plain assignment, as it > has been known. Aside from the restriction to binding only to a simple name, and the fact that they're also an expression with the same value, how are they not the same as plain assignment? Any discrepancies should be considered bugs. ChrisA From ethan at stoneleaf.us Tue Apr 24 16:19:04 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 24 Apr 2018 13:19:04 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE3B91.5090804@stoneleaf.us> <12E5DDE7-30E7-4870-83BC-AD772661962B@python.org> <14b34909-02ba-433e-ccfb-8657b058eca6@mgmiller.net> Message-ID: <5ADF9138.9030403@stoneleaf.us> On 04/24/2018 12:59 PM, Chris Angelico wrote: > On Wed, Apr 25, 2018 at 5:54 AM, Mike Miller wrote: >> Also, these "binding expressions" are not exactly plain assignment, as it >> has been known. > > Aside from the restriction to binding only to a simple name, and the > fact that they're also an expression with the same value, how are they > not the same as plain assignment? Any discrepancies should be > considered bugs. They only bind to simple names, and they can occur in expressions. ;) -- ~Ethan~ From python-dev at mgmiller.net Tue Apr 24 16:18:57 2018 From: python-dev at mgmiller.net (Mike Miller) Date: Tue, 24 Apr 2018 13:18:57 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <855A96BF-144C-4A80-9050-9323557B2D57@python.org> References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE3B91.5090804@stoneleaf.us> <855A96BF-144C-4A80-9050-9323557B2D57@python.org> Message-ID: <85312b72-efe4-5ca6-c984-417eef5c85d0@mgmiller.net> On 2018-04-23 14:19, Barry Warsaw wrote: > Me too. Plus we *already* have precedence for spelling name bindings in similar constructs, such as import statements, with statements, and exceptions. It seems like a natural and Pythonic approach to extend that same spelling to binding expressions rather than introducing new, weird, symbols. We've survived for decades without this syntax, so don't think the need is so great that we need to rush it. Let's not "jump the shark." :D In my opinion, "EXPR as NAME" is the only version worthy enough for the readability of Python. It reads like psuedo-code, as advocates have described in the past, and already used frequently in a fuzzy, non-dogmatic manner. The point about the potential of a bug in the "with" statement is worth serious consideration however. I don't have a problem with it, but if deemed intolerable there was an additional solution I read upthread, believe by Kirill and Steve. Merely that, since all current use cases are in the if/while/comprehension statements, it might be a good idea to limit binding-expressions there to avoid issues/overuse elsewhere. Well, there are a number of mitigations to the "with" issue that could be considered. (Hoping that this is my last post on the subject.) -Mike From python-dev at mgmiller.net Tue Apr 24 16:23:31 2018 From: python-dev at mgmiller.net (Mike Miller) Date: Tue, 24 Apr 2018 13:23:31 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE3B91.5090804@stoneleaf.us> <12E5DDE7-30E7-4870-83BC-AD772661962B@python.org> <14b34909-02ba-433e-ccfb-8657b058eca6@mgmiller.net> Message-ID: On 2018-04-24 12:59, Chris Angelico wrote: > Aside from the restriction to binding only to a simple name, and the > fact that they're also an expression with the same value, how are they > not the same as plain assignment? Any discrepancies should be > considered bugs. Slightly different in that they return the value, and are expressions not assignment statements. Now, I agree that's not a difference of much consequence, but the difference from "import as" is not either. -Mike From srkunze at mail.de Tue Apr 24 16:42:35 2018 From: srkunze at mail.de (Sven R. Kunze) Date: Tue, 24 Apr 2018 22:42:35 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> Message-ID: <83e528a7-9758-fcbf-eb91-b6f1763b5e2e@mail.de> On 23.04.2018 23:41, Tim Peters wrote: > Why? "Give the result of an expression a name" is already heavily > used in Python - it's just that the _contexts_ in which it can be done > are very limited now. Which is a good thing IMO. It keeps stuff simple to reason about. At least to me, the objective of an expression is to generate a result, not to introduce new names. YMMV. > After decades, CPython still does nothing of the sort, short of having > eventually made, e.g., "None" and "True" and "False" reserved words so > at least it can optimize uses of those. It knows nothing at all about > which library functions are pure - and there's no code in the > implementation currently capable of exploiting such information even > if it were known. That remains a fantasy in CPython. CPython optimizes on dicts pretty heavily and on a lot of other things as well. Victor, e.g., is pretty prolific here when it comes to optimizing things for the 95% usecases. Maybe, considering pure functions might be another step. > Guido gave better ones, where binding expressions would allow to > collapse arbitrarily deep levels of nesting to just one (if ... elif > ... elif ... elif ...). My example only eliminated a single level of > artificial indentation. But my example did have the advantage of > being taken verbatim from actual, working code ;-) I like the example, but not the solution it proposes. gcd(diff, n) is to me a perfect name, and please don't tell me g is better. ;) To me it seems, that many people consider function_of_parameter a better name than function(parameter). IMO it isn't. I've seen lots of code where people do things like foo_of_bar = bar['foo']. Because they don't want another dict access, or they think it better reflects the concept. But it does not as does not g. Writing gcd(diff, n) twice is not more repeating as is writing foo_of_bar twice. Because gcd is a pure function, gcd(diff, n) is just a synonym/name of the very same number. That it needs to be calced twice, is a detail, like a double dict access. In the end, it's only optimizing what could a be done by a machine much more reliably. Though, people now want to save that extra assignment line (and indentations) via syntax, because they do that kind of optimizing and want to do it by hand for some reason. Just my 2 cents to let you see where I'm coming from. From python-dev at mgmiller.net Tue Apr 24 16:48:18 2018 From: python-dev at mgmiller.net (Mike Miller) Date: Tue, 24 Apr 2018 13:48:18 -0700 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: References: Message-ID: <9969c00c-25ed-5e99-fe70-7fa8fc1512b0@mgmiller.net> On 2018-04-24 02:21, Victor Stinner wrote: > I have been asked to express myself on the PEP 572. I'm not sure that Thanks. Well written and I've had the same thoughts myself. From ethan at stoneleaf.us Tue Apr 24 17:04:35 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 24 Apr 2018 14:04:35 -0700 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: References: Message-ID: <5ADF9BE3.2010702@stoneleaf.us> On 04/24/2018 02:21 AM, Victor Stinner wrote: > WARNING! I was (strongly) opposed to PEP 448 Unpacking Generalizations > (ex: [1, 2, *list]) and PEP 498 f-string (f"Hello {name}"), whereas I > am now a happy user of these new syntaxes. So I'm not sure that I have > good tastes :-) Cool, you're against PEP 572! That means you'll end up liking it!! ;) -- ~Ethan~ From anthony.flury at btinternet.com Tue Apr 24 17:29:11 2018 From: anthony.flury at btinternet.com (Anthony Flury) Date: Tue, 24 Apr 2018 22:29:11 +0100 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: <5ADF527A.8000402@stoneleaf.us> <5ADF553A.60402@stoneleaf.us> Message-ID: <743e7a31-b854-c8ab-65d8-f060968adc13@btinternet.com> On 24/04/18 17:11, Yury Selivanov wrote: > On Tue, Apr 24, 2018 at 12:03 PM, Ethan Furman wrote: > [..] >> But I do write this: >> >> def wrapper(func, some_value): >> value_I_want = process(some_value) >> def wrapped(*args, **kwds): >> if value_I_want == 42: >> ... > But this pattern is more rare than comparing local variables. That's > the point I'm trying to use. Besides, to make it an assignment > expression under my proposal you would need to use parens. Which makes > it even less likely that you confuse '=' and '=='. Just because you wrap a set of character in parens doesn't mean that you wont potentially mistype what you should type inside the parens. The failure mode of in C : ??? if (a = 3) ??? ??? do_something_with_a(a); Is Incredibly common even with very experienced developers - so much so that most linters flag it as a likely error, and I think gcc has an option to flag it as a warning - even though it is valid and very occasionally it is useful. Also many developers who come to Python from languages such as C will still place parens around conditionals - this means that a typo which will cause a Syntax Error in current versions, but would cause a potentially subtle bug under your implementation (unless you maintain the rule that you can't rebind currently bound names - which renders the whole idea useless in loops (as already discussed at length). I also still can't think of a single other Python construct where the semantics of an operator are explicitly modified by syntaxtic elements outside the operator. For mathematical operators, the surrounding parens modifies the grouping of the operators but not the semantics (* means *, it is just the operands which potentially change). You could argue that your proposal overloads the semantics of the parens (similar to how braces are overloaded to implement dictionaries and set literals), but I don't think that overloading the semantics of parens is good idea. If Python is going to do assignment expressions we shouldn't overload parens in my opinion - we should have a separate operator - doing this avoids needing to exclude rebinding, and makes such expressions considerably more useful. -- Anthony Flury email : *Anthony.flury at btinternet.com* Twitter : *@TonyFlury * From rosuav at gmail.com Tue Apr 24 18:01:14 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Apr 2018 08:01:14 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: <743e7a31-b854-c8ab-65d8-f060968adc13@btinternet.com> References: <5ADF527A.8000402@stoneleaf.us> <5ADF553A.60402@stoneleaf.us> <743e7a31-b854-c8ab-65d8-f060968adc13@btinternet.com> Message-ID: On Wed, Apr 25, 2018 at 7:29 AM, Anthony Flury via Python-Dev wrote: > On 24/04/18 17:11, Yury Selivanov wrote: >> >> On Tue, Apr 24, 2018 at 12:03 PM, Ethan Furman wrote: >> [..] >>> >>> But I do write this: >>> >>> def wrapper(func, some_value): >>> value_I_want = process(some_value) >>> def wrapped(*args, **kwds): >>> if value_I_want == 42: >>> ... >> >> But this pattern is more rare than comparing local variables. That's >> the point I'm trying to use. Besides, to make it an assignment >> expression under my proposal you would need to use parens. Which makes >> it even less likely that you confuse '=' and '=='. > > > Just because you wrap a set of character in parens doesn't mean that you > wont potentially mistype what you should type inside the parens. The failure > mode of in C : > > if (a = 3) > do_something_with_a(a); > > Is Incredibly common even with very experienced developers - so much so that > most linters flag it as a likely error, and I think gcc has an option to > flag it as a warning - even though it is valid and very occasionally it is > useful. Technically what you have there trips *two* dodgy conditions, and could produce either warning or both: 1) Potential assignment where you meant comparison 2) Condition is always true. The combination makes it extremely likely that this wasn't intended. It's more dangerous, though, when the RHS is a function call... > Also many developers who come to Python from languages such as C will still > place parens around conditionals - this means that a typo which will cause a > Syntax Error in current versions, but would cause a potentially subtle bug > under your implementation (unless you maintain the rule that you can't > rebind currently bound names - which renders the whole idea useless in loops > (as already discussed at length). Yuri, look how many of the python-dev readers have completely misinterpreted this "can't rebind" rule. I think that's a fairly clear indication that the rule is not going to be well understood by anyone who isn't extremely familiar with the way the language is parsed. I hesitate to say outright that it is a *bad rule*, but that's the lines I'm thinking along. ChrisA From solipsis at pitrou.net Tue Apr 24 18:16:04 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 25 Apr 2018 00:16:04 +0200 Subject: [Python-Dev] assignment expressions: an alternative alternative proposal References: Message-ID: <20180425001604.228e8873@fsol> On Tue, 24 Apr 2018 09:38:33 -0400 Yury Selivanov wrote: > I propose to use the following syntax for assignment expressions: > > ( NAME = expr ) > > I know that it was proposed before and this idea was rejected, because > accidentally using '=' in place of '==' is a pain point in > C/C++/JavaScript. To solve this issue, I would suggest another syntax: var NAME = expr Strong points: - the "var" keyword makes it clear that it's not a mistyped equality ("var NAME == expr" would be a syntax error) - the "var" keyword can stand out thanks to syntax highlighting - the "=" which traditionally spells assignement is there as well Weak points: - we need a deprecation cycle before "var" can be used as a keyword (alternative keyword choices against "var": "using", "let", "bind"...) Regards Antoine. From guido at python.org Tue Apr 24 18:54:30 2018 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Apr 2018 15:54:30 -0700 Subject: [Python-Dev] assignment expressions: an alternative alternative proposal In-Reply-To: <20180425001604.228e8873@fsol> References: <20180425001604.228e8873@fsol> Message-ID: We should really take this back to python-ideas at this point. On Tue, Apr 24, 2018 at 3:16 PM, Antoine Pitrou wrote: > On Tue, 24 Apr 2018 09:38:33 -0400 > Yury Selivanov wrote: > > I propose to use the following syntax for assignment expressions: > > > > ( NAME = expr ) > > > > I know that it was proposed before and this idea was rejected, because > > accidentally using '=' in place of '==' is a pain point in > > C/C++/JavaScript. > > To solve this issue, I would suggest another syntax: > > var NAME = expr > > Strong points: > - the "var" keyword makes it clear that it's not a mistyped equality > ("var NAME == expr" would be a syntax error) > - the "var" keyword can stand out thanks to syntax highlighting > - the "=" which traditionally spells assignement is there as well > > Weak points: > - we need a deprecation cycle before "var" can be used as a keyword > > (alternative keyword choices against "var": "using", "let", "bind"...) > > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue Apr 24 19:19:11 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 25 Apr 2018 09:19:11 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <83e528a7-9758-fcbf-eb91-b6f1763b5e2e@mail.de> References: <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <83e528a7-9758-fcbf-eb91-b6f1763b5e2e@mail.de> Message-ID: <20180424231911.GN11616@ando.pearwood.info> On Tue, Apr 24, 2018 at 10:42:35PM +0200, Sven R. Kunze wrote: > gcd(diff, n) is to me a perfect name, and please don't tell me g is > better. ;) Sorry, gcd(diff, n) is not the "perfect name", and I will tell you that sometimes g is better. Which would you prefer to see and read? g = gcd(diff, n) result = [g, g+2, 3*g, g**5] or: result = [gcd(diff, n), gcd(diff, n)+2, 3*gcd(diff, n), gcd(diff, n)**5] Forget about the inefficiency of calculating the same result over and over again. Just think about reading the code, especially aloud. If you were describing the code to a colleague, would you really repeat the phrase "gcd of diff, n" *four* separate times? I don't know about you, but I wouldn't, and I wouldn't want to read it four separate times. Think about having to edit it. Would you rather edit it in one place or four? Even close together, in a single expression, it is annoying to have "gcd(diff, n)" repeated over and over again. > To me it seems, that many people consider function_of_parameter a better > name than function(parameter). IMO it isn't. Of course not. But in some contexts, p is a much better name than function(parameter). In other contexts, something more descriptive will be a better name: page_count = (len(document.chapter[previous_chapter].pages()) + len(document.chapter[this_chapter].pages())) page_count is a MUCH better name than repeating (len(document.chapter[previous_chapter].pages()) + len(document.chapter[this_chapter].pages())) over and over again, even if it is a pure function. > I've seen lots of code where people do things like foo_of_bar = > bar['foo']. Because they don't want another dict access, or they think > it better reflects the concept. But it does not as does not g. Writing > gcd(diff, n) twice is not more repeating as is writing foo_of_bar twice. > Because gcd is a pure function You might know that, but how does somebody reading the code know which functions are pure and which are not? How does the compiler know? It's easy to say that you recognise gcd as a pure function. How about len(document.chapter[this_chapter].pages()), is that a pure function? -- Steve From steve at pearwood.info Tue Apr 24 19:35:26 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 25 Apr 2018 09:35:26 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: <743e7a31-b854-c8ab-65d8-f060968adc13@btinternet.com> References: <5ADF527A.8000402@stoneleaf.us> <5ADF553A.60402@stoneleaf.us> <743e7a31-b854-c8ab-65d8-f060968adc13@btinternet.com> Message-ID: <20180424233526.GO11616@ando.pearwood.info> On Tue, Apr 24, 2018 at 10:29:11PM +0100, Anthony Flury via Python-Dev wrote: > If Python is going to do assignment expressions we shouldn't overload > parens in my opinion - we should have a separate operator - doing this > avoids needing to exclude rebinding, and makes such expressions > considerably more useful. Exactly! Yury's suggestion to prohibit rebinding existing names is throwing away the baby with the bathwater. Using assignment-expression to over-write an existing variable is only a error when it is a mistake, done by accident. To satisfy the compiler, we have to invent new and unique names for what is conceptually the same variable being reused. Instead of simplifying your code, it will make it more complex. Despite his statement that there's only one assignment, there are actually two: assignment that allows rebinding, and assignment that sometimes doesn't. The semantic differences between Yury's = and = are actually *greater* than the differences between = and := -- Steve From cs at cskk.id.au Tue Apr 24 18:32:27 2018 From: cs at cskk.id.au (Cameron Simpson) Date: Wed, 25 Apr 2018 08:32:27 +1000 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: <5ADF527A.8000402@stoneleaf.us> References: <5ADF527A.8000402@stoneleaf.us> Message-ID: <20180424223227.GA58477@cskk.homeip.net> On 24Apr2018 08:51, Ethan Furman wrote: >>When I compare to variables from outer scopes they *usually* are on >>the *right* side of '=='. > >You mean something like > > if 2 == x: > >? I never write code like that, and I haven't seen it, either. Just to this, I also never write code like that but I've certainly seen it advocated. I think the rationale was that it places the comparison value foremost in one's mind, versus the name being tested. I'm not persuaded, but it is another subjective situation. Cheers, Cameron Simpson From python at mrabarnett.plus.com Tue Apr 24 21:01:02 2018 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 25 Apr 2018 02:01:02 +0100 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: <20180424223227.GA58477@cskk.homeip.net> References: <5ADF527A.8000402@stoneleaf.us> <20180424223227.GA58477@cskk.homeip.net> Message-ID: <865814c5-0e1e-7a55-ad8e-9c870af789cf@mrabarnett.plus.com> On 2018-04-24 23:32, Cameron Simpson wrote: > On 24Apr2018 08:51, Ethan Furman wrote: >>>When I compare to variables from outer scopes they *usually* are on >>>the *right* side of '=='. >> >>You mean something like >> >> if 2 == x: >> >>? I never write code like that, and I haven't seen it, either. > > Just to this, I also never write code like that but I've certainly seen it > advocated. > > I think the rationale was that it places the comparison value foremost in one's > mind, versus the name being tested. I'm not persuaded, but it is another > subjective situation. > It's sometimes advocated in C/C++ code to help catch the inadvertent use of = instead of ==, but that's not a problem in Python. From tim.peters at gmail.com Tue Apr 24 21:10:49 2018 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 24 Apr 2018 20:10:49 -0500 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow Message-ID: [Victor Stinner] ... > Tim Peter gaves the following example. "LONG" version: > > diff = x - x_base > if diff: > g = gcd(diff, n) > if g > 1: > return g > > versus the "SHORT" version: > > if (diff := x - x_base) and (g := gcd(diff, n)) > 1: > return g > > == Write == > > If your job is to write code: the SHORT version can be preferred since > it's closer to what you have in mind and the code is shorter. When you > read your own code, it seems straightforward and you like to see > everything on the same line. All so, but a bit more: in context, this is just one block in a complex algorithm. The amount of _vertical_ screen space it consumes directly affects how much of what comes before and after it can be seen without scrolling. Understanding this one block in isolation is approximately useless unless you can also see how it fits into the whole. Saving 3 lines of 5 is substantial, but it's more often saving 1 of 5 or 6. Regardless, they add up. > The LONG version looks like your expressiveness is limited by the > computer. It's like having to use simple words when you talk to a > child, because a child is unable to understand more subtle and > advanced sentences. You want to write beautiful code for adults, > right? I want _the whole_ to be as transparent as possible. That's a complicated balancing act in practice. > == Read and Understand == > > In my professional experience, I spent most of my time on reading > code, rather than writing code. By reading, I mean: try to understand > why this specific bug that cannot occur... is always reproduced by the > customer, whereas we fail to reproduce it in our test lab :-) This bug > is impossible, you know it, right? > > So let's say that you never read the example before, and it has a bug. Then you're screwed - pay me to fix it ;-) Seriously, as above, this block on its own is senseless without understanding both the mathematics behind what it's doing, and on how all the code before it picked `x` and `x_base` to begin with. > By "reading the code", I really mean understanding here. In your > opinion, which version is easier to *understand*, without actually > running the code? Honestly, I find the shorter version a bit easier to understand: fewer indentation levels, and less semantically empty repetition of names. > IMHO the LONG version is simpler to understand, since the code is > straightforward, it's easy to "guess" the *control flow* (guess in > which order instructions will be executed). You're saying you don't know that in "x and y" Python evaluates x first, and only evaluates y if x "is truthy"? Sorry, but this seems trivial to me in either spelling. > Print the code on paper and try to draw lines to follow the control > flow. It may be easier to understand how SHORT is more complex to > understand than LONG. Since they're semantically identical, there's _something_ suspect about a conclusion that one is _necessarily_ harder to understand than the other ;-) I don't have a problem with you finding the longer version easier to understand, but I do have a problem if you have a problem with me finding the shorter easier. > == Debug == > > Now let's imagine that you can run the code (someone succeeded to > reproduce the bug in the test lab!). Since it has a bug, you now > likely want to try to understand why the bug occurs using a debugger. > > Sadly, most debugger are designed as if a single line of code can only > execute a single instruction. I tried pdb: you cannot only run (diff > := x - x_base) and then get "diff" value, before running the second > assingment, you can only execute the *full line* at once. > > I would say that the LONG version is easier to debug, at least using pdb. That might be a good reason to avoid, say, list comprehensions (highly complex expressions of just about any kind), but I think this overlooks the primary _point_ of "binding expressions": to give names to intermediate results. I couldn't care less if pdb executes the whole "if" statement in one gulp, because I get exactly the same info either way: the names `diff` and `g` bound to the results of the expressions they named. What actual difference does it make whether pdb binds the names one at a time, or both, before it returns to the prompt? Binding expressions are debugger-friendly in that they _don't_ just vanish without a trace. It's their purpose to _capture_ the values of the expressions they name. Indeed, you may want to add them all over the place inside expressions, never intending to use the names, just so that you can see otherwise-ephemeral intra-expression results in your debugger ;-) > ... Think about tracebacks. If you get an xception at "line 1" in the > SHORT example (the long "if" expression), what can you deduce > from the line number? What happened? > > If you get an exception in the LONG example, the line number gives you > a little bit more information... maybe just enough to understand the > bug? This one I wholly agree with, in general. In the specific example at hand, it's weak, because there's so little that _could_ raise an exception. For example, if the variables weren't bound to integers, in context the code would have blown up long before reaching this block. Python ints are unbounded, so overflow in "-" or "gcd" aren't possible either. MemoryError is theoretically possible, and in that case it would be good to know whether it happened during "-" or during "gcd()". Good to know, but not really helpful, because either way you ran out of memory :-( > == Write code for babies! == > > Please don't write code for yourself, but write code for babies! :-) > > These babies are going to maintain your code for the next 5 years, > while you moved to a different team or project in the meanwhile. Be > kind with your coworkers and juniors! > > I'm trying to write a single instruction per line whenever possible, > even if the used language allows me much more complex expressions. > Even if the C language allows assignments in if, I avoid them, because > I regularly have to debug my own code in gdb ;-) > > Now the question is which Python are allowed for babies. I recall that > a colleague was surprised and confused by context managers. Does it > mean that try/finally should be preferred? What about f'Hello > {name.title()}' which calls a method into a "string" (formatting)? Or > metaclasses? I guess that the limit should depend on your team, and > may be explained in the coding style designed by your whole team? It's the kind of thing I prefer to leave to team style guides, because consensus will never be reached. In a different recent thread, someone complained about using functions at all, because their names are never wholly accurate, and in any case they hide what's "really" going on. To my eyes, that was an unreasonably extreme "write code for babies" position. If a style guide banned using "and" or "or" in Python "if" or "while" tests, I'd find that less extreme, but also unreasonable. But if a style guide banned functions with more than 50 formal arguments, I'd find that unreasonably tolerant. Luckily, I only have to write code for me now, so am free to pick the perfect compromise in every case ;-) From drsalists at gmail.com Tue Apr 24 22:50:30 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Tue, 24 Apr 2018 19:50:30 -0700 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: References: Message-ID: On Tue, Apr 24, 2018 at 2:21 AM, Victor Stinner wrote: > == Write code for babies! == > > Please don't write code for yourself, but write code for babies! :-) > These babies are going to maintain your code for the next 5 years, > while you moved to a different team or project in the meanwhile. Be > kind with your coworkers and juniors! I don't like the idea of assignment expressions in Python. ?Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.? - Brian Kernighan Of the proposed syntaxes, I dislike identifer := expression less, but I'd still rather not see it added. From ncoghlan at gmail.com Tue Apr 24 23:24:59 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 25 Apr 2018 13:24:59 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <20180423204428.0aa91a39@fsol> <20180424082751.39708d4f@fsol> <20180424120739.43ac2d2e@fsol> Message-ID: On 25 April 2018 at 03:01, Tim Peters wrote: > assignment statement buried deep inside expressions. But in > conventional use, "binding" is restricted to identifiers, which vastly > simplifies the mental model for "the worst" that can happen. [snip] > But, in the absence of Guido chiming in, it's really up to you. A few > people have expressed positive feelings about changing the name to > "binding expressions", and none opposed it (that I saw), but the > sample size is too small to claim that "proves" anything. I go back and forth, as I suspect even if we call them "name binding expressions" in the language reference (which I think would be a good idea), they'll often be referred to colloquially as "assignment expressions". That's probably OK though - list displays and dict displays are regularly referred to as literals, and the meaning remains clear, even though the use of literal is technically inaccurate. I also think it would be good for the PEP to spell out the following semantic invariant for code running in a regular namespace: _rhs = expr assert (target := _rhs) is _rhs and target is _rhs It's the restriction to single names as targets that makes it possible to impose such a strong assertion about what the syntax means. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Tue Apr 24 23:40:49 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 25 Apr 2018 13:40:49 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <5ADE3B91.5090804@stoneleaf.us> <12E5DDE7-30E7-4870-83BC-AD772661962B@python.org> <14b34909-02ba-433e-ccfb-8657b058eca6@mgmiller.net> Message-ID: On 25 April 2018 at 06:23, Mike Miller wrote: > Now, I agree that's not a difference of much consequence, but the difference > from "import as" is not either. Since this example has been brought up by a few folks now, I'll note that "import x.y.z as c" has never been entirely equivalent to a simple assignment, since it *also* avoids the default binding of "x" in the current namespace. These days, it's diverged even further, since we added a fallback to looking for the full "x.y.z" key in sys.modules if any part of the submodule lookup chain gets an AttributeError. The proposed name binding expressions aren't like that - they really are just an ordinary binding of the result of the given expression to the given name. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From steve at pearwood.info Tue Apr 24 23:43:30 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 25 Apr 2018 13:43:30 +1000 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: References: Message-ID: <20180425034330.GP11616@ando.pearwood.info> On Tue, Apr 24, 2018 at 08:10:49PM -0500, Tim Peters wrote: > Binding expressions are debugger-friendly in that they _don't_ just > vanish without a trace. It's their purpose to _capture_ the values of > the expressions they name. Indeed, you may want to add them all over > the place inside expressions, never intending to use the names, just > so that you can see otherwise-ephemeral intra-expression results in > your debugger ;-) That's a fantastic point and I'm surprised nobody has thought of it until now (that I've seen). Chris, if you're still reading this and aren't yet heartedly sick and tired of the PEP *wink* this ought to go in as another motivating point. -- Steve From guido at python.org Tue Apr 24 23:56:16 2018 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Apr 2018 20:56:16 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <20180423204428.0aa91a39@fsol> <20180424082751.39708d4f@fsol> <20180424120739.43ac2d2e@fsol> Message-ID: On Tue, Apr 24, 2018 at 8:24 PM, Nick Coghlan wrote: > I also think it would be good for the PEP to spell out the following > semantic invariant for code running in a regular namespace: > > _rhs = expr > assert (target := _rhs) is _rhs and target is _rhs > > It's the restriction to single names as targets that makes it possible > to impose such a strong assertion about what the syntax means. > Can you elaborate? I don't understand what you mean by this. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.peters at gmail.com Tue Apr 24 23:56:32 2018 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 24 Apr 2018 22:56:32 -0500 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: <20180425034330.GP11616@ando.pearwood.info> References: <20180425034330.GP11616@ando.pearwood.info> Message-ID: [Tim] >> Binding expressions are debugger-friendly in that they _don't_ just >> vanish without a trace. It's their purpose to _capture_ the values of >> the expressions they name. Indeed, you may want to add them all over >> the place inside expressions, never intending to use the names, just >> so that you can see otherwise-ephemeral intra-expression results in >> your debugger ;-) [Steven D'Aprano ] wrote: > That's a fantastic point and I'm surprised nobody has thought of it > until now (that I've seen). > > Chris, if you're still reading this and aren't yet heartedly sick and > tired of the PEP *wink* this ought to go in as another motivating point. You know, I thought I was joking when I wrote that - but after I sent it I realized I wasn't ;-) It would actually be quite convenient, and far less error-prone, to add a binding construct inside a complicated expression for purposes of running under a debugger. The alternative is typing the sub-expression(s) of interest by hand at the debugger prompt, or adding print()s, both of which are prone to introducing typos, or changing results radically due to triggering side effects in the code invoked by the duplicated sub-expression(s). Adding a binding construct wouldn't change anything about how the code worked (apart from possibly clobbering a local name). From njs at pobox.com Wed Apr 25 00:05:27 2018 From: njs at pobox.com (Nathaniel Smith) Date: Tue, 24 Apr 2018 21:05:27 -0700 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: References: <20180425034330.GP11616@ando.pearwood.info> Message-ID: On Tue, Apr 24, 2018 at 8:56 PM, Tim Peters wrote: > It would actually be quite convenient, and far less error-prone, to > add a binding construct inside a complicated expression for purposes > of running under a debugger. The alternative is typing the > sub-expression(s) of interest by hand at the debugger prompt, or > adding print()s, both of which are prone to introducing typos, or > changing results radically due to triggering side effects in the code > invoked by the duplicated sub-expression(s). Adding a binding > construct wouldn't change anything about how the code worked (apart > from possibly clobbering a local name). I thought this was what q was for :-) https://pypi.org/project/q/ -n -- Nathaniel J. Smith -- https://vorpus.org From rosuav at gmail.com Wed Apr 25 00:56:52 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Apr 2018 14:56:52 +1000 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: <20180425034330.GP11616@ando.pearwood.info> References: <20180425034330.GP11616@ando.pearwood.info> Message-ID: On Wed, Apr 25, 2018 at 1:43 PM, Steven D'Aprano wrote: > On Tue, Apr 24, 2018 at 08:10:49PM -0500, Tim Peters wrote: > >> Binding expressions are debugger-friendly in that they _don't_ just >> vanish without a trace. It's their purpose to _capture_ the values of >> the expressions they name. Indeed, you may want to add them all over >> the place inside expressions, never intending to use the names, just >> so that you can see otherwise-ephemeral intra-expression results in >> your debugger ;-) > > That's a fantastic point and I'm surprised nobody has thought of it > until now (that I've seen). > > Chris, if you're still reading this and aren't yet heartedly sick and > tired of the PEP *wink* this ought to go in as another motivating point. > Yes, I'm still reading... but I use pdb approximately zero percent of the time, so I actually have no idea what's useful for single-stepping through Python code. So I'm going to write 'blind' here for a bit; do you reckon this sounds reasonably accurate? -- existing rationale -- Naming the result of an expression is an important part of programming, allowing a descriptive name to be used in place of a longer expression, and permitting reuse. Currently, this feature is available only in statement form, making it unavailable in list comprehensions and other expression contexts. Merely introducing a way to assign as an expression would create bizarre edge cases around comprehensions, though, and to avoid the worst of the confusions, we change the definition of comprehensions, causing some edge cases to be interpreted differently, but maintaining the existing behaviour in the majority of situations. -- end existing rationale, begin new text -- Additionally, naming sub-parts of a large expression can assist an interactive debugger, providing useful display hooks and partial results. Without a way to capture sub-expressions inline, this would require refactoring of the original code; with assignment expressions, this merely requires the insertion of a few ``name :=`` markers. Removing the need to refactor reduces the likelihood that the code be inadvertently changed as part of debugging (a common cause of Heisenbugs), and is easier to dictate to a student or junior programmer. -- end -- ChrisA From ncoghlan at gmail.com Wed Apr 25 01:15:51 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 25 Apr 2018 15:15:51 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <20180423204428.0aa91a39@fsol> <20180424082751.39708d4f@fsol> <20180424120739.43ac2d2e@fsol> Message-ID: On 25 April 2018 at 13:56, Guido van Rossum wrote: > On Tue, Apr 24, 2018 at 8:24 PM, Nick Coghlan wrote: >> >> I also think it would be good for the PEP to spell out the following >> semantic invariant for code running in a regular namespace: >> >> _rhs = expr >> assert (target := _rhs) is _rhs and target is _rhs >> >> It's the restriction to single names as targets that makes it possible >> to impose such a strong assertion about what the syntax means. > > Can you elaborate? I don't understand what you mean by this. The assertion is just spelling out in code form that given the name binding expression "target := expr", then: 1. the result of the expression itself is "expr", exactly as if the name binding was omitted 2. that result is also bound to the name "target" in the current scope The preceding "_rhs = expr" line is included to make the invariant generalise even to expressions that have side effects or can change their output based on mutable system state. Ironically, that may be clearer if I use another assignment statement to break it out as two separate invariants: _rhs = expr _inline_result = (bound_name := _rhs) assert _inline_result is _rhs assert bound_name is _rhs By contrast, the protocols involved in handling more complex assignment targets and in handling augmented assignment mean that it wouldn't be possible to define a similarly simple invariant of what they mean (since the exact runtime behaviour would be both type and target dependent). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From steve at pearwood.info Wed Apr 25 01:16:25 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 25 Apr 2018 15:16:25 +1000 Subject: [Python-Dev] assignment expressions: an alternative alternative proposal In-Reply-To: References: <20180425001604.228e8873@fsol> Message-ID: <20180425051623.GQ11616@ando.pearwood.info> On Tue, Apr 24, 2018 at 03:54:30PM -0700, Guido van Rossum wrote: > We should really take this back to python-ideas at this point. Please no :-( Variants of "let" etc were discussed earlier and didn't seem to get much interest. Although I don't think "var" specifically was suggested before, "let" certainly has been: https://mail.python.org/pipermail/python-ideas/2018-February/049031.html In the same thread, I also independently suggested "let": https://mail.python.org/pipermail/python-ideas/2018-February/048981.html This suggestion didn't seem to get much in the way of support, so much so that in the hundreds of emails for PEP 572 the idea seems to have been completely forgotten until now. Perhaps it ought to be added to the PEP in the rejected syntax section. I don't think there's anything to gain by rehashing this again: whether we choose "let", "var" or "bind", there's no real difference. It's still a new keyword, and it still looks like little else in Python today: while (var spam = expression) and spam in values: ... It's not really awful, but nor is it clearly better than := and the fact that it will require a keyword is a major point against it. I'm speaking for myself, of course, not Chris, but while consensus is of course desirable I think there comes a time where the PEP author is entitled to say "I've heard enough arguments, and I'm not convinced by them, this is my proposal and here it stays". If it were my PEP, at this point I would say that if anyone wants to champion an alternative syntax they can write their own PEP :-) I think this idea is going to be too divisive to expect a consensus to emerge, rather like the ternary if operator. I truly believe we've covered just about everything that needs covering, over many hundreds of emails in multiple threads, and unless somebody puts their hand up to write a competing PEP (or at least says something new beyond what we've already seen many times before) I think it is just about time for a BDFL decision between: 1. Reject the PEP. Not going to happen. 2. Leave the PEP pending. Maybe some time in the future. 3. Accept the PEP using the `name := expression` syntax. My preference would be for 3, even though := isn't my first choice for syntax, it's still a good choice. I think there are sufficient use-cases to justify this feature, including some long standing annoyances such as the dance we have to play with regexes, which only gets more annoying as you add more alternatives and cascading indentation. mo = re.match(pattern1, string) if mo: process(mo) else: mo = re.match(pattern2, string) if mo: handle(mo) else: mo = ... # you get the picture (I've changed my mind from earlier when I said this didn't count as a separate use-case from those already given. I think that cascading regexes is a big enough pain that it should be explicitly listed in the PEP as a motivation.) -- Steve From python-dev at mgmiller.net Wed Apr 25 01:48:42 2018 From: python-dev at mgmiller.net (Mike Miller) Date: Tue, 24 Apr 2018 22:48:42 -0700 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: References: <20180425034330.GP11616@ando.pearwood.info> Message-ID: On 2018-04-24 21:05, Nathaniel Smith wrote: > On Tue, Apr 24, 2018 at 8:56 PM, Tim Peters wrote: >> It would actually be quite convenient, and far less error-prone, to >> add a binding construct inside a complicated expression for purposes >> of running under a debugger. The alternative is typing the A bit like breaking complicated expressions into several lines, with or without assignments for readability. ;-) > I thought this was what q was for :-) > > https://pypi.org/project/q/ Where have you been all my life, q? -Mike From greg.ewing at canterbury.ac.nz Wed Apr 25 01:52:04 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 25 Apr 2018 17:52:04 +1200 Subject: [Python-Dev] assignment expressions: an alternative proposal In-Reply-To: References: Message-ID: <5AE01784.90305@canterbury.ac.nz> Nick Coghlan wrote: > I'd be +0 on an "is=" spelling But "is=' looks like some kind of comparison operator. This seems even more conusing to me. -- Greg From v+python at g.nevcal.com Wed Apr 25 01:19:12 2018 From: v+python at g.nevcal.com (Glenn Linderman) Date: Tue, 24 Apr 2018 22:19:12 -0700 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: References: <20180425034330.GP11616@ando.pearwood.info> Message-ID: <5689eee5-ca18-283d-24ff-639b1549b0c9@g.nevcal.com> On 4/24/2018 8:56 PM, Tim Peters wrote: > The alternative is typing the > sub-expression(s) of interest by hand at the debugger prompt, or > adding print()s, both of which are prone to introducing typos, or > changing results radically due to triggering side effects in the code > invoked by the duplicated sub-expression(s). Adding a binding > construct wouldn't change anything about how the code worked (apart > from possibly clobbering a local name). I've done both subexpression mangling (attempts at duplication) and added print statements and experienced these negative side effects, taking twice as long (or more) as intended to get the debugging information needed. While I'm no core developer, and would have a mild appreciation of avoiding those while True: loops so was generally in favor of this PEP, but not enough to be inpsired to speak up about it, I would frequently benefit from this capability... adding extra binding names, and printing _them_ instead of the duplicated subexpressions. +1 Glenn -------------- next part -------------- An HTML attachment was scrubbed... URL: From njs at pobox.com Wed Apr 25 02:55:16 2018 From: njs at pobox.com (Nathaniel Smith) Date: Tue, 24 Apr 2018 23:55:16 -0700 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: On Tue, Apr 24, 2018 at 8:31 AM, Chris Angelico wrote: > The most notable change since last posting is that the assignment > target is no longer as flexible as with the statement form of > assignment, but is restricted to a simple name. > > Note that the reference implementation has not been updated. I haven't read most of the discussion around this, so my apologies if I say anything that's redundant. But since it seems like this might be starting to converge, I just read through it for the first time, and have a few comments. First, though, let me say that this is a really nice document, and I appreciate the incredible amount of work it takes to write something like this and manage the discussions! Regardless of the final outcome it's definitely a valuable contribution. > Recommended use-cases > ===================== > > Simplifying list comprehensions > ------------------------------- > > A list comprehension can map and filter efficiently by capturing > the condition:: > > results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0] > > Similarly, a subexpression can be reused within the main expression, by > giving it a name on first use:: > > stuff = [[y := f(x), x/y] for x in range(5)] > > # There are a number of less obvious ways to spell this in current > # versions of Python, such as: > > # Inline helper function > stuff = [(lambda y: [y,x/y])(f(x)) for x in range(5)] > > # Extra 'for' loop - potentially could be optimized internally > stuff = [[y, x/y] for x in range(5) for y in [f(x)]] > > # Using a mutable cache object (various forms possible) > c = {} > stuff = [[c.update(y=f(x)) or c['y'], x/c['y']] for x in range(5)] > > In all cases, the name is local to the comprehension; like iteration variables, > it cannot leak out into the surrounding context. These examples all make me very nervous. The order of execution in comprehensions is pretty confusing to start with (right to left, except when it's left to right!). But usually this is fine, because comprehensions are mostly used in a functional/declative-ish style, where the exact order doesn't matter. (As befits their functional language heritage.) But := is a side-effecting operator, so when you start using it here, I suddenly have to become extremely aware of the exact order of execution. Concretely, I find it unnerving that two of these work, and one doesn't: # assignment on the right of usage results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0] # assignment on the left of usage stuff = [[y := f(x), x/y] for x in range(5)] # assignment on the right of usage stuff = [[x/y, y := f(x)] for x in range(5)] I guess this isn't limited to comprehensions either ? I rarely see complex expressions with side-effects embedded in the middle, so I'm actually a bit hazy about the exact order of operations inside Python expressions. I could probably figure it out if necessary, but even in simple cases like f(g(), h()), then do you really know for certain off the top of your head whether g() or h() runs first? Does the average user? With code like f(a := g(), h(a)) this suddenly matters a lot! But comprehensions suffer from a particularly extreme version of this, so it worries me that they're being put forward as the first set of motivating examples. > Capturing condition values > -------------------------- Note to Chris: your examples in this section have gotten their order scrambled; you'll want to fix that :-). And I'm going to reorder them yet again in my reply... > # Reading socket data until an empty string is returned > while data := sock.read(): > print("Received data:", data) I don't find this example very convincing. If it were written: for data in iter(sock.read, b""): ... then that would make it clearer what's happening ("oh right, sock.read uses b"" as a sentinel to indicate EOF). And the fact that this is needed at all is only because sockets are a low-level API with lots of complexity inherited from BSD sockets. If this were a normal python API, it'd just be for data in sock: ... (Hmm, I guess the original example is actually wrong because it should be sock.recv, and recv takes a mandatory argument. To be fair, adding that argument would also make the iter() version uglier, and that argument explains why we can't support 'for data in sock'. But this is still consistent with my argument that working directly with sockets is always going to be a bit awkward... I don't think bits of sugar like this are going to make any substantive difference to how easy it is read or write raw socket code.) > # Proposed syntax > while (command := input("> ")) != "quit": > print("You entered:", command) > > # Equivalent in current Python, not caring about function return value > while input("> ") != "quit": > print("You entered a command.") > > # To capture the return value in current Python demands a four-line > # loop header. > while True: > command = input("> "); > if command == "quit": > break > print("You entered:", command) > > Particularly with the ``while`` loop, this can remove the need to have an > infinite loop, an assignment, and a condition. It also creates a smooth > parallel between a loop which simply uses a function call as its condition, > and one which uses that as its condition but also uses the actual value. I dare you to describe that first version in English :-). I would say: "it reads the next line of input and stores it in 'command'; then checks if it was 'quit', and if so it exits the loop; otherwise, it prints the command". What I find interesting is that the English clauses exactly match the statements in the "expanded" version; that feels about right. Jamming three clauses into one line (that you have to read from the inside out!) feels really cramped. (Plus in a real version of this you'd have some command line parsing to do ? at least stripping off whitespace from the command, probably tokenizing it somehow ? before you could check what the command was, and then you're back to the final version anyway.) > # Capturing regular expression match objects > # See, for instance, Lib/pydoc.py, which uses a multiline spelling > # of this effect > if match := re.search(pat, text): > print("Found:", match.group(0)) Now this is a genuinely compelling example! re match objects are always awkward to work with. But this feels like a very big hammer to make re.match easier to use :-). I wonder if there's anything more focused we could do here? > Special-casing conditional statements > ------------------------------------- > > One of the most popular use-cases is ``if`` and ``while`` statements. Instead > of a more general solution, this proposal enhances the syntax of these two > statements to add a means of capturing the compared value:: > > if re.search(pat, text) as match: > print("Found:", match.group(0)) > > This works beautifully if and ONLY if the desired condition is based on the > truthiness of the captured value. It is thus effective for specific > use-cases (regex matches, socket reads that return `''` when done), and > completely useless in more complicated cases (eg where the condition is > ``f(x) < 0`` and you want to capture the value of ``f(x)``). It also has > no benefit to list comprehensions. > > Advantages: No syntactic ambiguities. Disadvantages: Answers only a fraction > of possible use-cases, even in ``if``/``while`` statements. It does only cover a fraction of possible use-cases, but interestingly, the fraction it covers includes: - two of the three real examples given in the rationale section - exactly the cases that *don't* force you to twist your brain in pretzels thinking about sequential side-effecting control flow in the middle of expressions. However, I do think it'd be kinda confusing if we had: if EXPR as X: while EXPR as X: with EXPR as X: and the first two assign the value of EXPR to X, while the last one does something more subtle. Or maybe it'd be fine? FWIW, -n -- Nathaniel J. Smith -- https://vorpus.org From rosuav at gmail.com Wed Apr 25 04:23:00 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Apr 2018 18:23:00 +1000 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: On Wed, Apr 25, 2018 at 4:55 PM, Nathaniel Smith wrote: > On Tue, Apr 24, 2018 at 8:31 AM, Chris Angelico wrote: >> The most notable change since last posting is that the assignment >> target is no longer as flexible as with the statement form of >> assignment, but is restricted to a simple name. >> >> Note that the reference implementation has not been updated. > > I haven't read most of the discussion around this, so my apologies if > I say anything that's redundant. But since it seems like this might be > starting to converge, I just read through it for the first time, and > have a few comments. > > First, though, let me say that this is a really nice document, and I > appreciate the incredible amount of work it takes to write something > like this and manage the discussions! Regardless of the final outcome > it's definitely a valuable contribution. Thank you, but I'm hoping to do more than just rejected PEPs. (I do have one co-authorship to my name, but that was something Guido started, so it kinda had a bit of an advantage there.) My reputation as champion of death march PEPs is not something I want to continue. :| > Concretely, I find it unnerving that two of these work, and one doesn't: > > # assignment on the right of usage > results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0] > > # assignment on the left of usage > stuff = [[y := f(x), x/y] for x in range(5)] > > # assignment on the right of usage > stuff = [[x/y, y := f(x)] for x in range(5)] Fair point. However, this isn't because of assignment operators, but because of comprehensions. There are two important constructs in Python that have out-of-order evaluation: x if cond else y # evaluates cond before x [result for x in iterable if cond] # evaluates result last target()[0] = value # evaluates target after value ... Amongst our weaponry ... err, I'll come in again. Ahem. Most of Python is evaluated left-to-right, top-to-bottom, just as anyone familiar with a Latin-derived language would expect. There are exceptions, however, and those are generally on the basis that "practicality beats purity". Those exceptions include assignment, the if/else expression, and comprehensions/genexps. But even within those constructs, evaluation is left-to-right as much as it possibly can be. A list comprehension places the target expression first (exception to the LTR rule), but evaluates all its 'for' and 'if' clauses in order. We *already* have some strange cases as a result of this out-of-order evaluation. For instance: reciprocals = [1/x for x in values if x] The 'if x' on the right means we won't get a ZeroDivisionError from the expression on the left. Were this loop to be unrolled, it would look like this: def listcomp(): result = [] for x in values: if x: result.append(1/x) return result reciprocals = listcomp() And you can easily audit the longhand form to confirm that, yes, "if x" comes before "1/x". It's the same with assignment expressions; the only exception to the "left before right" rule is the primary expression being evaluated after all for/if clauses. Unrolling your three examples gives (eliding the function wrappers for simplicity): # assignment on the right of usage for x in input_data: if (y := f(x)) > 0: results.append((x, y, x/y)) # assignment on the left of usage for x in range(5): stuff.append([y := f(x), x/y]) # assignment on the right of usage for x in range(5): stuff.append([x/y, y := f(x)]) Were list comprehensions *wrong* to put the expression first? I'm not sure, but I can't see a better way to write them; at best, you'd end up with something like: [for x in numbers if x % 2: x * x] which introduces its own sources of confusion (though I have to say, it would look pretty clean in the "one for loop, no conditions" case). But whether they're right or wrong, they're what we have, and side effects are already legal, and assignment expressions are just another form of side effect. > I guess this isn't limited to comprehensions either ? I rarely see > complex expressions with side-effects embedded in the middle, so I'm > actually a bit hazy about the exact order of operations inside Python > expressions. I could probably figure it out if necessary, but even in > simple cases like f(g(), h()), then do you really know for certain off > the top of your head whether g() or h() runs first? Does the average > user? With code like f(a := g(), h(a)) this suddenly matters a lot! > But comprehensions suffer from a particularly extreme version of this, > so it worries me that they're being put forward as the first set of > motivating examples. Honestly, I would fully expect that g() is run first, but I know there are more complicated cases. For instance, here are three ways to print "1" followed by "2", and create a dictionary mapping None to None: >>> x={} >>> x[print(2)] = print(1) 1 2 >>> x={print(1): print(2)} 1 2 >>> x={print(2): print(1) for _ in [1]} 1 2 Hmmmmmmmmm. One of these is not like the others... >> Capturing condition values >> -------------------------- > > Note to Chris: your examples in this section have gotten their order > scrambled; you'll want to fix that :-). And I'm going to reorder them > yet again in my reply... > >> # Reading socket data until an empty string is returned >> while data := sock.read(): >> print("Received data:", data) > > I don't find this example very convincing. If it were written: > > for data in iter(sock.read, b""): > ... > > then that would make it clearer what's happening ("oh right, sock.read > uses b"" as a sentinel to indicate EOF). That's assuming that you want an equality comparison, which is the case for the thing you're assuming of it, but that's all. Recommending that people use iter() and a 'for' loop has a couple of consequences: 1) It uses a syntax that's distinctly unobvious. You're achieving the end result, but what exactly does it mean to iterate over sock.read? It doesn't read very cleanly. 2) You encourage the use of == as the one and only comparison. If you have an API that returns None when it's done, and you use "iter(func, None)", you're actually checking if the yielded value == None, not if it is None. Suppose you want to change the check so that there are two termination conditions. Do you stick with iter() and then add the other check inside the loop? Do you rewrite the code back into the four-line loop header with the infinite loop, so you can use "in {x, y}" as your condition? iter() can solve a few problems, but not many of them, and not always correctly. > And the fact that this is > needed at all is only because sockets are a low-level API with lots of > complexity inherited from BSD sockets. If this were a normal python > API, it'd just be > > for data in sock: > ... The complexity they inherit is fundamental to the nature of sockets, so that part isn't going to change. I've simplified it down to a coherent example, in the hopes that it'd make sense that way. Maybe I need a different example; people keep getting hung up on the details of sockets instead of the proposed syntax. >> # Proposed syntax >> while (command := input("> ")) != "quit": >> print("You entered:", command) >> >> # Equivalent in current Python, not caring about function return value >> while input("> ") != "quit": >> print("You entered a command.") >> >> # To capture the return value in current Python demands a four-line >> # loop header. >> while True: >> command = input("> "); >> if command == "quit": >> break >> print("You entered:", command) >> >> Particularly with the ``while`` loop, this can remove the need to have an >> infinite loop, an assignment, and a condition. It also creates a smooth >> parallel between a loop which simply uses a function call as its condition, >> and one which uses that as its condition but also uses the actual value. > > I dare you to describe that first version in English :-). I would say: > "it reads the next line of input and stores it in 'command'; then > checks if it was 'quit', and if so it exits the loop; otherwise, it > prints the command". While the command (from the input function) is not equal to "quit", print out "You entered:" and the command. > (Plus in a real version of this you'd have some command line parsing > to do ? at least stripping off whitespace from the command, probably > tokenizing it somehow ? before you could check what the command was, > and then you're back to the final version anyway.) Not if this is a wire protocol. But again, people keep getting hung up on the specifics of the examples, saying "oh but if you change what you're doing, it won't fit the := operator any more". Well, yeah, of course that's true. >> # Capturing regular expression match objects >> # See, for instance, Lib/pydoc.py, which uses a multiline spelling >> # of this effect >> if match := re.search(pat, text): >> print("Found:", match.group(0)) > > Now this is a genuinely compelling example! re match objects are > always awkward to work with. But this feels like a very big hammer to > make re.match easier to use :-). I wonder if there's anything more > focused we could do here? Rejected proposals: Dedicated syntax for regular expression matching? >> Special-casing conditional statements >> ------------------------------------- >> >> One of the most popular use-cases is ``if`` and ``while`` statements. Instead >> of a more general solution, this proposal enhances the syntax of these two >> statements to add a means of capturing the compared value:: >> >> if re.search(pat, text) as match: >> print("Found:", match.group(0)) >> >> This works beautifully if and ONLY if the desired condition is based on the >> truthiness of the captured value. It is thus effective for specific >> use-cases (regex matches, socket reads that return `''` when done), and >> completely useless in more complicated cases (eg where the condition is >> ``f(x) < 0`` and you want to capture the value of ``f(x)``). It also has >> no benefit to list comprehensions. >> >> Advantages: No syntactic ambiguities. Disadvantages: Answers only a fraction >> of possible use-cases, even in ``if``/``while`` statements. > > It does only cover a fraction of possible use-cases, but > interestingly, the fraction it covers includes: > > - two of the three real examples given in the rationale section > - exactly the cases that *don't* force you to twist your brain in > pretzels thinking about sequential side-effecting control flow in the > middle of expressions. It *only* is able to cover the regex case because the regex API has been deliberately crafted to make this possible. If you need any sort of comparison other than "is the captured object truthy?", this syntax is insufficient. > However, I do think it'd be kinda confusing if we had: > > if EXPR as X: > while EXPR as X: > with EXPR as X: > > and the first two assign the value of EXPR to X, while the last one > does something more subtle. Or maybe it'd be fine? Nope, it definitely would not be fine. This is covered by your opening acknowledgement that you haven't read all the previous posts. :) ChrisA From steve at holdenweb.com Wed Apr 25 05:25:23 2018 From: steve at holdenweb.com (Steve Holden) Date: Wed, 25 Apr 2018 10:25:23 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <20180422011308.GA11616@ando.pearwood.info> <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <20180423204428.0aa91a39@fsol> <20180424082751.39708d4f@fsol> <20180424120739.43ac2d2e@fsol> Message-ID: On Wed, Apr 25, 2018 at 6:15 AM, Nick Coghlan wrote: > On 25 April 2018 at 13:56, Guido van Rossum wrote: > > On Tue, Apr 24, 2018 at 8:24 PM, Nick Coghlan > wrote: > >> > >> I also think it would be good for the PEP to spell out the following > >> semantic invariant for code running in a regular namespace: > >> > >> _rhs = expr > >> assert (target := _rhs) is _rhs and target is _rhs > >> > >> It's the restriction to single names as targets that makes it possible > >> to impose such a strong assertion about what the syntax means. > > > > Can you elaborate? I don't understand what you mean by this. > > The assertion is just spelling out in code form that given the name > binding expression "target := expr", then: > > 1. the result of the expression itself is "expr", exactly as if the > name binding was omitted > 2. that result is also bound to the name "target" in the current scope > > The preceding "_rhs = expr" line is included to make the invariant > generalise even to expressions that have side effects or can change > their output based on mutable system state. > > Ironically, that may be clearer if I use another assignment statement > to break it out as two separate invariants: > > _rhs = expr > _inline_result = (bound_name := _rhs) > assert _inline_result is _rhs > assert bound_name is _rhs > > By contrast, the protocols involved in handling more complex > assignment targets and in handling augmented assignment mean that it > wouldn't be possible to define a similarly simple invariant of what > they mean (since the exact runtime behaviour would be both type and > target dependent). > > ?While it's handy that one _could_ use any valid assignment target, allowing this wouldn't (IMHO) necessarily be a good idea. Binding/assignment expressions fit well into Python's semantics (where bindings copy references rather than data) precisely because names are effectively and straightforwardly shorthand for values at the point of assignment. Restricting the targets in target := expression to simple names would avoid a lot of the tricksiness that less experienced programmers might be tempter to indulge, leading to simpler code without undue restrictions on what can be done. The PEP is currently somewhat confused on naming, since it is entitled "Assignment Expressions" but appears to then exclusively use the name "named expressions" to reference the concept under "Syntax and Semantics" and "assignment expression" elsewhere. I'd prefer the term "name binding expressions," since that implies the stricture that more complex targets are excluded. Whatever is chosen, usage in the PEP should be consistent. regards Steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Wed Apr 25 05:27:46 2018 From: steve at holdenweb.com (Steve Holden) Date: Wed, 25 Apr 2018 10:27:46 +0100 Subject: [Python-Dev] assignment expressions: an alternative alternative proposal In-Reply-To: <20180425051623.GQ11616@ando.pearwood.info> References: <20180425001604.228e8873@fsol> <20180425051623.GQ11616@ando.pearwood.info> Message-ID: On Wed, Apr 25, 2018 at 6:16 AM, Steven D'Aprano wrote: > On Tue, Apr 24, 2018 at 03:54:30PM -0700, Guido van Rossum wrote: > > > We should really take this back to python-ideas at this point. > > Please no :-( > ?+1 ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From v+python at g.nevcal.com Tue Apr 24 22:47:22 2018 From: v+python at g.nevcal.com (Glenn Linderman) Date: Tue, 24 Apr 2018 19:47:22 -0700 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: References: Message-ID: <99fd8190-8551-7c8e-70dc-149e9b0c79ee@g.nevcal.com> On 4/24/2018 6:10 PM, Tim Peters wrote: > Luckily, I only have to write code for me now, so am free to pick the > perfect compromise in every case;-) QOTD !? I'm in the same situation. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Wed Apr 25 06:10:27 2018 From: steve at holdenweb.com (Steve Holden) Date: Wed, 25 Apr 2018 11:10:27 +0100 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: References: <20180425034330.GP11616@ando.pearwood.info> Message-ID: On Wed, Apr 25, 2018 at 4:56 AM, Tim Peters wrote: > [Tim] > >> Binding expressions are debugger-friendly in that they _don't_ just > >> vanish without a trace. It's their purpose to _capture_ the values of > >> the expressions they name. Indeed, you may want to add them all over > >> the place inside expressions, never intending to use the names, just > >> so that you can see otherwise-ephemeral intra-expression results in > >> your debugger ;-) > > > [Steven D'Aprano ] > wrote: > > That's a fantastic point and I'm surprised nobody has thought of it > > until now (that I've seen). > > > > Chris, if you're still reading this and aren't yet heartedly sick and > > tired of the PEP *wink* this ought to go in as another motivating point. > > You know, I thought I was joking when I wrote that - but after I sent > it I realized I wasn't ;-) > > ?You just don't realise how perspicacious you truly are, Tim! ? > It would actually be quite convenient, and far less error-prone, to > add a binding construct inside a complicated expression for purposes > of running under a debugger. The alternative is typing the > sub-expression(s) of interest by hand at the debugger prompt, or > adding print()s, both of which are prone to introducing typos, or > changing results radically due to triggering side effects in the code > invoked by the duplicated sub-expression(s). Adding a binding > construct wouldn't change anything about how the code worked (apart > from possibly clobbering a local name). ?Indeed, in the cases where I currently find myself unwrapping expressions to capture their values in local variables for debugging purposes it would usually be far less intrusive to bind a name to the expression inline, then use the debugger to inspect the value. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Wed Apr 25 07:37:35 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 25 Apr 2018 21:37:35 +1000 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: <20180425113735.GS11616@ando.pearwood.info> On Tue, Apr 24, 2018 at 11:55:16PM -0700, Nathaniel Smith wrote: > These examples all make me very nervous. The order of execution in > comprehensions is pretty confusing to start with (right to left, > except when it's left to right!). I don't think comprehensions are ever right to left. With one small irregularity, execution follows Python's normal order, namely (mostly) left to right except where precedence requires something different, the if operator etc. The one small irregularity is that the values produced by the comprehension are written first, rather than last. That is, given: [expression for item in iterable ...] the initial expression doesn't get evaluated until the loop is entered. But the expression still evaluates in left-to-write order (modulo operator precedence etc) and everything following the first "for" keywords also evaluates in left-to-right order. [...] > Concretely, I find it unnerving that two of these work, and one doesn't: > > # assignment on the right of usage > results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0] This shouldn't be surprising. We place the expression first because that's the most important part of the comprehension, the values it generates. Everything from the "for" onwards is just scaffolding needed to produce those values, its the leading expression that matters. But clearly we can't expect to evaluate the expression *before* the loop, even though we want it written first. So although the comprehension is written expression first, the expression is actually evaluated *last*. results = [ for x in input_data if (y := f(x)) > 0 (x, y, x/y) ] So when reading a comprehension in execution order: - look ahead to the first "for" keyword; - read to the end of the comprehension, using normal left-to-right execution order (modulo the usual exceptions); - jump back to the expression at the start; - read in normal left-to-right execution order. So aside from that small anomaly of the expression coming first, comprehensions use the same order as regular Python code. Binding expressions won't change that. > I could probably figure it out if necessary, but even in > simple cases like f(g(), h()), then do you really know for certain off > the top of your head whether g() or h() runs first? Of course. Left-to-right execution order (modulo the usual...) is a language guarantee. > Does the average user? I dare say the average user probably doesn't even think about it, and merely assumes that everything is left to right until they learn differently. In this case, that reasonable default position is correct. Everything is left to right until you learn differently. > With code like f(a := g(), h(a)) this suddenly matters a lot! That's hardly different from any other precedence issue, and even when precedence is specified by the language, sometimes adding a few extra brackets makes things clearer even when they're not needed: func(arg, (spam*n == token), (a := g()), h(a)) > But comprehensions suffer from a particularly extreme version of this, > so it worries me that they're being put forward as the first set of > motivating examples. Ha, I think that comprehensions are one of the weaker motivating examples, but it was a discussion on Python-Ideas about adding special syntax *only* to comprehensions which lead to Chris writing the PEP. So it is purely an accident of history why comprehensions are the first example in the PEP. > > # Capturing regular expression match objects > > # See, for instance, Lib/pydoc.py, which uses a multiline spelling > > # of this effect > > if match := re.search(pat, text): > > print("Found:", match.group(0)) > > Now this is a genuinely compelling example! re match objects are > always awkward to work with. But this feels like a very big hammer to > make re.match easier to use :-). I wonder if there's anything more > focused we could do here? A reasonable point. [...] > However, I do think it'd be kinda confusing if we had: > > if EXPR as X: > while EXPR as X: > with EXPR as X: > > and the first two assign the value of EXPR to X, while the last one > does something more subtle. Or maybe it'd be fine? It would be fine, right up to the point that it wasn't, and then it would be a brain melting bug magnet that would cause programmers to curse us onto the 20th generation. -- Steve From sf at fermigier.com Wed Apr 25 06:57:31 2018 From: sf at fermigier.com (=?UTF-8?Q?St=C3=A9fane_Fermigier?=) Date: Wed, 25 Apr 2018 12:57:31 +0200 Subject: [Python-Dev] Meta-question about this mailing list Message-ID: Hi, the description on https://mail.python.org/mailman/listinfo/python-dev reads: "On this list the key Python developers discuss the future of the language and its implementation." I interpret this sentence as "only key (core?) developers are supposed to participate in the discussion (ie. post messages), the rest is welcome to lurk." Is this the intended meaning ? There is an additional description in the devguide: "python-dev is the primary mailing list for discussions about Python?s development. The list is open to the public and is subscribed to by all core developers plus many people simply interested in following Python?s development. Discussion is focused on issues related to Python?s development, such as how to handle a specific issue, a PEP, etc." I would interpret it as "it's OK for non core developers to post", except for the "many people simply interested in following Python?s development" part which hints that we are welcome to "follow" but not actively participate. So in either case (and both descriptions) it would IMHO be useful to use clearer language. Another point: later in the Mailman description, one reads: "Consider using Gmane." (with a link to http://dir.gmane.org/gmane.comp.python.devel ). I went to Gmane and found all the links broken. I remember there was some turmoil at Gmane a couple of years back: https://en.wikipedia.org/wiki/Gmane#References => It seems the migration didn't work that well and the service shouldn't been relied upon. Regards, S. -- Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/ Chairman, Free&OSS Group @ Systematic Cluster - http://www.gt-logiciel-libre.org/ Co-Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/ Founder & Organiser, PyParis & PyData Paris - http://pyparis.org/ & http://pydata.fr/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From daveshawley at gmail.com Wed Apr 25 07:03:31 2018 From: daveshawley at gmail.com (David Shawley) Date: Wed, 25 Apr 2018 07:03:31 -0400 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <2e543bd2-ba07-f7a4-ee46-735e28ff0ca7@mrabarnett.plus.com> References: <874lk6bwv2.fsf@grothesque.org> <23257.24268.962140.505078@turnbull.sk.tsukuba.ac.jp> <3b82d4945e3839c687b4d19bc6485ced@grothesque.org> <87h8o5x36p.fsf@grothesque.org> <70f57020-7d99-a401-d273-00587422256e@mrabarnett.plus.com> <2e543bd2-ba07-f7a4-ee46-735e28ff0ca7@mrabarnett.plus.com> Message-ID: On Apr 24, 2018, at 2:10 PM, MRAB wrote: > > On 2018-04-21 03:15, Tim Peters wrote: >> [Tim] >> >> And I'll take this opportunity to repeat the key point for me: I >> >> tried hard, but never found a single case based on staring at real >> >> code where allowing _fancier_ (than "plain name") targets would be a >> >> real improvement. In every case I thought it _might_ help, it turned >> >> out that it really didn't unless Python _also_ grew an analog to C's >> >> "comma operator" (take only the last result from a sequence of >> >> expressions). I'll also note that I asked if anyone else had a >> >> real-life example, and got no responses. >> >> [MRAB ] >> > Could a semicolon in a parenthesised expression be an equivalent to C's >> > "comma operator"? >> >> I expect it could, but I it's been many years since I tried hacking >> Python's grammar, and I wouldn't want a comma operator anyway ;-) > [snip] > Just reading this: > > https://www.bfilipek.com/2018/04/refactoring-with-c17-stdoptional.html > > about C++17, and what did I see? An example with a semicolon in parentheses! A similar pattern shows up in Go's if statement syntax. It is interesting to note that it is part of the grammar specifically for the if statement and *not* general expression syntax. IfStmt = "if" [ SimpleStmt ";" ] Expression Block [ "else" ( IfStmt | Block ) ] . Bindings that occur inside of `SimpleStmt` are only available within the `Expression` and blocks that make up the if statement. https://golang.org/ref/spec#If_statements This isn't a good reason to parrot the syntax in Python though. IMO, I consider the pattern to be one of the distinguishing features of golang and would be happy leaving it there. I have often wondered if adding the venerable for loop syntax from C (and many other languages) would solve some of the needs here though. The for loop syntax in golang is interesting in that it serves as both a standard multipart for statement as well as a while statement. Changing something like this is more of a Python 4 feature and I think that I would be -0 on the concept. I did want to mention the similarities for the posterity though. ChrisA - we might want to add explicit mentions of golang's if statement and for loop as "considered" syntaxes since they are in a sibling programing language (e.g., similar to async/await in PEP 492). - dave -- "Syntactic sugar causes cancer of the semicolon" - Alan Perlis -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Wed Apr 25 11:46:12 2018 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2018 08:46:12 -0700 Subject: [Python-Dev] assignment expressions: an alternative alternative proposal In-Reply-To: References: <20180425001604.228e8873@fsol> <20180425051623.GQ11616@ando.pearwood.info> Message-ID: On Wed, Apr 25, 2018 at 2:27 AM, Steve Holden wrote: > On Wed, Apr 25, 2018 at 6:16 AM, Steven D'Aprano > wrote: > >> On Tue, Apr 24, 2018 at 03:54:30PM -0700, Guido van Rossum wrote: >> >> > We should really take this back to python-ideas at this point. >> >> Please no :-( >> > > ?+1 > Maybe I should have just said "no". Thanks Steven D'A for saving everyone yet another detour on this proposal. I'm pretty excited myself about NAME := and am mostly ignoring the current crop of counter-proposal. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Wed Apr 25 11:51:17 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Apr 2018 01:51:17 +1000 Subject: [Python-Dev] assignment expressions: an alternative alternative proposal In-Reply-To: References: <20180425001604.228e8873@fsol> <20180425051623.GQ11616@ando.pearwood.info> Message-ID: On Thu, Apr 26, 2018 at 1:46 AM, Guido van Rossum wrote: > On Wed, Apr 25, 2018 at 2:27 AM, Steve Holden wrote: >> >> On Wed, Apr 25, 2018 at 6:16 AM, Steven D'Aprano >> wrote: >>> >>> On Tue, Apr 24, 2018 at 03:54:30PM -0700, Guido van Rossum wrote: >>> >>> > We should really take this back to python-ideas at this point. >>> >>> Please no :-( >> >> >> +1 > > > Maybe I should have just said "no". Thanks Steven D'A for saving everyone > yet another detour on this proposal. > > I'm pretty excited myself about NAME := and am mostly ignoring > the current crop of counter-proposal. Of course, if someone wants to start a python-bad-ideas mailing list, I'm sure some of these alternatives would be perfect for it... If anyone hasn't seen the latest iteration of the PEP, I recently re-posted it. And it's always available here: https://www.python.org/dev/peps/pep-0572/ ChrisA From encukou at gmail.com Wed Apr 25 14:33:09 2018 From: encukou at gmail.com (Petr Viktorin) Date: Wed, 25 Apr 2018 14:33:09 -0400 Subject: [Python-Dev] PEP 573 -- Module State Access from C Extension Methods In-Reply-To: <5ADF6582.5080301@UGent.be> References: <2854d9cdad73400aa968a084f40908d7@xmail101.UGent.be> <5ADE202B.2060501@UGent.be> <43a667611aa04e60a66372d75c759fa6@xmail101.UGent.be> <5ADEE826.9070806@UGent.be> <9333b45af89e47b4a0b349f284894435@xmail101.UGent.be> <5ADF405C.4060607@UGent.be> <5ADF6582.5080301@UGent.be> Message-ID: <31838921-8833-0ccd-a533-d5e38d9cd697@gmail.com> On 04/24/18 13:12, Jeroen Demeyer wrote: > On 2018-04-24 16:34, Jeroen Demeyer wrote: >> On the other hand, if you are passing the function object, then you can >> get __self__ from it (unless it's an unbound method: in that case >> __self__ is NULL and self is really args[0]). So there wouldn't be a >> need for passing "self". I'm not saying that this is better than passing >> "self" explicitly... I haven't yet decided what is best. > > One thing I realized from PEP 573: the fact that __self__ for built-in > functions is set to the module is considered a feature. I never > understood the reason for it (and I don't know if the original reason > was the same as the reason in PEP 573). > > If we want to continue supporting that and we also want to support > __get__ for built-in functions (to make them act as methods), then there > are really two "selfs": there is the "self" from the method (the object > that it's bound to) and the "self" from the built-in function (the > module). To support that, passing *both* the function and "self" seems > like the best way. You're talking about functions with METH_BINDING here, right? There the other "self" would be the defining module. It might make sense to pass that also in the struct, rather than as an additional argument. Perhaps "m_objclass" could point to the module in this case, or a new pointer could be added. From J.Demeyer at UGent.be Wed Apr 25 14:46:02 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Wed, 25 Apr 2018 20:46:02 +0200 Subject: [Python-Dev] PEP 573 -- Module State Access from C Extension Methods In-Reply-To: <15262b8f09724d1d801cc81289aa4ba2@xmail101.UGent.be> References: <2854d9cdad73400aa968a084f40908d7@xmail101.UGent.be> <5ADE202B.2060501@UGent.be> <43a667611aa04e60a66372d75c759fa6@xmail101.UGent.be> <5ADEE826.9070806@UGent.be> <9333b45af89e47b4a0b349f284894435@xmail101.UGent.be> <5ADF405C.4060607@UGent.be> <5ADF6582.5080301@UGent.be> <15262b8f09724d1d801cc81289aa4ba2@xmail101.UGent.be> Message-ID: <5AE0CCEA.2030806@UGent.be> On 2018-04-25 20:33, Petr Viktorin wrote: > Perhaps "m_objclass" could point to the module in this case That was exactly my idea also today. Instead of treating m_objclass as the defining class, we should generalize it to be the "parent" of the function: either the class or the module. From encukou at gmail.com Wed Apr 25 15:04:36 2018 From: encukou at gmail.com (Petr Viktorin) Date: Wed, 25 Apr 2018 15:04:36 -0400 Subject: [Python-Dev] PEP 573 -- Module State Access from C Extension Methods In-Reply-To: <5AE0CCEA.2030806@UGent.be> References: <2854d9cdad73400aa968a084f40908d7@xmail101.UGent.be> <5ADE202B.2060501@UGent.be> <43a667611aa04e60a66372d75c759fa6@xmail101.UGent.be> <5ADEE826.9070806@UGent.be> <9333b45af89e47b4a0b349f284894435@xmail101.UGent.be> <5ADF405C.4060607@UGent.be> <5ADF6582.5080301@UGent.be> <15262b8f09724d1d801cc81289aa4ba2@xmail101.UGent.be> <5AE0CCEA.2030806@UGent.be> Message-ID: <029808b4-8944-74d7-3877-93479c227db9@gmail.com> On 04/25/18 14:46, Jeroen Demeyer wrote: > On 2018-04-25 20:33, Petr Viktorin wrote: >> Perhaps "m_objclass" could point to the module in this case > > That was exactly my idea also today. Instead of treating m_objclass as > the defining class, we should generalize it to be the "parent" of the > function: either the class or the module. Great to hear we think alike. However, I think that while reusing the pointer is nice to save space, the two concepts should still be separate, because "defining module" is a reasonable concept even for methods. In particular: - There should be *separate* accessor functions for: - getting the defining class - getting the defining module - The latter would later (in PEP 573) be extended to return the defining module even for class methods (when available) - In Python code, __objclass__ should be the defining class, not the module. - The C field should have a different name (m_parent?), so it isn't that strongly associated with __objclass__. Does that sound reasonable? From encukou at gmail.com Wed Apr 25 15:10:10 2018 From: encukou at gmail.com (Petr Viktorin) Date: Wed, 25 Apr 2018 15:10:10 -0400 Subject: [Python-Dev] PEP 394 update proposal: Allow changing the `python` command in some cases Message-ID: <4c4ec942-e17b-c4f4-4b2f-459fed2c32aa@gmail.com> Hello, In Fedora, I found that PEP 394's strict recommendation that `python` points to `python2` is holding us back. From discussions on Zulip and elsewhere it's clear that this recommendation is not changing any time soon, but I would like to officially relax it in several cases. The problems are: - For developers that are not following the language's development, the fact that `python` invokes `python2` sends a strong signal that 2 is somehow the preferred version, and it's OK to start new projects in it. - Users and sysadmins that *do* want to ?live in the future? are switching the symlink to `python3` themselves. We would like to give them a supported, documented way to do so -- and make surer they're aware of the caveats. - The `python` command is still not available out-of-the box on macOS, so it didn't completely live up to the expectation of being the cross-platform way to launch 2/3 source compatile scripts. - `python` in the shebang line can mean *either* that a script is carefully written to be 2/3 compatible, *or* that the author/maintainer is lazy or unaware of the recommendations. While Fedora guidelines have long banned the unversioned command, we feel that the only way to *enforce* that guidelines is to provide environments where the `python` command does not work (unless explicitly installed). To help solve these, I would like to relax recommendations on the Unix ``python -> python2`` symlink in these cases: - Users and administrators can, by a deliberate action, change ``python`` to invoke Python 3. (Activating a venv counts as such an action, but so would e.g. using alternates, installing a non-default overriding package, or replacing /usr/bin/python.) - In controlled environments where being explicit is valued more than user experience (test environments, build systems, etc.), distributions can omit the `python` command even when `python2` is installed. I have filed these changes as a pull request here: https://github.com/python/peps/pull/630 The PR also spells out several other things, which I felt were hidden between the lines -- but correct me if you disagree with my reading. From lukasz at langa.pl Wed Apr 25 16:21:19 2018 From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=) Date: Wed, 25 Apr 2018 13:21:19 -0700 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 Message-ID: PEP 572 caused a strong emotional reaction in me. I wanted to first understand my intuitive objection to the idea before posting anything. I feel that (name := expression) doesn't fit the narrative of PEP 20. It doesn't remove complexity, it only moves it. What was its own assignment before now is part of the logic test. This saves on vertical whitespace but makes parsing and understanding logic tests harder. This is a bad bargain: logic tests already contain a lot of complexity that human readers have to cope with. Proponents of := argue it makes several patterns flatter (= better than nested) to express. Serial regular expression matching is a popular example. However, (name := expression) itself is making logic tests more nested, not flatter. It makes information in the logic test denser (= worse than sparse). Since it also requires an additional pair of parentheses, it forces the reader to decompose the expression in their head. := also goes against having one obvious way to do it. Since it's an expression, it can also be placed on its own line or in otherwise weird places like function call arguments. I anticipate PEP 8 would have to be extended to explicitly discourage such abuse. Linters would grow rules against it. This is noise. I'm -1 on PEP 572, I think it's very similar in spirit to the rejected PEP 463. -- ? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From rosuav at gmail.com Wed Apr 25 16:24:56 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Apr 2018 06:24:56 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: Message-ID: On Thu, Apr 26, 2018 at 6:21 AM, ?ukasz Langa wrote: > := also goes against having one obvious way to do it. Since it's an expression, > it can also be placed on its own line or in otherwise weird places like > function call arguments. I anticipate PEP 8 would have to be extended to > explicitly discourage such abuse. Linters would grow rules against it. This is > noise. Does this argument also apply to the if/else expression? Do linters need rules to advise against people writing code like: print(x) if x is None else print(y) ? It's perfectly legal to write code like this. But I don't see people abusing this sort of thing. ChrisA From guido at python.org Wed Apr 25 16:28:45 2018 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2018 13:28:45 -0700 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: Message-ID: A very emotional appeal, you don't seem to grasp the usability improvements this will give. I hear you but at this point appeals to Python's "Zen" don't help you. On Wed, Apr 25, 2018 at 1:21 PM, ?ukasz Langa wrote: > PEP 572 caused a strong emotional reaction in me. I wanted to first > understand > my intuitive objection to the idea before posting anything. > > I feel that (name := expression) doesn't fit the narrative of PEP 20. It > doesn't remove complexity, it only moves it. What was its own assignment > before > now is part of the logic test. This saves on vertical whitespace but makes > parsing and understanding logic tests harder. This is a bad bargain: logic > tests already contain a lot of complexity that human readers have to cope > with. > > Proponents of := argue it makes several patterns flatter (= better than > nested) > to express. Serial regular expression matching is a popular example. > However, > (name := expression) itself is making logic tests more nested, not > flatter. It > makes information in the logic test denser (= worse than sparse). Since it > also > requires an additional pair of parentheses, it forces the reader to > decompose > the expression in their head. > > := also goes against having one obvious way to do it. Since it's an > expression, > it can also be placed on its own line or in otherwise weird places like > function call arguments. I anticipate PEP 8 would have to be extended to > explicitly discourage such abuse. Linters would grow rules against it. > This is > noise. > > I'm -1 on PEP 572, I think it's very similar in spirit to the rejected PEP > 463. > > -- ? > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukasz at langa.pl Wed Apr 25 16:55:37 2018 From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=) Date: Wed, 25 Apr 2018 13:55:37 -0700 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: Message-ID: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> > On 25 Apr, 2018, at 1:28 PM, Guido van Rossum wrote: > > You don't seem to grasp the usability improvements this will give. I hear you but at this point appeals to Python's "Zen" don't help you. This reads dismissive to me. I did read the PEP and followed the discussion on python-dev. I referred to PEP 20 because it distills what's unique about the value proposition of Python. It's our shared vocabulary. Can you address the specific criticism I had? To paraphrase it without PEP 20 jargon: > (name := expression) makes code less uniform. It inserts more information > into a place that is already heavily packed with information (logic tests). -- ? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From lukasz at langa.pl Wed Apr 25 17:06:47 2018 From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=) Date: Wed, 25 Apr 2018 14:06:47 -0700 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: Message-ID: > On 25 Apr, 2018, at 1:24 PM, Chris Angelico wrote: > > On Thu, Apr 26, 2018 at 6:21 AM, ?ukasz Langa wrote: >> := also goes against having one obvious way to do it. Since it's an expression, >> it can also be placed on its own line or in otherwise weird places like >> function call arguments. I anticipate PEP 8 would have to be extended to >> explicitly discourage such abuse. Linters would grow rules against it. This is >> noise. > > Does this argument also apply to the if/else expression? Do linters > need rules to advise against people writing code like: > > print(x) if x is None else print(y) > > ? It's perfectly legal to write code like this. But I don't see people > abusing this sort of thing. Ternary expressions are different because their flow is deliberately different from a regular if statement. It's also different from the C equivalent. `:=` on the other hand is deciptively similar to `=`. But yeah, I think worrying about abuse of the feature is a red herring. The gist of my criticism of your PEP is about the decreased balance in information density. -- ? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From tjreedy at udel.edu Wed Apr 25 17:17:50 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 25 Apr 2018 17:17:50 -0400 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: References: <20180425034330.GP11616@ando.pearwood.info> Message-ID: On 4/25/2018 6:10 AM, Steve Holden wrote: > On Wed, Apr 25, 2018 at 4:56 AM, Tim Peters > wrote: > > [Tim] > >> Binding expressions are debugger-friendly in that they _don't_ just > >> vanish without a trace.? It's their purpose to _capture_ the values of > >> the expressions they name.? Indeed, you may want to add them all over > >> the place inside expressions, never intending to use the names, just > >> so that you can see otherwise-ephemeral intra-expression results in > >> your debugger ;-) > > > [Steven D'Aprano >] > ?wrote: > > That's a fantastic point and I'm surprised nobody has thought of it > > until now (that I've seen). > > > > Chris, if you're still reading this and aren't yet heartedly sick and > > tired of the PEP *wink* this ought to go in as another motivating point. > > You know, I thought I was joking when I wrote that - but after I sent > it I realized I wasn't ;-) > > ?You just don't realise how perspicacious you truly are, Tim! > ? > > It would actually be quite convenient, and far less error-prone, to > add a binding construct inside a complicated expression for purposes > of running under a debugger.? The alternative is typing the > sub-expression(s) of interest by hand at the debugger prompt, or > adding print()s, both of which are prone to introducing typos, or > changing results radically due to triggering side effects in the code > invoked by the duplicated sub-expression(s).? Adding a binding > construct wouldn't change anything about how the code worked (apart > from possibly clobbering a local name). > > > ?Indeed, in the cases where I currently find myself unwrapping > expressions to capture their values in local variables for debugging > purposes it would usually be far less intrusive to bind a name to the > expression inline, then use the debugger to inspect the value. I agree that this is a definite plus feature. Being able to tag subexpressions would make visual debuggers that show all local variables as one steps (like IDLE's) even more useful relative to print statements. -- Terry Jan Reedy From jeanpierreda at gmail.com Wed Apr 25 17:45:19 2018 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Wed, 25 Apr 2018 14:45:19 -0700 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: References: <20180425034330.GP11616@ando.pearwood.info> Message-ID: On Wed, Apr 25, 2018 at 2:17 PM, Terry Reedy wrote: > On 4/25/2018 6:10 AM, Steve Holden wrote: >> Indeed, in the cases where I currently find myself unwrapping expressions >> to capture their values in local variables for debugging purposes it would >> usually be far less intrusive to bind a name to the expression inline, then >> use the debugger to inspect the value. > > > I agree that this is a definite plus feature. Being able to tag > subexpressions would make visual debuggers that show all local variables as > one steps (like IDLE's) even more useful relative to print statements. Some other programming languages (thinking of Racket) solve this by having the debugger let you step through expression evaluation, without editing the code. e.g. in the line x = 1 + 2 * 3, we might step through and first evaluate 2*3 (-> 6), and then 1 + (-> 7). Similar to how Python already lets you step into and see the result of function calls. This is especially powerful in visual debuggers, where the stepping and output can be displayed very intuitively. -- Devin From tim.peters at gmail.com Wed Apr 25 17:55:43 2018 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 25 Apr 2018 16:55:43 -0500 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: [Guido] >> You don't seem to grasp the usability improvements this will give. >> I hear you but at this point appeals to Python's "Zen" don't help you. [?ukasz Langa ] > This reads dismissive to me. I did read the PEP and followed the discussion on > python-dev. I referred to PEP 20 because it distills what's unique about the > value proposition of Python. It's our shared vocabulary. > > Can you address the specific criticism I had? To paraphrase it without PEP 20 > jargon: > (name := expression) makes code less uniform. It inserts more information > into a place that is already heavily packed with information (logic tests). I'll take a crack at that. It's not about "head arguments" at all. I sat out the first hundred messages about this on python-ideas, and looked at code instead. What I found had little to do with any of the head (abstract) arguments passionately debated for the duration ;-) In real life, I found a great many conditional tests that not only weren't "heavily packed" with information, they were simply of the form: NAME = expression if NAME: ... use NAME ... That looks more like assembly language than Python ;-) I saw no harm at all, and a little gain, in if NAME := expression: ... use NAME ... instead. But even a little gain adds up when it happens so often. Of course there have been better examples given of bigger gains. But in no case have the tests in those examples been "heavily packed with information". If they had been, I would have suggested instead breaking the test clauses _out_ of the conditional statements, and giving them names each on their own dedicated lines, with comments explaining what the heck the _intents_ are, even at the cost of adding an indentation level or two. Sometimes conditionals are _already_ "too dense". But more often they're very sparse. This becomes a question of seasoned judgment. For example, here's a real loop summing a series expansion, until the new terms become so small they make no difference to the running total (a common enough pattern in code slinging floats or decimals): while True: old = total total += term if old == total: return total term *= mx2 / (i*(i+1)) i += 2 To my eyes, this is genuinely harder to follow, despite its relative brevity: while total != (total := total + term): term *= mx2 / (i*(i+1)) i += 2 return total So I wouldn't use binding expressions in that case. I don't have a compelling head argument for _why_ I find the latter spelling harder to follow, but I don't need a theory to know that I in fact do. But neither do I need a compelling head argument for "why" to know that in many other cases I find that the use of binding expressions improves the code. You shouldn't believe me even if I pretended to have one and passionately argued for it. But, by the same token, I'm spectacularly unmoved by other peoples' head arguments. For that reason, the messages that sway me are those showing real code, or at least plausibly realistic code. In the majority of those so far, binding expressions would be a small-to-major win. From guido at python.org Wed Apr 25 17:56:32 2018 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2018 14:56:32 -0700 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: On Wed, Apr 25, 2018 at 1:55 PM, ?ukasz Langa wrote: > > > On 25 Apr, 2018, at 1:28 PM, Guido van Rossum wrote: > > > > You don't seem to grasp the usability improvements this will give. I > hear you but at this point appeals to Python's "Zen" don't help you. > > This reads dismissive to me. I did read the PEP and followed the > discussion on > python-dev. It was meant dismissive. With Chris, I am tired of every core dev starting their own thread about how PEP 572 threatens readability or doesn't reach the bar for new syntax (etc.). These arguments are entirely emotional and subjective. And that's how big decisions get made. Nobody can predict the outcome with sufficient accuracy. It's like buying a new car or house. In the end you decide with your gut. > I referred to PEP 20 because it distills what's unique about the > value proposition of Python. It's our shared vocabulary. > It's poetry, not a set of axioms. You can't *prove* anything with an appeal to PEP 20. You can appeal to it, for sure, but such an appeal *by definition* is subjective and emotional. (There's Only One Way To Do It? Give me a break. :-) > Can you address the specific criticism I had? To paraphrase it without PEP > 20 > jargon: > > > (name := expression) makes code less uniform. It inserts more > information > > into a place that is already heavily packed with information (logic > tests). > Most Python features make code less uniform in order to make it less repetitive. (Who needs classes? :-) -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Wed Apr 25 18:08:39 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 26 Apr 2018 00:08:39 +0200 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: <20180426000839.27d940c4@fsol> On Wed, 25 Apr 2018 16:55:43 -0500 Tim Peters wrote: > > To my eyes, this is genuinely harder to follow, despite its relative brevity: > > while total != (total := total + term): Does it even work? Perhaps if the goal is to stop when total is NaN, but otherwise? > For that reason, the messages that sway me are those showing real > code, or at least plausibly realistic code. In the majority of those > so far, binding expressions would be a small-to-major win. I'm sure it's possible to find thousands of line of code where binding expressions wouldn't be a win, but I'm not sure that would be a constructive use of mailing-list bandwidth. Regards Antoine. From ethan at stoneleaf.us Wed Apr 25 18:15:53 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 25 Apr 2018 15:15:53 -0700 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: <5AE0FE19.6040401@stoneleaf.us> On 04/25/2018 02:55 PM, Tim Peters wrote: > This becomes a question of seasoned judgment. For example, here's a > real loop summing a series expansion, until the new terms become so > small they make no difference to the running total (a common enough > pattern in code slinging floats or decimals): > > while True: > old = total > total += term > if old == total: > return total > term *= mx2 / (i*(i+1)) > i += 2 > > To my eyes, this is genuinely harder to follow, despite its relative brevity: > > while total != (total := total + term): > term *= mx2 / (i*(i+1)) > i += 2 > return total > > So I wouldn't use binding expressions in that case. I don't have a > compelling head argument for _why_ I find the latter spelling harder > to follow, but I don't need a theory to know that I in fact do. I know why I do: I see "while total != total" and my gears start stripping. On the other hand, while total != (total + term as total): ... I find still intelligible. (Yes, I know "as" is dead, just wanted to throw that out there.) -- ~Ethan~ From steve at pearwood.info Wed Apr 25 18:18:37 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 26 Apr 2018 08:18:37 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: <20180425221837.GB7400@ando.pearwood.info> On Wed, Apr 25, 2018 at 01:55:37PM -0700, ?ukasz Langa wrote: > > > On 25 Apr, 2018, at 1:28 PM, Guido van Rossum wrote: > > > > You don't seem to grasp the usability improvements this will give. I hear you but at this point appeals to Python's "Zen" don't help you. > > This reads dismissive to me. I did read the PEP and followed the discussion on > python-dev. I referred to PEP 20 because it distills what's unique about the > value proposition of Python. It's our shared vocabulary. Every programming language has a shared vocabulary. That's hardly unique to Python. > Can you address the specific criticism I had? To paraphrase it without PEP 20 > jargon: > > > (name := expression) makes code less uniform. It inserts more information > > into a place that is already heavily packed with information (logic tests). I'm not Guido, but I'll make an attempt. I think the comment about "less uniform" isn't meaningful. Uniform in what way? I don't even know how to interpret the uniform comment here, unless you mean to imply that every statement and expression in Python currently has the same information density, and binding-expressions will violate that. That's clearly not the case, so I'm left puzzled by what you mean. As for your observation that binding-expressions don't reduce complexity, they merely move it, I think you may be right. But then it is a truism that complexity is never reduced, only moved, so that's likely to be true for any feature (including existing ones). Should we move back to assembly language programming because Python hasn't reduced complexity, only moved it? I don't think so. Clearly binding-expressions do add a little more complexity to the language, and they do move code from vertically separated statements to horizontally laid-out expressions. But why is this necessarily a bad thing? Exactly the same complaint can be made about comprehensions, and look at how wildly successful they have been. Offset against the increase in horizontal complexity is a corresponding decrease in vertical complexity, and that's beneficial. Whatever cost they have has to be offset against the benefits, and I think the feature will come ahead on the plus side overall. Of course, like any syntactic feature, it may be abused by those who (by accident or design) write obfuscated or excessively complex code. We shouldn't ignore that risk, but nor should we use that as an excuse to dismiss the feature's benefits. -- Steve From tim.peters at gmail.com Wed Apr 25 18:22:23 2018 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 25 Apr 2018 17:22:23 -0500 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <20180426000839.27d940c4@fsol> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <20180426000839.27d940c4@fsol> Message-ID: [Tim] >> To my eyes, this is genuinely harder to follow, despite its relative brevity: >> >> while total != (total := total + term): [Antoine] > Does it even work? Perhaps if the goal is to stop when total is NaN, > but otherwise? I don't follow you. You snipped all the text explaining why it would work, so trying reading that again? When, e.g., `total` reaches 1.0 and `term` reaches 1e-30, this becomes: while 1.0 != (total := 1.0 + 1-e30): which leaves `total` unchanged (1.0 + 1e-30 == 1.0) and then while 1.0 != 1.0: causes the loop to exit (`while False:`). >> For that reason, the messages that sway me are those showing real >> code, or at least plausibly realistic code. In the majority of those >> so far, binding expressions would be a small-to-major win. > I'm sure it's possible to find thousands of line of code where binding > expressions wouldn't be a win, but I'm not sure that would be a > constructive use of mailing-list bandwidth. And that "argument" is? ;-) Note that I managed to move the PEP _away_ from general "assignment expressions" to the much simpler "binding expressions" precisely _by_ illustrating, via real code, why the generality of the former wasn't actually useful in any case I looked at. If something is always - or almost always - useless, that can be shown via considering realistic code. That was far more productive than endless abstract debates. From ethan at stoneleaf.us Wed Apr 25 18:36:02 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 25 Apr 2018 15:36:02 -0700 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <5AE0FE19.6040401@stoneleaf.us> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <5AE0FE19.6040401@stoneleaf.us> Message-ID: <5AE102D2.70204@stoneleaf.us> On 04/25/2018 03:15 PM, Ethan Furman wrote: > On 04/25/2018 02:55 PM, Tim Peters wrote: > >> This becomes a question of seasoned judgment. For example, here's a >> real loop summing a series expansion, until the new terms become so >> small they make no difference to the running total (a common enough >> pattern in code slinging floats or decimals): >> >> while True: >> old = total >> total += term >> if old == total: >> return total >> term *= mx2 / (i*(i+1)) >> i += 2 >> >> To my eyes, this is genuinely harder to follow, despite its relative brevity: >> >> while total != (total := total + term): >> term *= mx2 / (i*(i+1)) >> i += 2 >> return total >> >> So I wouldn't use binding expressions in that case. I don't have a >> compelling head argument for _why_ I find the latter spelling harder >> to follow, but I don't need a theory to know that I in fact do. > > I know why I do: I see "while total != total" and my gears start stripping. On the other hand, > > while total != (total + term as total): > ... > > I find still intelligible. (Yes, I know "as" is dead, just wanted to throw that out there.) Having said that, since whomever mentioned reading ":=" as "which is", I'm good with ":=". -- ~Ethan~ From rosuav at gmail.com Wed Apr 25 18:38:51 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Apr 2018 08:38:51 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <20180426000839.27d940c4@fsol> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <20180426000839.27d940c4@fsol> Message-ID: On Thu, Apr 26, 2018 at 8:08 AM, Antoine Pitrou wrote: > On Wed, 25 Apr 2018 16:55:43 -0500 > Tim Peters wrote: >> >> To my eyes, this is genuinely harder to follow, despite its relative brevity: >> >> while total != (total := total + term): > > Does it even work? Perhaps if the goal is to stop when total is NaN, > but otherwise? Yes, it does, because the first "total" is looked up before the rebinding happens. It's 100% unambiguous to the compiler... but still pretty unclear to a human. And I think the multiple use of 'total' is to blame for that. So I agree with Tim that this particular example is better in longhand. ChrisA From guido at python.org Wed Apr 25 18:39:15 2018 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2018 15:39:15 -0700 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <5AE0FE19.6040401@stoneleaf.us> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <5AE0FE19.6040401@stoneleaf.us> Message-ID: On Wed, Apr 25, 2018 at 3:15 PM, Ethan Furman wrote: > On 04/25/2018 02:55 PM, Tim Peters wrote: > >> To my eyes, this is genuinely harder to follow, despite its relative >> brevity: >> >> while total != (total := total + term): >> term *= mx2 / (i*(i+1)) >> i += 2 >> return total >> >> So I wouldn't use binding expressions in that case. I don't have a >> compelling head argument for _why_ I find the latter spelling harder >> to follow, but I don't need a theory to know that I in fact do. >> > > I know why I do: I see "while total != total" and my gears start > stripping. On the other hand, > > while total != (total + term as total): > ... > > I find still intelligible. (Yes, I know "as" is dead, just wanted to > throw that out there.) > The problem with either variant is that they hinge on subtle left-to-right evaluation rules. Python tries to promise left-to-right evaluation "except when it doesn't apply", e.g. in assignments the RHS is typically evaluated before subexpressions in the LHS: a[f()] = g() calls g() before f(). The example is supposed to load the left operand to != on the stack before loading the right operand, but the rule that says the left operand is evaluated before the right operand is much weaker than other evaluation order rules (like the rule stating that the arguments are evaluated before the function is called -- and before you laugh, in Algol-60 that wasn't always the case). This argument applies regardless of which syntactic form you use, and no matter what we choose, the PEP will have to clarify evaluation order in more cases than the current reference manual. (IIRC Nathaniel brought this up.) -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Wed Apr 25 18:44:48 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 26 Apr 2018 00:44:48 +0200 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <20180426000839.27d940c4@fsol> Message-ID: <20180426004448.344dda53@fsol> On Thu, 26 Apr 2018 08:38:51 +1000 Chris Angelico wrote: > On Thu, Apr 26, 2018 at 8:08 AM, Antoine Pitrou wrote: > > On Wed, 25 Apr 2018 16:55:43 -0500 > > Tim Peters wrote: > >> > >> To my eyes, this is genuinely harder to follow, despite its relative brevity: > >> > >> while total != (total := total + term): > > > > Does it even work? Perhaps if the goal is to stop when total is NaN, > > but otherwise? > > Yes, it does, because the first "total" is looked up before the > rebinding happens. It's 100% unambiguous to the compiler... but still > pretty unclear to a human. And I think the multiple use of 'total' is > to blame for that. So I agree with Tim that this particular example is > better in longhand. "Better" is an understatement :-( Now that I understood it (thanks for the explanation), the shorthand version appears completely bonkers. Regards Antoine. From vstinner at redhat.com Wed Apr 25 19:05:42 2018 From: vstinner at redhat.com (Victor Stinner) Date: Thu, 26 Apr 2018 01:05:42 +0200 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: > # Handle a matched regex > if (match := pattern.search(data)) is not None: > ... > > # A more explicit alternative to the 2-arg form of iter() invocation > while (value := read_next_item()) is not None: > ... > > # Share a subexpression between a comprehension filter clause and its output > filtered_data = [y for x in data if (y := f(x)) is not None] What do you think of adding the variant without the new ":=" syntax? It would help to see the benefit of the new ":=" syntax, and help to compare the two syntaxes. if: if (match := pattern.search(data)) is not None: ... vs match = pattern.search(data) if match is not None: ... while: while (value := read_next_item()) is not None: ... vs while True: value = read_next_item() if value is None: break ... list-comprehension: filtered_data = [y for x in data if (y := f(x)) is not None] vs filtered_data = [f(x) for x in data] filtered_data = [x for x in filtered_data if x is not None] Victor From rosuav at gmail.com Wed Apr 25 19:42:58 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Apr 2018 09:42:58 +1000 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: On Thu, Apr 26, 2018 at 9:05 AM, Victor Stinner wrote: >> # Handle a matched regex >> if (match := pattern.search(data)) is not None: >> ... >> >> # A more explicit alternative to the 2-arg form of iter() invocation >> while (value := read_next_item()) is not None: >> ... >> >> # Share a subexpression between a comprehension filter clause and its output >> filtered_data = [y for x in data if (y := f(x)) is not None] > > What do you think of adding the variant without the new ":=" syntax? > It would help to see the benefit of the new ":=" syntax, and help to > compare the two syntaxes. > > if: > if (match := pattern.search(data)) is not None: ... > vs > match = pattern.search(data) > if match is not None: ... > > while: > while (value := read_next_item()) is not None: ... > vs > while True: > value = read_next_item() > if value is None: break > ... > > list-comprehension: > filtered_data = [y for x in data if (y := f(x)) is not None] > vs > filtered_data = [f(x) for x in data] > filtered_data = [x for x in filtered_data if x is not None] Doing that for everything would put the PEP solidly into "TL;DR" territory, I'm afraid. There's already too much verbiage in some of those sections. Plus, we'd get right back into debates about how if you just change your intended semantics slightly, there's a completely different way to achieve something actually quite different, and we've already been around that a few times already. ChrisA From tim.peters at gmail.com Wed Apr 25 19:55:56 2018 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 25 Apr 2018 18:55:56 -0500 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <20180426004448.344dda53@fsol> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <20180426000839.27d940c4@fsol> <20180426004448.344dda53@fsol> Message-ID: [Tim] >>>> To my eyes, this is genuinely harder to follow, despite its relative brevity: >>>> >>>> while total != (total := total + term): [Antoine] >>> Does it even work? Perhaps if the goal is to stop when total is NaN, >>> but otherwise? [Chris] >> Yes, it does, because the first "total" is looked up before the >> rebinding happens. It's 100% unambiguous to the compiler... but still >> pretty unclear to a human. And I think the multiple use of 'total' is >> to blame for that. So I agree with Tim that this particular example is >> better in longhand. [Antoine] > "Better" is an understatement :-( Now that I understood it (thanks > for the explanation), Ah, sorry - I had no idea it was the "left to right evaluation" part you weren't seeing. Next time explain why you think something is broken? > the shorthand version appears completely bonkers. I wouldn't go that far, but I already said I wouldn't write it that way. However, without looking at real code, people are just flat-out guessing about how bad - or good - things _can_ get, no matter how confident they sound. So at least give me credit for presenting the _worst_ brief binding-expression example you've seen too ;-) From mikhailwas at gmail.com Wed Apr 25 19:54:43 2018 From: mikhailwas at gmail.com (Mikhail V) Date: Thu, 26 Apr 2018 02:54:43 +0300 Subject: [Python-Dev] Visual similarity of "=" and "==" (in context of PEP 572) Message-ID: Since the discussion about operator choice has completely migrated here, I'll put my note also here. >From the very beginning of PEP 572 discussion I have noticed a strange fact - there is told a lot about visual similarity of "=" and "==" in Python. *Same is told in the PEP 572 (frequently asked question) as the reason for rejection of "=" as operator.* I realize that currently there are other technical arguments against "=", but I am the opinion that this very reason, namely similarity of "=" and "==", cannot be taken seriously. I'll try to explain: I have opened Notepad++ and went to syntax highlighting configurator. It took me 2 minutes to choose a syntax setup that defines "==" (and other comparison operators if you wish) as a token with custom style. Now I can apply *custom color and size* to it and this alone makes the operator very well seen and different from "=". If that is not enough, I can set different *font* for the operator: so I can basically turn "==" into anything I want. With this in mind, I personally can't understand why you complain about "=="? Ok, you could say - not all editors can do this. But even without that tweaks - I still don't find the problem worth exaggeration. It is not like I accidentally choose very similar names for variables and then spend an hour trying to figure out where is the bug comes from. With "==" it just takes some attention to look at -if- statements and recheck if I write "==" and not "=" where I need comparison. I see it as as a "typo-magnet" but no way as a "bug-magnet" as it was claimed repeatedly in the discussion. A programmer already knows about this similarity and can fix the typos easily. The ":=" operator is of course slightly more different, but still, only _slightly_ better in this regard. With that said, I am not much interested in either adding of the expression assignment, although I am really worrying about the reasoning made about "=" and "==" in the discussion. Hope you get my point here. IMO either the PEP 572 should remove or at least rephrase this point somehow or, even better, concentrate on something more convincing in this regard. Also I think Yury has made good point about the introduction of ":=" -- it would co-exist with the "=". So imo, if you get familiar with ":=", later on it will just cause additional visual noise. And I don't think a regular user will be much interested in nuances of internal implementation of new assignment - they'll just follow the logic of the code and another operator will constantly gain unnecessary attention at reading. Mikhail From rosuav at gmail.com Wed Apr 25 20:03:53 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Apr 2018 10:03:53 +1000 Subject: [Python-Dev] Visual similarity of "=" and "==" (in context of PEP 572) In-Reply-To: References: Message-ID: On Thu, Apr 26, 2018 at 9:54 AM, Mikhail V wrote: > Since the discussion about operator choice has completely migrated > here, I'll put my note also here. > From the very beginning of PEP 572 discussion I have noticed > a strange fact - there is told a lot about visual similarity > of "=" and "==" in Python. > *Same is told in the PEP 572 (frequently asked question) > as the reason for rejection of "=" as operator.* > > Hope you get my point here. > IMO either the PEP 572 should remove or at > least rephrase this point somehow or, even better, > concentrate on something more convincing in this regard. https://www.python.org/dev/peps/pep-0572/#why-not-just-turn-existing-assignment-into-an-expression Half a century of C programming demonstrates that this is ample argument. ChrisA From yselivanov.ml at gmail.com Wed Apr 25 20:11:05 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Thu, 26 Apr 2018 00:11:05 +0000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: On Wed, Apr 25, 2018 at 5:58 PM Guido van Rossum wrote: [..] > It was meant dismissive. With Chris, I am tired of every core dev starting their own thread about how PEP 572 threatens readability or doesn't reach the bar for new syntax (etc.). These arguments are entirely emotional and subjective. FWIW I started my thread for allowing '=' in expressions to make sure that we fully explore that path. I don't like ':=' and I thought that using '=' can make the idea more appealing to myself and others. It didn't, sorry if it caused any distraction. Although adding a new ':=' operator isn't my main concern. I think it's a fact that PEP 572 makes Python more complex. Teaching/learning Python will inevitably become harder, simply because there's one more concept to learn. Just yesterday this snippet was used on python-dev to show how great the new syntax is: my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf)) To my eye this is an anti-pattern. One line of code was saved, but the other line becomes less readable. The fact that 'buf' can be used after that line means that it will be harder for a reader to trace the origin of the variable, as a top-level "buf = " statement would be more visible. The PEP lists this example as an improvement: [(x, y, x/y) for x in input_data if (y := f(x)) > 0] I'm an experienced Python developer and I can't read/understand this expression after one read. I have to read it 2-3 times before I trace where 'y' is set and how it's used. Yes, an expanded form would be ~4 lines long, but it would be simple to read and therefore review, maintain, and update. Assignment expressions seem to optimize the *writing code* part, while making *reading* part of the job harder for some of us. I write a lot of Python, but I read more code than I write. If the PEP gets accepted I'll use the new syntax sparingly, sure. My main concern, though, is that this PEP will likely make my job as a code maintainer harder in the end, not easier. I hope I explained my -1 on the PEP without sounding emotional. Thank you, Yury Yury From solipsis at pitrou.net Wed Apr 25 20:14:14 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 26 Apr 2018 02:14:14 +0200 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <20180426000839.27d940c4@fsol> <20180426004448.344dda53@fsol> Message-ID: <20180426021414.2307268a@fsol> On Wed, 25 Apr 2018 18:55:56 -0500 Tim Peters wrote: > > > the shorthand version appears completely bonkers. > > I wouldn't go that far, but I already said I wouldn't write it that way. > > However, without looking at real code, people are just flat-out > guessing about how bad - or good - things _can_ get, no matter how > confident they sound. > > So at least give me credit for presenting the _worst_ brief > binding-expression example you've seen too ;-) I had no idea you were a bit short on them, so I'll gladly give you credits for it :-) But I hope you'll use them responsibly! Regards Antoine. From rosuav at gmail.com Wed Apr 25 20:20:40 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Apr 2018 10:20:40 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: On Thu, Apr 26, 2018 at 10:11 AM, Yury Selivanov wrote: > Just yesterday this snippet was used on python-dev to show how great the > new syntax is: > > my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf)) > > To my eye this is an anti-pattern. One line of code was saved, but the > other line becomes less readable. The fact that 'buf' can be used after > that line means that it will be harder for a reader to trace the origin of > the variable, as a top-level "buf = " statement would be more visible. Making 'buf' more visible is ONLY a virtue if it's going to be used elsewhere. Otherwise, the name 'buf' is an implementation detail of the fact that this function wants both a buffer and a size. Should you want to expand this out over more lines, you could do this: template = [None] buf = template*get_size() length = len(buf) my_func(arg, buffer=buf, size=length) What are the names 'template' and 'length' achieving? Why should they claim your attention? They are useless relics of a done-and-dusted calculation, being retained for no reason. They do not deserve top-level placement. The form as given above is starting to get a bit noisy, but I strongly disagree that 'buf' deserves to be a stand-alone name. It is as valueless as 'template' is. ChrisA From solipsis at pitrou.net Wed Apr 25 20:41:44 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 26 Apr 2018 02:41:44 +0200 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: <20180426024144.0884c1a0@fsol> On Thu, 26 Apr 2018 10:20:40 +1000 Chris Angelico wrote: > On Thu, Apr 26, 2018 at 10:11 AM, Yury Selivanov > wrote: > > Just yesterday this snippet was used on python-dev to show how great the > > new syntax is: > > > > my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf)) > > > > To my eye this is an anti-pattern. One line of code was saved, but the > > other line becomes less readable. The fact that 'buf' can be used after > > that line means that it will be harder for a reader to trace the origin of > > the variable, as a top-level "buf = " statement would be more visible. > > Making 'buf' more visible is ONLY a virtue if it's going to be used > elsewhere. Otherwise, the name 'buf' is an implementation detail of > the fact that this function wants both a buffer and a size. Should you > want to expand this out over more lines, you could do this: > > template = [None] > buf = template*get_size() > length = len(buf) > my_func(arg, buffer=buf, size=length) > > What are the names 'template' and 'length' achieving? Why should they > claim your attention? What is the name 'buf' in the binding expression achieving? Why should it claim my attention? It's not any different: it's just something that's used in a statement then unnecessary. Yet it will persist until the end of the enclosing scope, being retained for no reason. Perhaps we need C-like nested scopes, if such is the concern about names that live for too long? (of course, the fact that `my_func` needs you to pass its argument's length as a separate argument, while it could compute it by itself, is a bit silly) As a side note, personally, I'm usually much more concerned about the lifetime of *values* than the lifetime of names. The latter are cheap, the former can represent expensive resources. Regards Antoine. > They are useless relics of a done-and-dusted > calculation, being retained for no reason. They do not deserve > top-level placement. > > The form as given above is starting to get a bit noisy, but I strongly > disagree that 'buf' deserves to be a stand-alone name. It is as > valueless as 'template' is. > > ChrisA From yselivanov.ml at gmail.com Wed Apr 25 20:46:09 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Thu, 26 Apr 2018 00:46:09 +0000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: On Wed, Apr 25, 2018 at 8:22 PM Chris Angelico wrote: [..] > > my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf)) > > > > To my eye this is an anti-pattern. One line of code was saved, but the > > other line becomes less readable. The fact that 'buf' can be used after > > that line means that it will be harder for a reader to trace the origin of > > the variable, as a top-level "buf = " statement would be more visible. > Making 'buf' more visible is ONLY a virtue if it's going to be used > elsewhere. Otherwise, the name 'buf' is an implementation detail of > the fact that this function wants both a buffer and a size. Should you > want to expand this out over more lines, you could do this: Chris, you didn't read that paragraph in my email to the end or I did a poor job at writing it. My point is that "buf" can still be used below that line, and therefore sometimes it will be used, as a result of quick refactoring or poor coding style. It's just how things happen when you write code: it gets rewritten and parts of it left outdated or not properly revised. *If* "buf" is used below that line it *will* be harder to find where it was initially set. Anyways, I don't want to distract everyone further so I'm not interested in continuing the discussion about what is readable and what is not. My own opinion on this topic is unlikely to change. I wanted to explain my -1; hopefully it will be noted. Yury From lukasz at langa.pl Wed Apr 25 21:52:34 2018 From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=) Date: Wed, 25 Apr 2018 18:52:34 -0700 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: > On 25 Apr, 2018, at 5:20 PM, Chris Angelico wrote: > > On Thu, Apr 26, 2018 at 10:11 AM, Yury Selivanov > wrote: >> Just yesterday this snippet was used on python-dev to show how great the >> new syntax is: >> >> my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf)) >> >> To my eye this is an anti-pattern. One line of code was saved, but the >> other line becomes less readable. The fact that 'buf' can be used after >> that line means that it will be harder for a reader to trace the origin of >> the variable, as a top-level "buf = " statement would be more visible. > > Making 'buf' more visible is ONLY a virtue if it's going to be used > elsewhere. Otherwise, the name 'buf' is an implementation detail of > the fact that this function wants both a buffer and a size. You're claiming that `:=` is nicer in this situation because it's less prominent than regular assignment and thus doesn't suggest that the name stays visible later. But as others said, `:=` *does* make the name visible later until the enclosing scope ends. In fact, a large part of its appeal is that you can use the result later (as in the `re.search()` example). Will it be visible enough to the reaser in those cases then? There seems to be a conflict there. The question of assignment visibility also makes me think about unintentional name shadowing:: buf = some_value ... # 20 lines my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf)) ... # 20 lines buf # <-- What value does this have? Even if we're not using the call pattern, there can be plenty of logic tests which aren't very obvious:: buf = some_value ... # 20 lines if node.parent is not None and (buf := node.parent.buffer): ... # 10 lines ... # 20 lines buf # <-- What value does this have? This is even more interesting because now `buf` isn't rebound *always*. So if I'm confused about an unexpected change in value of `buf`, I'll skim the code, fail to find the assignment, and then grep for `buf =` and also fail to find the assignment. Yes, I could have searched for just `buf` instead but that will give me too many false positives, especially if I'm using a primitive text editor search or don't know about \b in regular expressions. Debugging this can be confusing. I know it can since a similar annoyance can be observed with the magic pseudo-scope of `except`:: err = some_value try: ... except Error as err: ... err # <-- now sometimes it's not defined Just like Barry, I debugged a few cases of this in the past and within larger functions this can be hard to find. -- ? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From rymg19 at gmail.com Wed Apr 25 22:36:31 2018 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Wed, 25 Apr 2018 21:36:31 -0500 Subject: [Python-Dev] Is PEP 572 really the most effective way to solve the problems it's targeting? Message-ID: <162ffcf2f98.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> I have to say I'm not overly thrilled with PEP 572...it's almost odd, because if you asked me back when I first joined this list when I was 13, I would've no doubt said *YES*. But, since then, I've gone across many projects and languages, and fundamentally *I have never felt hurt by the lack of assignment in an expression*, and I always regretted every time I tried it in C or Crystal. I understand this experience is pretty insignificant in comparison to many of the wizards here, but I thought I'd still share it as an opener for what I'm about to say. With this being said, I'd encourage everyone to take a bit of a step back: what exactly are people looking for in PEP 572? I see two main goals: - Assignment in a conditional structure. - Assignment in a list comprehension. Most other use cases would significantly hurt readability and seem pretty rare. Now let's break down the top one: - Assignment in an if condition. - Assignment in a while condition. So there are roughly three main goals here overall. Now, are there better ways to solve these? (FWIW C solved the while condition one with the C-style for loop, but I'm pretty sure few people here would really go for that.) C++ has recently solved the if condition by allowing declarations inside the conditions: if (auto a = 123; a != 456) { Many languages have a 'let' expression (using Felix as my example): if let a = 1, b = 2 in a == b then Swift has taken a bit of a hybrid between the above two: if let a = 1, b = 2, a == b { Now, what's the common theme here? **Declarations should be separate from expressions.** We've got languages that range from baggage-filled to functional to a bit of all of the above, and none of them have added assignment *inside* an expression. The argument is roughly the same across all boards: you're putting major but easy-to-miss side effects in the midst of expressions that *seem* pure. All this is to say: I'd really encourage everyone here to think a bit more about *why* exactly you want this feature, and then think if there's really no better way. Any solution that separates declarations would be far more readable, (arguably) more Pythonic, and play more nicely with the new-ish typing features to boot I understand reluctance to add a syntax exception like this, but I really feel it'd be worth it. As a side note, I was a strong supporter of comprehension generalizations, f-strings, *and* dataclasses. However, this proposal seems a bit ugly and excessive for what it's trying to accomplish. P.S. Yes, the unmatched curly braces were intentional to drive you crazy for a few hours. I blame Randall Monroe. You're welcome. -- Ryan (????) Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone else https://refi64.com/ From wes.turner at gmail.com Wed Apr 25 23:07:01 2018 From: wes.turner at gmail.com (Wes Turner) Date: Wed, 25 Apr 2018 23:07:01 -0400 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: On Wednesday, April 25, 2018, ?ukasz Langa wrote: > > On 25 Apr, 2018, at 5:20 PM, Chris Angelico wrote: > > On Thu, Apr 26, 2018 at 10:11 AM, Yury Selivanov > wrote: > > Just yesterday this snippet was used on python-dev to show how great the > new syntax is: > > my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf)) > > To my eye this is an anti-pattern. One line of code was saved, but the > other line becomes less readable. The fact that 'buf' can be used after > that line means that it will be harder for a reader to trace the origin of > the variable, as a top-level "buf = " statement would be more visible. > > > Making 'buf' more visible is ONLY a virtue if it's going to be used > elsewhere. Otherwise, the name 'buf' is an implementation detail of > the fact that this function wants both a buffer and a size. > > > You're claiming that `:=` is nicer in this situation because it's less > prominent than regular assignment and thus doesn't suggest that the name > stays visible later. > > But as others said, `:=` *does* make the name visible later until the > enclosing scope ends. In fact, a large part of its appeal is that you > can use the result later (as in the `re.search()` example). Will it be > visible enough to the reaser in those cases then? > > There seems to be a conflict there. > > The question of assignment visibility also makes me think about > unintentional name shadowing:: > > buf = some_value > > ... # 20 lines > > my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf)) > > ... # 20 lines > > buf # <-- What value does this have? > > > Even if we're not using the call pattern, there can be plenty of logic > tests which aren't very obvious:: > > buf = some_value > > ... # 20 lines > > if node.parent is not None and (buf := node.parent.buffer): > ... # 10 lines > > ... # 20 lines > > buf # <-- What value does this have? > > > This is even more interesting because now `buf` isn't rebound > *always*. > > So if I'm confused about an unexpected change in value of `buf`, I'll > skim the code, fail to find the assignment, and then grep for `buf =` > and also fail to find the assignment. Yes, I could have searched for > just `buf` instead but that will give me too many false positives, > especially if I'm using a primitive text editor search or don't know > about \b in regular expressions. > > Debugging this can be confusing. I know it can since a similar > annoyance can be observed with the magic pseudo-scope of `except`:: > > err = some_value > try: > ... > except Error as err: > ... > > err # <-- now sometimes it's not defined > > > Just like Barry, I debugged a few cases of this in the past and within > larger functions this can be hard to find. > Would this make it easier to put too much code on one line? Is there a good way to get *branch coverage* stats instead of just *line coverage*? Someone can probably explain with some tested pretty code for me why this would be necessary or helpful; why it wouldn't make line coverage stats more misleading for the sake of lazy? > > -- ? > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.hettinger at gmail.com Wed Apr 25 23:34:44 2018 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Wed, 25 Apr 2018 23:34:44 -0400 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: > On Apr 25, 2018, at 8:11 PM, Yury Selivanov wrote: > > FWIW I started my thread for allowing '=' in expressions to make sure that > we fully explore that path. I don't like ':=' and I thought that using '=' > can make the idea more appealing to myself and others. It didn't, sorry if > it caused any distraction. Although adding a new ':=' operator isn't my main > concern. > > I think it's a fact that PEP 572 makes Python more complex. > Teaching/learning Python will inevitably become harder, simply because > there's one more concept to learn. > > Just yesterday this snippet was used on python-dev to show how great the > new syntax is: > > my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf)) > > To my eye this is an anti-pattern. One line of code was saved, but the > other line becomes less readable. The fact that 'buf' can be used after > that line means that it will be harder for a reader to trace the origin of > the variable, as a top-level "buf = " statement would be more visible. > > The PEP lists this example as an improvement: > > [(x, y, x/y) for x in input_data if (y := f(x)) > 0] > > I'm an experienced Python developer and I can't read/understand this > expression after one read. I have to read it 2-3 times before I trace where > 'y' is set and how it's used. Yes, an expanded form would be ~4 lines > long, but it would be simple to read and therefore review, maintain, and > update. > > Assignment expressions seem to optimize the *writing code* part, while > making *reading* part of the job harder for some of us. I write a lot of > Python, but I read more code than I write. If the PEP gets accepted I'll > use > the new syntax sparingly, sure. My main concern, though, is that this PEP > will likely make my job as a code maintainer harder in the end, not easier. > > I hope I explained my -1 on the PEP without sounding emotional. FWIW, I concur with all of Yuri's thoughtful comments. After re-reading all the proposed code samples, I believe that adopting the PEP will make the language harder to teach to people who are not already software engineers. To my eyes, the examples give ample opportunity for being misunderstood and will create a need to puzzle-out the intended semantics. On the plus side, the proposal does address the occasional minor irritant of writing an assignment on a separate line. On the minus side, the visual texture of the new code is less appealing. The proposal also messes with my mental model for the distinction between expressions and statements. It probably doesn't matter at this point (minds already seem to be made up), but put me down for -1. This is a proposal we can all easily live without. Raymond From steve at pearwood.info Thu Apr 26 00:03:32 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 26 Apr 2018 14:03:32 +1000 Subject: [Python-Dev] Is PEP 572 really the most effective way to solve the problems it's targeting? In-Reply-To: <162ffcf2f98.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> References: <162ffcf2f98.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> Message-ID: <20180426040332.GD7400@ando.pearwood.info> On Wed, Apr 25, 2018 at 09:36:31PM -0500, Ryan Gonzalez wrote: > > > I have to say I'm not overly thrilled with PEP 572...it's almost odd, > because if you asked me back when I first joined this list when I was 13, I > would've no doubt said *YES*. I have the opposite experience: I've warmed to it the more I have read it. Before Chris had even written this PEP, there was a Python-Ideas thread asking for dedicated syntax in comprehensions alone that would have been equivalent to a binding-expression. I was rather negative about the whole thing (but no where near as negative as I've been in the past when this has come up before), until Chris pointed out that such binding-expressions have value outside of comprehensions. [...] > Now, what's the common theme here? **Declarations should be separate from > expressions.** Declarations and assignments are not the same thing. > We've got languages that range from baggage-filled to > functional to a bit of all of the above, and none of them have added > assignment *inside* an expression. And your claim about assignment inside expressions is only true if you ignore the languages where assignment is an expression with a return value. You know, strange and exotic languages with hardly any users or influence, like C, C++, C#, Java, Javascript, Ruby, Perl, PHP, Lisp ... -- Steve From tim.peters at gmail.com Thu Apr 26 00:19:03 2018 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 25 Apr 2018 23:19:03 -0500 Subject: [Python-Dev] Is PEP 572 really the most effective way to solve the problems it's targeting? In-Reply-To: <162ffcf2f98.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> References: <162ffcf2f98.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> Message-ID: [Ryan Gonzalez ] > I have to say I'm not overly thrilled with PEP 572...it's almost odd, > because if you asked me back when I first joined this list when I was 13, I > would've no doubt said *YES*. But, since then, I've gone across many > projects and languages, and fundamentally *I have never felt hurt by the > lack of assignment in an expression*, and I always regretted every time I > tried it in C or Crystal. I understand this experience is pretty > insignificant in comparison to many of the wizards here, but I thought I'd > still share it as an opener for what I'm about to say. The older you get, the more you'll regret not still being 13 ;-) > With this being said, I'd encourage everyone to take a bit of a step back: > what exactly are people looking for in PEP 572? > > I see two main goals: > > - Assignment in a conditional structure. > - Assignment in a list comprehension. > > Most other use cases would significantly hurt readability and seem pretty > rare. I haven't been much impressed by suggested uses outside conditional contexts either. > Now let's break down the top one: > > - Assignment in an if condition. > - Assignment in a while condition. > > So there are roughly three main goals here overall. Now, are there better > ways to solve these? > ... > C++ has recently solved the if condition by allowing declarations inside the > conditions: But C++ has always had assignment expressions. This: > if (auto a = 123; a != 456) { is solving a different (albeit related) problem: that C/C++ require declaring variables before use. Python doesn't. They could have done the same via, .e.g,, { auto a = 123; if (a != 456) { ... } } and still have had the scope of `a` limited to one block. auto-initializers in conditionals just gave a bit of syntactic sugar for what was already easily (although with more typing) done. > Many languages have a 'let' expression (using Felix as my example): > > if let a = 1, b = 2 in a == b then I don't read Felix, but I assume the _scope_ of `a` & `b` there ends immediately before the "then". If the names can't be used in the _body_ of a Python `if` (or `while`) block, it's essentially useless to allow binding names for use solely in the conditional test. So it would help if you picked "real Python examples" from the many other earlier messages in these threads. Python expressions can't span Python statement boundaries - only Python blocks can do that. A form of `let` that _would_ work would be block-structured: let m = regexp.match(pattern. line) in: if m: print(m.group(0)) That solves "a scope problem" the current version of the PEP gave up on, but in all other respects seems a step back from the current: m = regexp.match(pattern, line) if m: print(m.group(0)) > Swift has taken a bit of a hybrid between the above two: > > if let a = 1, b = 2, a == b { That seems plain incoherent ;-) > Now, what's the common theme here? **Declarations should be separate from > expressions.** We've got languages that range from baggage-filled to > functional to a bit of all of the above, and none of them have added > assignment *inside* an expression. C++ and C have always had assignment expressions . Ditto Java, Javascript, Perl, Icon, ... (many, many others). I don't see a good reason to grant that Felix and Swift are necessarily improvements over the former (with the exception of Icon, which I'm merely fond of) very widely used languages. > The argument is roughly the same across all boards: you're putting major but > easy-to-miss side effects in the midst of expressions that *seem* pure. > > All this is to say: I'd really encourage everyone here to think a bit more > about *why* exactly you want this feature, and then think if there's really > no better way. Any solution that separates declarations would be far more > readable, (arguably) more Pythonic, and play more nicely with the new-ish > typing features to boot People have been trying for years. If you come up with a realistic (for Python) idea, that's great - share it! But it's probably better suited to python-ideas than python-dev. > ... From rymg19 at gmail.com Thu Apr 26 00:23:36 2018 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Wed, 25 Apr 2018 23:23:36 -0500 Subject: [Python-Dev] Is PEP 572 really the most effective way to solve the problems it's targeting? In-Reply-To: <20180426040332.GD7400@ando.pearwood.info> References: <162ffcf2f98.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> <20180426040332.GD7400@ando.pearwood.info> Message-ID: <16300313940.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> On April 25, 2018 11:05:04 PM Steven D'Aprano wrote: On Wed, Apr 25, 2018 at 09:36:31PM -0500, Ryan Gonzalez wrote: I have to say I'm not overly thrilled with PEP 572...it's almost odd, because if you asked me back when I first joined this list when I was 13, I would've no doubt said *YES*. I have the opposite experience: I've warmed to it the more I have read it. Before Chris had even written this PEP, there was a Python-Ideas thread asking for dedicated syntax in comprehensions alone that would have been equivalent to a binding-expression. I was rather negative about the whole thing (but no where near as negative as I've been in the past when this has come up before), until Chris pointed out that such binding-expressions have value outside of comprehensions. [...] Now, what's the common theme here? **Declarations should be separate from expressions.** Declarations and assignments are not the same thing. Yeah, this is what happens when I write these things when tired... I meant to also mention assignments here, and anywhere else I may have used "declaration". We've got languages that range from baggage-filled to functional to a bit of all of the above, and none of them have added assignment *inside* an expression. And your claim about assignment inside expressions is only true if you ignore the languages where assignment is an expression with a return value. You know, strange and exotic languages with hardly any users or influence, like C, C++, C#, Java, Javascript, Ruby, Perl, PHP, Lisp ... In the majority of these, however, mixing assignments into expressions is considered bad practice. For instance: - C++ core guidelines: https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md#Res-complicated - MediaWiki style guide: https://m.mediawiki.org/wiki/Manual:Coding_conventions/PHP Some of the other languages (e.g. Perl) aren't some I'd regard as positive influences... -- Steve _______________________________________________ Python-Dev mailing list Python-Dev at python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com -- Ryan (????) Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone else https://refi64.com/ From tim.peters at gmail.com Thu Apr 26 00:40:32 2018 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 25 Apr 2018 23:40:32 -0500 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: [Raymond Hettinger ] > After re-reading all the proposed code samples, I believe that > adopting the PEP will make the language harder to teach to people > who are not already software engineers. Can you elaborate on that? I've used dozens of languages over the decades, most of which did have some form of embedded assignment. Yes, I'm a software engineer, but I've always pitched in on "help forums" too. One language feature conspicuous by absence in newbie confusions was, consistently, assignment expressions. Read any book or tutorial for such a language, and you'll find very little space devoted to them too. What's to learn? If they understand "binding a name" _at all_ (which they must to even begin to write a non-trivial program), the only twist is that a binding expression returns the value being bound. Binding expressions certainly wouldn't be the _first_ thing to teach people. But by the time it would make sense to teach them, it's hard for me to grasp how a student could struggle with such a tiny variation on what they've already learned (all the subtleties are in what - exactly - "binding"means - which they already faced the first time they saw "j = 1"). > To my eyes, the examples give ample opportunity for being > misunderstood and will create a need to puzzle-out the intended semantics. Some do, many don't. The same can be said of a great many constructs ;-) > ... From lukasz at langa.pl Thu Apr 26 01:14:11 2018 From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=) Date: Wed, 25 Apr 2018 22:14:11 -0700 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> [Uncle T] > One language feature conspicuous by absence in newbie > confusions was, consistently, assignment expressions. Read any book > or tutorial for such a language, and you'll find very little space > devoted to them too. Well, you have an entire code style built around this feature called Yoda conditions. You teach people on Day 1 to never ever confuse == with =. Some compilers even warn about this because so many people did it wrong. > What's to learn? If they understand "binding a name" _at all_ (which > they must to even begin to write a non-trivial program), the only > twist is that a binding expression returns the value being bound. Ha, not in Python! Here we have *different syntax* for assignments in expressions. Well, you can also use it as a statement. But don't! We have a better one for that. And that one supports type annotations, can unpack and assign to many targets at the same time, and can even increment, multiply and so on, at once. But the other one can't. So only use the Pascal one in expressions. But don't forget parentheses, otherwise it will bind the thing you probably didn't want anyway. >> To my eyes, the examples give ample opportunity for being >> misunderstood and will create a need to puzzle-out the intended semantics. > > Some do, many don't. As soon as we have to wrap a part of an expression in parentheses, parsing the entire thing becomes more complex. Often enough it will cause the expression to exceed whatever line length limit the codebase pledged not to exceed, causing one line to become three. And again, making it trickier for a regular ?ukasz to understand what's going on. -- ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Thu Apr 26 01:22:58 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 26 Apr 2018 17:22:58 +1200 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: Message-ID: <5AE16232.9030807@canterbury.ac.nz> ?ukasz Langa wrote: > What was its own assignment before > now is part of the logic test. This saves on vertical whitespace but makes > parsing and understanding logic tests harder. Another way to say this is that expressions are no longer restricted to being trees, but can be general DAGs, which require more mental effort to understand. Hmmm, maybe they should be called "spaghetti expressions". :-) -- Greg From steve at pearwood.info Thu Apr 26 01:34:17 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 26 Apr 2018 15:34:17 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <5AE16232.9030807@canterbury.ac.nz> References: <5AE16232.9030807@canterbury.ac.nz> Message-ID: <20180426053417.GE7400@ando.pearwood.info> On Thu, Apr 26, 2018 at 05:22:58PM +1200, Greg Ewing wrote: > ?ukasz Langa wrote: > >What was its own assignment before > >now is part of the logic test. This saves on vertical whitespace but makes > >parsing and understanding logic tests harder. > > Another way to say this is that expressions are no longer > restricted to being trees, but can be general DAGs, which > require more mental effort to understand. Is that right? I presume you mean that there can be cycles in expressions involving binding-expressions. If not, what do you mean? Can you give an example of a Python expression, involving PEP 572 binding-expressions, that is not a tree but a more general DAG or that contains cycles? -- Steve From rosuav at gmail.com Thu Apr 26 01:38:54 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Apr 2018 15:38:54 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <20180426053417.GE7400@ando.pearwood.info> References: <5AE16232.9030807@canterbury.ac.nz> <20180426053417.GE7400@ando.pearwood.info> Message-ID: On Thu, Apr 26, 2018 at 3:34 PM, Steven D'Aprano wrote: > On Thu, Apr 26, 2018 at 05:22:58PM +1200, Greg Ewing wrote: >> ?ukasz Langa wrote: >> >What was its own assignment before >> >now is part of the logic test. This saves on vertical whitespace but makes >> >parsing and understanding logic tests harder. >> >> Another way to say this is that expressions are no longer >> restricted to being trees, but can be general DAGs, which >> require more mental effort to understand. > > Is that right? I presume you mean that there can be cycles in > expressions involving binding-expressions. If not, what do you mean? > > Can you give an example of a Python expression, involving PEP 572 > binding-expressions, that is not a tree but a more general DAG or that > contains cycles? A DAG is a directed *acyclic* graph, so it still can't contain cycles. But I have no idea what kind of expression isn't a tree as a consequence of having an assignment in it. ChrisA From raymond.hettinger at gmail.com Thu Apr 26 01:50:16 2018 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Thu, 26 Apr 2018 01:50:16 -0400 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: <2AF82160-56E7-4DFB-8978-AB963C46B53B@gmail.com> > On Apr 26, 2018, at 12:40 AM, Tim Peters wrote: > > [Raymond Hettinger ] >> After re-reading all the proposed code samples, I believe that >> adopting the PEP will make the language harder to teach to people >> who are not already software engineers. > > Can you elaborate on that? Just distinguishing between =, :=, and == will be a forever recurring discussion, far more of a source of confusion than the occasional question of why Python doesn't have embedded assignment. Also, it is of concern that a number of prominent core dev respondents to this thread have reported difficulty scanning the posted code samples. > I've used dozens of languages over the > decades, most of which did have some form of embedded assignment. Python is special, in part, because it is not one of those languages. It has virtues that make it suitable even for elementary school children. We can show well-written Python code to non-computer folks and walk them through what it does without their brains melting (something I can't do with many of the other languages I've used). There is a virtue in encouraging simple statements that read like English sentences organized into English-like paragraphs, presenting itself like "executable pseudocode". Perl does it or C++ does it is unpersuasive. Its omission from Python was always something that I thought Guido had left-out on purpose, intentionally stepping away from constructs that would be of help in an obfuscated Python contest. > Yes, I'm a software engineer, but I've always pitched in on "help > forums" too. That's not really the same. I've taught Python to many thousands of professionals, almost every week for over six years. That's given me a keen sense of what is hard to teach. It's okay to not agree with my assessment, but I would like for fruits of my experience to not be dismissed in a single wisp of a sentence. Any one feature in isolation is usually easy to explain, but showing how to combine them into readable, expressive code is another matter. And as Yuri aptly noted, we spend more time reading code than writing code. If some fraction of our users finds the code harder to scan because the new syntax, then it would be a net loss for the language. I hesitated to join this thread because you and Guido seemed to be pushing back so hard against anyone's who design instincts didn't favor the new syntax. It would be nice to find some common ground and perhaps stipulate that the grammar would grow in complexity, that a new operator would add to the current zoo of operators, that the visual texture of the language would change (and in a way that some including me do not find pleasing), and that while simplest cases may afford a small net win, it is a certitude that the syntax will routinely be pushed beyond our comfort zone. While the regex conditional example looks like a win, it is very modest win and IMHO not worth the overall net increase language complexity. Like Yuri, I'll drop-out now. Hopefully, you all wind find some value in what I had to contribute to the conversation. Raymond From turnbull.stephen.fw at u.tsukuba.ac.jp Thu Apr 26 01:52:38 2018 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 26 Apr 2018 14:52:38 +0900 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: References: <20180425034330.GP11616@ando.pearwood.info> Message-ID: <23265.26918.552277.553688@turnbull.sk.tsukuba.ac.jp> Devin Jeanpierre writes: > Some other programming languages (thinking of Racket) solve this by > having the debugger let you step through expression evaluation, > without editing the code. Good tools are a wonderful thing, and I think pdb should be enhanced that way (by somebody who has the time and interest, not me and not necessarily you ;-). Nevertheless, "printf debugging" continues to be very popular, and good support for printf debugging is indeed the killer app for binding expressions as far as I'm concerned. Tim's humorous insight took me from -0.8 all the way to +1 Nice job, Chris! Good luck with the pronouncement! Steve From turnbull.stephen.fw at u.tsukuba.ac.jp Thu Apr 26 01:54:13 2018 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 26 Apr 2018 14:54:13 +0900 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: References: <20180425034330.GP11616@ando.pearwood.info> Message-ID: <23265.27013.849510.817475@turnbull.sk.tsukuba.ac.jp> Chris Angelico writes: > Additionally, naming sub-parts of a large expression can assist an > interactive debugger, providing useful display hooks and partial > results. Without a way to capture sub-expressions inline, this > would require refactoring of the original code; with assignment > expressions, this merely requires the insertion of a few ``name > :=`` markers. Removing the need to refactor reduces the likelihood > that the code be inadvertently changed as part of debugging (a > common cause of Heisenbugs), Period here preferred. > and is easier to dictate to a student or junior programmer. True but gratuitous. It's also true that it's easier to dictate to Guido or Tim, though you might be happier if you let them refactor! From tim.peters at gmail.com Thu Apr 26 01:57:44 2018 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 26 Apr 2018 00:57:44 -0500 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> Message-ID: [Tim] >> One language feature conspicuous by absence in newbie >> confusions was, consistently, assignment expressions. Read any book >> or tutorial for such a language, and you'll find very little space >> devoted to them too. [?ukasz Langa ] > Well, you have an entire code style built around this feature called Yoda > conditions. You teach people on Day 1 to never ever confuse == with =. Some > compilers even warn about this because so many people did it wrong. Sorry, I couldn't follow that. In languages like C that use easily confused operator symbols, sure, people are forever typing "=" when they mean "==". That's nothing to do with whether they _understand_ what the different operators do, though. They do. In languages like Icon (that use "=" for numeric comparison and ":=" for assignment), that never occurs. But I'm not sure that addressed the point you were making. >> What's to learn? If they understand "binding a name" _at all_ (which >> they must to even begin to write a non-trivial program), the only >> twist is that a binding expression returns the value being bound. > Ha, not in Python! Here we have *different syntax* for assignments in > expressions. Yes, binding expressions in the current PEP support an extremely limited subset of what Python's assignment statements support. That they use different operator symbols is irrelevant to that the meaning of "binding a name" is exactly the same for both.. _That's_ the "hard part" to learn. > Well, you can also use it as a statement. But don't! Why not? _Every_ expression in Python can be used as a statement. Nothing forbids it, and that's even (very!) useful at an interactive prompt. > We have a better one for that. As a matter of style, sure, it's best to use the simplest thing that works. As a statement in a program (as opposed to typed at a shell), "a := 3" has the unnecessary (in that context) property of returning (and discarding 3), so it's better style to use "a = 3" in that context. > And that one supports type annotations, can unpack and assign to many > targets at the same time, and can even increment, multiply and so on, at once. > But the other one can't. So? math.sqrt() blows up when passed -1, but cmath.sqrt() doesn't. Different tools for different tasks. > So only use the Pascal one in expressions. But don't forget parentheses, > otherwise it will bind the thing you probably didn't want anyway. [Raymond] >>> To my eyes, the examples give ample opportunity for being >>> misunderstood and will create a need to puzzle-out the intended >>> semantics. >> Some do, many don't. > As soon as we have to wrap a part of an expression in parentheses, parsing > the entire thing becomes more complex. Often enough it will cause the > expression to exceed whatever line length limit the codebase pledged not to > exceed, causing one line to become three. And again, making it trickier for > a regular ?ukasz to understand what's going on. At this point I think you must have a lower opinion of Python programmers than I have ;-) If adding even a dozen characters to a line makes it exceed a reasonable line-length guide, the code was almost certainly too confusingly dense to begin with. All the binding-expression examples I've given as "improvements" had _oceans_ of horizontal white space to swim in. Guido's if/elif/elif/elif/ ... complex text-processing example didn't, but because the current lack of an ability to bind-and-test in one gulp forced the `elif` parts to be ever-more-deeply-indented `if` blocks instead. So, to match your sarcasm, here's mine: try using a feature for what it's good at instead of for what it's bad at ;-) From steve at pearwood.info Thu Apr 26 02:10:00 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 26 Apr 2018 16:10:00 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> Message-ID: <20180426061000.GF7400@ando.pearwood.info> On Wed, Apr 25, 2018 at 10:14:11PM -0700, ?ukasz Langa wrote: > So only use the Pascal one in expressions. But don't forget > parentheses, otherwise it will bind the thing you probably didn't want > anyway. Binding expressions are no worse than any other expression: sometimes you need to bracket terms to change the default precedence, and sometimes you don't. And sometimes, even if we don't *need* parens, we use them anyway because it makes the expression easier to read and understand. Unless you have a language with no operator precedence at all, a purely left-to-right evaluation order like Forth or (I think?) APL, there will always be circumstances where parens are needed. Criticising binding- expressions for that reason, especially implying that we must always use parens, is simply FUD. [...] > As soon as we have to wrap a part of an expression in parentheses, > parsing the entire thing becomes more complex. Unless it becomes less complex to read and understand. I for one always have difficulty parsing complex boolean tests unless I bracket some or all of the parts, even when they're not strictly needed. Consequently I try very hard not to write complex bool tests in the first place, but when I can't avoid it, a few extra brackets really helps simplify the logic. > Often enough it will > cause the expression to exceed whatever line length limit the codebase > pledged not to exceed, causing one line to become three. Just how often are your lines within two characters of the maximum column so that adding a pair of brackets () will "often enough" put it over the limit? Seems pretty unlikely to me. This sounds really like picking at straws. -- Steve From greg.ewing at canterbury.ac.nz Thu Apr 26 02:32:32 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 26 Apr 2018 18:32:32 +1200 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: <5AE17280.9050605@canterbury.ac.nz> > On Thu, Apr 26, 2018 at 10:11 AM, Yury Selivanov > wrote: > >> my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf)) Obviously what we want here is a variant of the binding expression that also makes it a keyword argument: my_func(arg, buffer ::= [None]*get_size(), size = len(buffer)) -- Greg From greg.ewing at canterbury.ac.nz Thu Apr 26 02:52:30 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 26 Apr 2018 18:52:30 +1200 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> Message-ID: <5AE1772E.9060106@canterbury.ac.nz> ?ukasz Langa wrote: > Ha, not in Python! Here we have *different syntax* for assignments in > expressions. Well, you can also use it as a statement. But don't! This is what I least like about the proposal. We should be moving in the direction of removing warts, but here we would be *creating* one (two different assignment operators with overlapping use cases) that we won't be able to get rid of without a Python 4000 (that Guido has promised won't happen). -- Greg From rosuav at gmail.com Thu Apr 26 02:53:17 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Apr 2018 16:53:17 +1000 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: <23265.27013.849510.817475@turnbull.sk.tsukuba.ac.jp> References: <20180425034330.GP11616@ando.pearwood.info> <23265.27013.849510.817475@turnbull.sk.tsukuba.ac.jp> Message-ID: On Thu, Apr 26, 2018 at 3:54 PM, Stephen J. Turnbull wrote: > Chris Angelico writes: > > > Additionally, naming sub-parts of a large expression can assist an > > interactive debugger, providing useful display hooks and partial > > results. Without a way to capture sub-expressions inline, this > > would require refactoring of the original code; with assignment > > expressions, this merely requires the insertion of a few ``name > > :=`` markers. Removing the need to refactor reduces the likelihood > > that the code be inadvertently changed as part of debugging (a > > common cause of Heisenbugs), > > Period here preferred. > > > and is easier to dictate to a student or junior programmer. > > True but gratuitous. It's also true that it's easier to dictate to > Guido or Tim, though you might be happier if you let them refactor! > Well, true. The point isn't WHO you're dictating to, but that you can dictate it at all. "Hmm, let's see. Toss a 'foo colon-equals' in front of X, then print out what foo is." My day job involves a lot of helping students learn how to debug, so I say this kind of thing a lot (even if it's obvious to me what the problem is, because the student needs to learn debugging, not just be told what to fix). Refactoring just for the sake of a print call is overkill and potentially risky (if the student edits the wrong thing). Feel free to suggest an alternate wording. ChrisA From tim.peters at gmail.com Thu Apr 26 03:00:52 2018 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 26 Apr 2018 02:00:52 -0500 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <2AF82160-56E7-4DFB-8978-AB963C46B53B@gmail.com> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <2AF82160-56E7-4DFB-8978-AB963C46B53B@gmail.com> Message-ID: [Raymond Hettinger ] >>> After re-reading all the proposed code samples, I believe that >>> adopting the PEP will make the language harder to teach to people >>> who are not already software engineers. [Tim] >> Can you elaborate on that? [Raymond] > Just distinguishing between =, :=, and == will be a forever recurring > discussion, far more of a source of confusion than the occasional > question of why Python doesn't have embedded assignment. To be clear, is distinguishing between "=" and "==" already a forever recurring discussion in your experience? Or are you predicting that adding ":=" will create that situation? > Also, it is of concern that a number of prominent core dev > respondents to this thread have reported difficulty scanning > the posted code samples. Yes, it is - although some of the examples sucked ;-) >> I've used dozens of languages over the >> decades, most of which did have some form of embedded assignment. > Python is special, in part, because it is not one of those languages. > It has virtues that make it suitable even for elementary school children. > We can show well-written Python code to non-computer folks and walk > them through what it does without their brains melting (something I can't > do with many of the other languages I've used). There is a virtue > in encouraging simple statements that read like English sentences > organized into English-like paragraphs, presenting itself like > "executable pseudocode". It's certainly possible to stick to a subset of Python for which that's true. But I didn't mention those dozens of languages because I seek to emulate them, but to establish that I've had decades of experience with embedded assignments in a wide variety of languages and language communities. > Perl does it or C++ does it is unpersuasive. Wasn't meant to be. > Its omission from Python was always something that I thought Guido had > left-out on purpose, intentionally stepping away from constructs that would > be of help in an obfuscated Python contest. He left out lots of stuff at first, but warmed to it later. Probably the most profound: there were exactly and only 3 scopes at first: local, global, and builtin. Functions (for example) could still nest, but had no way to access names local to enclosing functions save via deep trickery. That was a noble experiment (it was a deliberate attempt to avoid complex scoping rules), but eventually proved too restrictive in practice. This is nothing compared to that ;-) But it's a tiny bit related in that biting the arbitrarily-deeply-nested-scopes bullet was aimed more at experienced programmers than at newbies. The scoping rules became far harder to explain as a result - but far more what experienced programmers expected. >> Yes, I'm a software engineer, but I've always pitched in on "help >> forums" too. > That's not really the same. I believe it! > I've taught Python to many thousands of professionals, almost > every week for over six years. That's given me a keen sense of > what is hard to teach. It's okay to not agree with my assessment, > but I would like for fruits of my experience to not be dismissed in a > single wisp of a sentence. I asked you to elaborate - I didn't dismiss anything. You merely made a raw assertion in your original message, without enough detail to even know _what_ it is you thought would be hard to teach. Your elaboration is helping. > Any one feature in isolation is usually easy to explain, but showing > how to combine them into readable, expressive code is another matter. OK, so it's not binding expressions in isolation that you expect will be hard to teach if they're added, but ... how to use them intelligently (if ever)? That's progress, if so. That part I can see having major trouble with. Even the proponents of this PEP don't always agree with each other about which examples are "good ones". > And as Yuri aptly noted, we spend more time reading code than writing code. > If some fraction of our users finds the code harder to scan > because the new syntax, then it would be a net loss for the language. It would be a tradeoff pitting their losses against others' gains, of course. I don't know how to quantify that (not even to the extent of determining the sign bit) in advance. I'm also at least as concerned about - indeed - professional software engineers as beginners. > I hesitated to join this thread because you and Guido seemed to be > pushing back so hard against anyone's who design instincts didn't favor > the new syntax. That's just vigorous debate, at least on my part. Guido gets annoyed by emotional tirades and FUD, of which there's always plenty in threads that have gone on for hundreds of messages (I don't know whether you followed any of this on python-ideas, but most arguments on python-dev were already many-times-over old by the time it first appeared here). > It would be nice to find some common ground and perhaps stipulate that the > grammar would grow in complexity, that a new operator would add to the > current zoo of operators, that the visual texture of the language would change > (and in a way that some including me do not find pleasing), and that while > simplest cases may afford a small net win, it is a certitude that the syntax > will routinely be pushed beyond our comfort zone. > > While the regex conditional example looks like a win, it is very modest win > and IMHO not worth the overall net increase language complexity. > > Like Yuri, I'll drop-out now. Hopefully, you all wind find some value > in what I had to contribute to the conversation. Absolutely! While I've slowly moved from -1 to +1 on this one, I respect your -1, and am grateful for your thoughtful elaboration. Indeed, I'll feel better now when Guido rejects it ;-) From greg.ewing at canterbury.ac.nz Thu Apr 26 03:16:28 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 26 Apr 2018 19:16:28 +1200 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> Message-ID: <5AE17CCC.5070707@canterbury.ac.nz> Tim Peters wrote: > As a statement in a program (as opposed to typed at a shell), > "a := 3" has the unnecessary (in that context) property of returning > (and discarding 3), so it's better style to use "a = 3" in that > context. That seems like a post-hoc justification. If := were the one and only assignment symbol, the compiler could easily optimise away the extra DUP_TOP or whatever is involved. -- Greg From greg.ewing at canterbury.ac.nz Thu Apr 26 02:00:07 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 26 Apr 2018 18:00:07 +1200 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: <5AE16AE7.6000609@canterbury.ac.nz> Tim Peters wrote: > To my eyes, this is genuinely harder to follow, despite its relative brevity: > > while total != (total := total + term): Not surprising, since there are at least two deeper levels of subtlety at play: 1. total isn't just naming a subexpression, it's being rebound to something that depends on its previous value. 2. Order of evaluation is being relied on to ensure that the new value of total is compared to its old value. -- Greg From steve at pearwood.info Thu Apr 26 03:29:17 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 26 Apr 2018 17:29:17 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <5AE17CCC.5070707@canterbury.ac.nz> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <5AE17CCC.5070707@canterbury.ac.nz> Message-ID: <20180426072917.GG7400@ando.pearwood.info> On Thu, Apr 26, 2018 at 07:16:28PM +1200, Greg Ewing wrote: > Tim Peters wrote: > >As a statement in a program (as opposed to typed at a shell), > >"a := 3" has the unnecessary (in that context) property of returning > >(and discarding 3), so it's better style to use "a = 3" in that > >context. > > That seems like a post-hoc justification. If := were the one > and only assignment symbol, the compiler could easily optimise > away the extra DUP_TOP or whatever is involved. Its still bad form to rely on compiler-dependent optimizations which aren't part of the language specification. And to steal an earlier idea from Tim, it is especially unfortunate if you copy data := sorted(huge_list_with_billions_of_items) from a program and paste it into your REPL, then can't type again for an hour or two. The longer I think about this, the more I am convinced that having two forms of assignment, one a statement with no return value and the other an expression with a return value, is a feature, not a wart or bug. Yes, it does add some complexity to the language, but it is *useful* complexity. If all complexity was bad, we'd still be programming by flicking toggle switches. -- Steve From tjreedy at udel.edu Thu Apr 26 03:31:13 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 26 Apr 2018 03:31:13 -0400 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: On 4/25/2018 8:20 PM, Chris Angelico wrote: > On Thu, Apr 26, 2018 at 10:11 AM, Yury Selivanov > wrote: >> Just yesterday this snippet was used on python-dev to show how great the >> new syntax is: >> >> my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf)) What strikes me as awful about this example is that len(buf) is get_size(), so the wrong value is being named and saved. 'size=len(buf)' is, in a sense, backwards. buflen = get_size() my_func(arg, buffer = [None]*buflen, size=buflen) Is standard, clear Python code. I do not see that my_func(arg, buffer=[None]*(buflen:=get_size()), size=buflen) is an improvement. -- Terry Jan Reedy From steve at pearwood.info Thu Apr 26 03:56:28 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 26 Apr 2018 17:56:28 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: <20180426075628.GH7400@ando.pearwood.info> On Thu, Apr 26, 2018 at 03:31:13AM -0400, Terry Reedy wrote: > On 4/25/2018 8:20 PM, Chris Angelico wrote: > >On Thu, Apr 26, 2018 at 10:11 AM, Yury Selivanov > > wrote: > >>Just yesterday this snippet was used on python-dev to show how great the > >>new syntax is: > >> > >> my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf)) > > What strikes me as awful about this example is that len(buf) is > get_size(), so the wrong value is being named and saved. > 'size=len(buf)' is, in a sense, backwards. Terry is absolutely right, and I'm to blame for that atrocity. Mea culpa. But Yury is misrepresenting the context in which I came up with that snippet. It was not "to show how great the new syntax is", but to try to give a *more realistic example of what we might right, instead of the toy example he had given: Yuri claimed that my_func(a=(b:=foo)) was "barely readable" and I responded: There's no advantage to using binding-expressions unless you're going to re-use the name you just defined, and that re-use will give you a hint as to what is happening: Alas, my spur of the moment example was crap, as you point out, but the point still stands: Yuri's example is mysterious, because there's a local variable b assigned to which doesn't seem to be used anywhere. It is either a mistake of some sort, simply poor code, or maybe b is used somewhere else in the function. Which seems poor style: if b is intended to be used elsewhere in the function, why not use normal assignment? Using a binding expression is a hint that it is likely *intended* to only be used in the current expression, or block. That's not a rule that the compiler ought to enforce, but it is a reasonable stylistic idiom, like using ALLCAPS for constants. At least, that's my opinion. So, crappy example or not, if I see a binding expression, that hints that the name used is needed in the current expression multiple times. It certainly should motivate me to look further ahead in the current expression to see where the newly defined variable is used next, and if it is only used once, to wonder if there has been some mistake. Whereas a stand alone assignment doesn't really give any hint (apart from vertical proximity, which is a very poor hint) as to how often and where a variable is used. -- Steve From solipsis at pitrou.net Thu Apr 26 04:17:34 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 26 Apr 2018 10:17:34 +0200 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: <20180426101734.1f7fef31@fsol> On Wed, 25 Apr 2018 18:52:34 -0700 ?ukasz Langa wrote: > > On 25 Apr, 2018, at 5:20 PM, Chris Angelico wrote: > > > > On Thu, Apr 26, 2018 at 10:11 AM, Yury Selivanov > > wrote: > >> Just yesterday this snippet was used on python-dev to show how great the > >> new syntax is: > >> > >> my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf)) > >> > >> To my eye this is an anti-pattern. One line of code was saved, but the > >> other line becomes less readable. The fact that 'buf' can be used after > >> that line means that it will be harder for a reader to trace the origin of > >> the variable, as a top-level "buf = " statement would be more visible. > > > > Making 'buf' more visible is ONLY a virtue if it's going to be used > > elsewhere. Otherwise, the name 'buf' is an implementation detail of > > the fact that this function wants both a buffer and a size. > > You're claiming that `:=` is nicer in this situation because it's less > prominent than regular assignment and thus doesn't suggest that the name > stays visible later. > > But as others said, `:=` *does* make the name visible later until the > enclosing scope ends. In fact, a large part of its appeal is that you > can use the result later (as in the `re.search()` example). Will it be > visible enough to the reaser in those cases then? > > There seems to be a conflict there. Not only, but seeing `:=` hints that something *special* is going on (some inner expression is being bound to a name). So now we have to be extra careful when reading and reviewing code written that people who like using that syntactical feature. I also wonder how long it will be before someone writes: def f(arg): global _lazy_value if predicate(arg) and (_lazy_value := frobnicate()) > arg: ... (or something similar with "nonlocal") Regards Antoine. From njs at pobox.com Thu Apr 26 04:25:31 2018 From: njs at pobox.com (Nathaniel Smith) Date: Thu, 26 Apr 2018 01:25:31 -0700 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: References: <20180425034330.GP11616@ando.pearwood.info> <23265.27013.849510.817475@turnbull.sk.tsukuba.ac.jp> Message-ID: On Wed, Apr 25, 2018 at 11:53 PM, Chris Angelico wrote: > Well, true. The point isn't WHO you're dictating to, but that you can > dictate it at all. "Hmm, let's see. Toss a 'foo colon-equals' in front > of X, then print out what foo is." My day job involves a lot of > helping students learn how to debug, so I say this kind of thing a lot > (even if it's obvious to me what the problem is, because the student > needs to learn debugging, not just be told what to fix). Refactoring > just for the sake of a print call is overkill and potentially risky > (if the student edits the wrong thing). This is overstating things slightly... the best alternative to 'foo colon-equals' isn't risky refactoring so you can call print, it's a helper like: def p(obj): print(obj) return obj that you can sprinkle inside existing expressions. Expecting new users to realize that this is possible, and a good idea, and to implement it, and get it right, while they're in the middle of being confused about basic python things, is not terribly reasonable, so it's probably underused. But there are ways we could address that. -n -- Nathaniel J. Smith -- https://vorpus.org From solipsis at pitrou.net Thu Apr 26 04:30:59 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 26 Apr 2018 10:30:59 +0200 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <5AE17CCC.5070707@canterbury.ac.nz> <20180426072917.GG7400@ando.pearwood.info> Message-ID: <20180426103059.0fdbfc4e@fsol> On Thu, 26 Apr 2018 17:29:17 +1000 Steven D'Aprano wrote: > On Thu, Apr 26, 2018 at 07:16:28PM +1200, Greg Ewing wrote: > > Tim Peters wrote: > > >As a statement in a program (as opposed to typed at a shell), > > >"a := 3" has the unnecessary (in that context) property of returning > > >(and discarding 3), so it's better style to use "a = 3" in that > > >context. > > > > That seems like a post-hoc justification. If := were the one > > and only assignment symbol, the compiler could easily optimise > > away the extra DUP_TOP or whatever is involved. > > Its still bad form to rely on compiler-dependent optimizations which > aren't part of the language specification. If such were the need, you could very well make it part of the language specification. We are talking about a trivial optimization that any runtime could easily implement (e.g. if a sequence `DUP_TOP, STORE_FAST, POP_TOP` occurs, replace it with `STORE_FAST`). Any runtime already has to implement a set of performance properties that's far less trivial than that. For example, any decent runtime is expected to provide amortized O(1) list append or dict insertion. You are breaking user expectations if you don't. > And to steal an earlier idea from Tim, it is especially unfortunate if > you copy > > data := sorted(huge_list_with_billions_of_items) > > from a program and paste it into your REPL, then can't type again for an > hour or two. Well, how do languages where assignment is an expression returning the assigned value make their REPLs work? I'm sure they don't inflict that on their users, so it's certainly a solvable problem. (abstractly: if the user is executing a top-level assignment expression, don't display the assignment result) > The longer I think about this, the more I am convinced that having two > forms of assignment, one a statement with no return value and the other > an expression with a return value, is a feature, not a wart or bug. Yet, curiously, no other language seems to replicate that "feature" :-) It's nice to innovate, but being the only one to do something may very well mean that you're doing something ridiculous. Regards Antoine. From solipsis at pitrou.net Thu Apr 26 04:36:14 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 26 Apr 2018 10:36:14 +0200 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 References: <5AE16232.9030807@canterbury.ac.nz> <20180426053417.GE7400@ando.pearwood.info> Message-ID: <20180426103614.4c34c060@fsol> On Thu, 26 Apr 2018 15:34:17 +1000 Steven D'Aprano wrote: > On Thu, Apr 26, 2018 at 05:22:58PM +1200, Greg Ewing wrote: > > ?ukasz Langa wrote: > > >What was its own assignment before > > >now is part of the logic test. This saves on vertical whitespace but makes > > >parsing and understanding logic tests harder. > > > > Another way to say this is that expressions are no longer > > restricted to being trees, but can be general DAGs, which > > require more mental effort to understand. > > Is that right? I presume you mean that there can be cycles in > expressions involving binding-expressions. If not, what do you mean? > > Can you give an example of a Python expression, involving PEP 572 > binding-expressions, that is not a tree but a more general DAG or that > contains cycles? Depends if you mean a graph between names or values? If between names, you can even have cycles AFAICT: ((a: = a + b), (b: = a)) Regards Antoine. From J.Demeyer at UGent.be Thu Apr 26 05:04:07 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Thu, 26 Apr 2018 11:04:07 +0200 Subject: [Python-Dev] PEP 573 -- Module State Access from C Extension Methods In-Reply-To: References: <2854d9cdad73400aa968a084f40908d7@xmail101.UGent.be> <5ADE202B.2060501@UGent.be> <43a667611aa04e60a66372d75c759fa6@xmail101.UGent.be> <5ADEE826.9070806@UGent.be> <9333b45af89e47b4a0b349f284894435@xmail101.UGent.be> <5ADF405C.4060607@UGent.be> <5ADF6582.5080301@UGent.be> <15262b8f09724d1d801cc81289aa4ba2@xmail101.UGent.be> <5AE0CCEA.2030806@UGent.be> Message-ID: <5AE19607.7020006@UGent.be> > - In Python code, __objclass__ should be the defining class, not the module. Indeed. My idea would be to add an accessor __parent__ returning the m_parent field (whatever it is) and then implement __objclass__ as something like: @property def __objclass__(self): parent = getattr(self, "__parent__", None) if isinstance(parent, type): return parent else: raise AttributeError In PEP 575, I don't plan to add a Python attribute specifically for getting the defining module: I'll leave that to PEP 573. From rosuav at gmail.com Thu Apr 26 05:19:05 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Apr 2018 19:19:05 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <20180426103059.0fdbfc4e@fsol> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <5AE17CCC.5070707@canterbury.ac.nz> <20180426072917.GG7400@ando.pearwood.info> <20180426103059.0fdbfc4e@fsol> Message-ID: On Thu, Apr 26, 2018 at 6:30 PM, Antoine Pitrou wrote: > On Thu, 26 Apr 2018 17:29:17 +1000 > Steven D'Aprano wrote: >> On Thu, Apr 26, 2018 at 07:16:28PM +1200, Greg Ewing wrote: >> > Tim Peters wrote: >> > >As a statement in a program (as opposed to typed at a shell), >> > >"a := 3" has the unnecessary (in that context) property of returning >> > >(and discarding 3), so it's better style to use "a = 3" in that >> > >context. >> > >> > That seems like a post-hoc justification. If := were the one >> > and only assignment symbol, the compiler could easily optimise >> > away the extra DUP_TOP or whatever is involved. >> >> Its still bad form to rely on compiler-dependent optimizations which >> aren't part of the language specification. > > If such were the need, you could very well make it part of the language > specification. We are talking about a trivial optimization that any > runtime could easily implement (e.g. if a sequence `DUP_TOP, STORE_FAST, > POP_TOP` occurs, replace it with `STORE_FAST`). Not at the REPL, no. At the REPL, you need to actually print out that value. It's semantically different. > Any runtime already has to implement a set of performance properties > that's far less trivial than that. For example, any decent runtime is > expected to provide amortized O(1) list append or dict insertion. You > are breaking user expectations if you don't. You assume that, but it isn't always the case. Did you know, for instance, that string subscripting (an O(1) operation in CPython) is allowed to be O(n) in other Python implementations? Now you do. ChrisA From solipsis at pitrou.net Thu Apr 26 05:37:59 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 26 Apr 2018 11:37:59 +0200 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <5AE17CCC.5070707@canterbury.ac.nz> <20180426072917.GG7400@ando.pearwood.info> <20180426103059.0fdbfc4e@fsol> Message-ID: <20180426113759.6e2e8212@fsol> On Thu, 26 Apr 2018 19:19:05 +1000 Chris Angelico wrote: > > > > If such were the need, you could very well make it part of the language > > specification. We are talking about a trivial optimization that any > > runtime could easily implement (e.g. if a sequence `DUP_TOP, STORE_FAST, > > POP_TOP` occurs, replace it with `STORE_FAST`). > > Not at the REPL, no. At the REPL, you need to actually print out that > value. It's semantically different. The REPL compiles expressions in a different mode than regular modules, so that's entirely a strawman. The REPL doesn't care that it will spend a fraction of microseconds executing two additional bytecodes before presenting something to the user. > > Any runtime already has to implement a set of performance properties > > that's far less trivial than that. For example, any decent runtime is > > expected to provide amortized O(1) list append or dict insertion. You > > are breaking user expectations if you don't. > > You assume that, but it isn't always the case. Yeah, so what? > Did you know, for instance, that string subscripting (an O(1) operation in CPython) is > allowed to be O(n) in other Python implementations? Why would I give a sh*t? Here, we are not talking about a non-trivial design decision such as how to represent string values internally (utf-8 vs. fixed-width, etc.). We are talking about optimizing a trivial bytecode sequence into another more trivial bytecode sequence. CPython can easily do it. PyPy can easily do it. Other runtimes can easily do it. If some implementation is significantly slower because it can't optimize away pointless DUP_TOPs *and* it implements DUP_TOP inefficiently enough to have user-noticeable effect, then it's either 1) a deliberate toy, a proof-of-concept not meant for serious use, or 2) a pile of crap produced by incompetent people. So there's zero reason to bother about efficiency issues here. Regards Antoine. From steve at holdenweb.com Thu Apr 26 05:59:43 2018 From: steve at holdenweb.com (Steve Holden) Date: Thu, 26 Apr 2018 10:59:43 +0100 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: On Wed, Apr 25, 2018 at 9:55 PM, ?ukasz Langa wrote: > > > On 25 Apr, 2018, at 1:28 PM, Guido van Rossum wrote: > > > > You don't seem to grasp the usability improvements this will give. I > hear you but at this point appeals to Python's "Zen" don't help you. > > This reads dismissive to me. I did read the PEP and followed the > discussion on > python-dev. I referred to PEP 20 because it distills what's unique about > the > value proposition of Python. It's our shared vocabulary. > > ?Perhaps so, but no PEP is chiselled in stone, and I would suggest that PEP 20 is the least authoritative from a didactic point of view. ? > Can you address the specific criticism I had? To paraphrase it without PEP > 20 > jargon: > > > (name := expression) makes code less uniform. It inserts more > information > > into a place that is already heavily packed with information (logic > tests). > > ?One could argue the same about list comprehensions if one chose: they make code denser (by expressing the same algorithm in a shorter spelling). I'm not sure what you mean by "less uniform."? -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Thu Apr 26 06:20:31 2018 From: steve at holdenweb.com (Steve Holden) Date: Thu, 26 Apr 2018 11:20:31 +0100 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <20180426075628.GH7400@ando.pearwood.info> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <20180426075628.GH7400@ando.pearwood.info> Message-ID: On Thu, Apr 26, 2018 at 8:56 AM, Steven D'Aprano wrote: > On Thu, Apr 26, 2018 at 03:31:13AM -0400, Terry Reedy wrote: > > On 4/25/2018 8:20 PM, Chris Angelico wrote: > > >On Thu, Apr 26, 2018 at 10:11 AM, Yury Selivanov > > > wrote: > > >>Just yesterday this snippet was used on python-dev to show how great > the > > >>new syntax is: > > >> > > >> my_func(arg, buffer=(buf := [None]*get_size()), > size=len(buf)) > > > > What strikes me as awful about this example is that len(buf) is > > get_size(), so the wrong value is being named and saved. > > 'size=len(buf)' is, in a sense, backwards. > > Terry is absolutely right, and I'm to blame for that atrocity. Mea > culpa. > > ?Perhaps a better spelling would be my_func(arg, buffer=[None]*(buflen := get_size()), size=buflen) ?[...] -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Thu Apr 26 06:34:29 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 26 Apr 2018 22:34:29 +1200 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <20180426103059.0fdbfc4e@fsol> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <5AE17CCC.5070707@canterbury.ac.nz> <20180426072917.GG7400@ando.pearwood.info> <20180426103059.0fdbfc4e@fsol> Message-ID: <5AE1AB35.6090609@canterbury.ac.nz> Antoine Pitrou wrote: > Well, how do languages where assignment is an expression returning the > assigned value make their REPLs work? I'm sure they don't inflict that > on their users, so it's certainly a solvable problem. I can't think of any such language that has a REPL offhand, but here's a possible solution: x := expr # doesn't print anything (x := expr) # prints the result I.e. special-case a stand-alone assignment, but allow overriding that with parens if needed. -- Greg From greg.ewing at canterbury.ac.nz Thu Apr 26 06:39:31 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 26 Apr 2018 22:39:31 +1200 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <20180426103614.4c34c060@fsol> References: <5AE16232.9030807@canterbury.ac.nz> <20180426053417.GE7400@ando.pearwood.info> <20180426103614.4c34c060@fsol> Message-ID: <5AE1AC63.3090008@canterbury.ac.nz> Antoine Pitrou wrote: > Depends if you mean a graph between names or values? > > If between names, you can even have cycles AFAICT: > > ((a: = a + b), (b: = a)) I was thinking more of the dataflow graph. That's not a cycle between *values*, since the new values being bound are calculated from the previous values of the names. -- Greg From turnbull.stephen.fw at u.tsukuba.ac.jp Thu Apr 26 07:23:45 2018 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 26 Apr 2018 20:23:45 +0900 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: References: <20180425034330.GP11616@ando.pearwood.info> <23265.27013.849510.817475@turnbull.sk.tsukuba.ac.jp> Message-ID: <23265.46785.367828.16582@turnbull.sk.tsukuba.ac.jp> Chris Angelico writes: > Well, true. The point isn't WHO you're dictating to, By "period here preferred," I meant I think it's mostly a waste of space to mention dictation at all in that document. But it's not a big deal to me, so how about changing "a student or junior programmer" to "another programmer"? From turnbull.stephen.fw at u.tsukuba.ac.jp Thu Apr 26 07:27:26 2018 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 26 Apr 2018 20:27:26 +0900 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> Message-ID: <23265.47006.283579.443517@turnbull.sk.tsukuba.ac.jp> ?ukasz Langa writes: > > On 25 Apr, 2018, at 5:20 PM, Chris Angelico wrote: > You're claiming that `:=` is nicer in this situation because it's less > prominent than regular assignment and thus doesn't suggest that the name > stays visible later. FWIW, I read what he wrote as "*assuming* buf is not going to be used later", and thus took a more nuanced idea from it: Use a separate statement when you do use it later, and use a binding expression when its scope is *in fact* only that line. BTW, I don't find the "it could be misused, so it will be misused" argument persuasive. I agree it's true, but don't think it matters, per the "consenting adults" principle, since binding expressions have other, important, use cases. We could add a statement to PEP 8 mildly deprecating this particular use. How about this: In some examples, the binding expression is used in function arguments where one argument depends on an earlier one, such as foo(buffer=(buf := [None]*get_size()), size=len(buf)) In examples like this one, it is preferable where possible to refactor the function to calculate the dependent variable itself, or to use an assignment statement to define the bound variable. The latter style is *strongly* preferred when the bound variable will be used later in the scope. Steve From turnbull.stephen.fw at u.tsukuba.ac.jp Thu Apr 26 07:35:20 2018 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 26 Apr 2018 20:35:20 +0900 Subject: [Python-Dev] Is PEP 572 really the most effective way to solve the problems it's targeting? In-Reply-To: <20180426040332.GD7400@ando.pearwood.info> References: <162ffcf2f98.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> <20180426040332.GD7400@ando.pearwood.info> Message-ID: <23265.47480.848418.283150@turnbull.sk.tsukuba.ac.jp> Steven D'Aprano writes: > On Wed, Apr 25, 2018 at 09:36:31PM -0500, Ryan Gonzalez wrote: > > Now, what's the common theme here? **Declarations should be separate from > > expressions.** > > Declarations and assignments are not the same thing. Ryan mostly meant "initialization" rather than "declaration", I suspect. In C's for() statement, the first clause in parentheses is initialization, the third is assignment. FWIW, I had been thinking the same thing (that what is really wanted is initialization clauses in loop statements) but had no concrete suggestions. Now I too have warmed to the binding expression approach (partly because Guido has, so I'm preparing for the inevitable :^), mostly because of Tim's "humorous observation" about use in printf debugging. The fact that there is this use case independent of block variable initialization (ie, in a loop or if statement) is quite attractive to me. It's true, as I think Antoine pointed out, that it's easy enough to define a wrapper function that prints a value and returns it. But that's not as flexible, and if you have more than one variable to "watch", you either need an argument to the wrapper to provide the variable's name, or different functions for different variables. From solipsis at pitrou.net Thu Apr 26 07:42:33 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 26 Apr 2018 13:42:33 +0200 Subject: [Python-Dev] Is PEP 572 really the most effective way to solve the problems it's targeting? References: <162ffcf2f98.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> <20180426040332.GD7400@ando.pearwood.info> <23265.47480.848418.283150@turnbull.sk.tsukuba.ac.jp> Message-ID: <20180426134233.21eb4165@fsol> On Thu, 26 Apr 2018 20:35:20 +0900 "Stephen J. Turnbull" wrote: > Steven D'Aprano writes: > > On Wed, Apr 25, 2018 at 09:36:31PM -0500, Ryan Gonzalez wrote: > > > > Now, what's the common theme here? **Declarations should be separate from > > > expressions.** > > > > Declarations and assignments are not the same thing. > > Ryan mostly meant "initialization" rather than "declaration", I > suspect. In C's for() statement, the first clause in parentheses is > initialization, the third is assignment. > > FWIW, I had been thinking the same thing (that what is really wanted > is initialization clauses in loop statements) but had no concrete > suggestions. Now I too have warmed to the binding expression approach > (partly because Guido has, so I'm preparing for the inevitable :^), > mostly because of Tim's "humorous observation" about use in printf > debugging. The fact that there is this use case independent of block > variable initialization (ie, in a loop or if statement) is quite > attractive to me. > > It's true, as I think Antoine pointed out, that it's easy enough to > define a wrapper function that prints a value and returns it. That wasn't me, but I agree with the idea anyway :-) Regards Antoine. From mertz at gnosis.cx Thu Apr 26 07:49:02 2018 From: mertz at gnosis.cx (David Mertz) Date: Thu, 26 Apr 2018 11:49:02 +0000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <2AF82160-56E7-4DFB-8978-AB963C46B53B@gmail.com> Message-ID: > > [Raymond Hettinger ] > > Python is special, in part, because it is not one of those languages. > > It has virtues that make it suitable even for elementary school children. > > We can show well-written Python code to non-computer folks and walk > > them through what it does without their brains melting (something I can't > > do with many of the other languages I've used). There is a virtue > > in encouraging simple statements that read like English sentences > > organized into English-like paragraphs, presenting itself like > > "executable pseudocode". While this is true and good for most Python code, can you honestly explain asyncio code with async/await to these non-programmers?! What about the interfaces between async and synchronous portions? I've been programming for 40 years, in Python for 20 of them. I cannot read any block of async code without thinking VERY SLOWLY about what's going on, then getting it wrong half the time. I even teach Python almost as much as Raymond does. There's a certain hand-waving approach to teaching async/await where you say not to worry about those keywords, and just assume the blocks are coordinated "somehow, behind the scenes." That's not awful for reading *working* code, but doesn't let you write it. I'm not saying binding expressions are likewise reserved for a special but important style of programming. If included, I expect them to occur more-or-less anywhere. So Raymond's concern about teachability is more pressing (I've only taught async twice, and I know Raymond's standard course doesn't do it, all the other code is unaffected by that unused 'await' lurking in the syntax). Still, there are good reasons why not all Python code is aimed at non-computer folks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Thu Apr 26 08:09:57 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 26 Apr 2018 22:09:57 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <20180426101734.1f7fef31@fsol> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <20180426101734.1f7fef31@fsol> Message-ID: <20180426120955.GI7400@ando.pearwood.info> On Thu, Apr 26, 2018 at 10:17:34AM +0200, Antoine Pitrou wrote: > I also wonder how long it will be before someone writes: > > def f(arg): > global _lazy_value > if predicate(arg) and (_lazy_value := frobnicate()) > arg: > ... > > (or something similar with "nonlocal") What if they did? I don't think that's especially worse than any other use of a global, including the current version: def f(arg): global _lazy_value _lazy_value = frobnicate() if predicate(arg) and _lazy_value > arg: ... I'm not putting either version forward as paragons of Pythonic code, but honestly, they're not so awful that we should reject this PEP because of the risk that somebody will do this. People write crappy code and misuse features all the time. I cannot count the number of times people write list comps just for the side-effects, intentionally throwing away the resulting list: [alist.sort() for alist in list_of_lists] [print(alist) for alist in list_of_lists] and the honest truth is that when I'm engaged in exploratory coding in the REPR, sometimes if I'm feeling lazy I'll do it too. The world goes on, and comprehensions are still useful even if people sometimes misuse them. -- Steve From steve at pearwood.info Thu Apr 26 08:31:53 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 26 Apr 2018 22:31:53 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <5AE1AB35.6090609@canterbury.ac.nz> References: <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <5AE17CCC.5070707@canterbury.ac.nz> <20180426072917.GG7400@ando.pearwood.info> <20180426103059.0fdbfc4e@fsol> <5AE1AB35.6090609@canterbury.ac.nz> Message-ID: <20180426123153.GJ7400@ando.pearwood.info> On Thu, Apr 26, 2018 at 10:34:29PM +1200, Greg Ewing wrote: > Antoine Pitrou wrote: > >Well, how do languages where assignment is an expression returning the > >assigned value make their REPLs work? I'm sure they don't inflict that > >on their users, so it's certainly a solvable problem. > > I can't think of any such language that has a REPL > offhand, but here's a possible solution: Here's the Rhino Javascript REPL: [steve at ando ~]$ rhino Rhino 1.7 release 0.7.r2.3.el5_6 2011 05 04 js> x = (a = 99) + 1 100 Here's the standard Ruby REPL: [steve at ando ~]$ irb irb(main):001:0> x = (a = 99) + 1 => 100 So both of these REPLs do print the result of the expression. R, on the other hand, doesn't print the results of assignment expressions: > x <- (a <- 99) + 1 > c(x, a) [1] 100 99 Rhino, however, does suppress printing if you prefix the variable with "var" or "const". -- Steve From rosuav at gmail.com Thu Apr 26 08:39:07 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Apr 2018 22:39:07 +1000 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: <23265.46785.367828.16582@turnbull.sk.tsukuba.ac.jp> References: <20180425034330.GP11616@ando.pearwood.info> <23265.27013.849510.817475@turnbull.sk.tsukuba.ac.jp> <23265.46785.367828.16582@turnbull.sk.tsukuba.ac.jp> Message-ID: On Thu, Apr 26, 2018 at 9:23 PM, Stephen J. Turnbull wrote: > Chris Angelico writes: > > > Well, true. The point isn't WHO you're dictating to, > > By "period here preferred," I meant I think it's mostly a waste of > space to mention dictation at all in that document. But it's not a > big deal to me, so how about changing "a student or junior programmer" > to "another programmer"? > > Sure, that works. ChrisA From mertz at gnosis.cx Thu Apr 26 07:31:04 2018 From: mertz at gnosis.cx (David Mertz) Date: Thu, 26 Apr 2018 11:31:04 +0000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> Message-ID: FWIW, the combination of limiting the PEP to binding expressions and the motivating example of sequential if/elif tests that each need to utilize an expression in their body (e.g. matching various regexen by narrowing, avoiding repeated indent) gets me to +1. I still think the edge case changes to comprehension semantics is needless for this PEP. However, it concerns a situation I don't think I've ever encountered in the wild, and certainly never relied on the old admittedly odd behavior. On Thu, Apr 26, 2018, 2:01 AM Tim Peters wrote: > Yes, binding expressions in the current PEP support an extremely > limited subset of what Python's assignment statements support.[...] > Guido's if/elif/elif/elif/ ... complex text-processing example didn't, > but because the current lack of an ability to bind-and-test in one > gulp forced the `elif` parts to be ever-more-deeply-indented `if` > blocks instead. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lkb.teichmann at gmail.com Thu Apr 26 09:13:19 2018 From: lkb.teichmann at gmail.com (Martin Teichmann) Date: Thu, 26 Apr 2018 15:13:19 +0200 Subject: [Python-Dev] PEP 572: Why not := as standard assignment operator? Message-ID: Hi list, when reading PEP 572 I actually liked it a lot - I think it's actually a cool idea. I think it's actually that cool an idea that it should be made the default way of doing an assignment, over time phasing out the good ole =. This would have several benefits: - people wouldn't have to worry about two different options - different things would have a different look: assignment is :=, keyword args is =, while comparison is ==. Especially beginners would benefit from this clarity. in this case, for sure, we should make it possible to chain :=s, for example by making it bind right-to-left, so that a := b := 3 would be a := (b := 3) I'm sorry if somebody brought that up already, but the discussion has grown so huge that I couldn't read through it entirely. Greetings Martin From thomas at python.org Thu Apr 26 09:32:38 2018 From: thomas at python.org (Thomas Wouters) Date: Thu, 26 Apr 2018 15:32:38 +0200 Subject: [Python-Dev] PEP 573 -- Module State Access from C Extension Methods In-Reply-To: References: Message-ID: Thanks for working on this, Marcel (and Petr). This looks like an ambitious intern project :) Couple of questions and comments in-line. On Mon, Apr 23, 2018 at 12:36 PM, Marcel Plch wrote: > Hello, > I am an intern at Red Hat mentored by Petr Viktorin. As a part of my > internship, I learned the CPython internals and how to contribute > to the CPython interpreter. > > As a result, I have prepared PEP 573, which solves some problems > that PEP 489 (Multi-phase extension module initialization) has left open. > Specifically, this PEP proposes a way to access per-module state from > methods of > built-in and extension types. > Like PEP 489, it aims to make subinterpreter-friendly built-in/extension > modules > easier to create. > > A big problem found when converting many modules to PEP 489 multi-phase > initialization is subinterpreter-friendly access to exception > types defined in built-in/extension modules. > This PEP solves this by introducing "immutable exception types". > The current implementation requires one new type flag and two new > pointers in the heap type structure. > It should be possible to remove eiher the flag or one of the two pointers, > if we agree on the other mechanics in the PEP . > > > =================== > > PEP: 573 > Title: Module State Access from C Extension Methods > Version: $Revision$ > Last-Modified: $Date$ > Author: Petr Viktorin , > Nick Coghlan , > Eric Snow , > Marcel Plch > Discussions-To: import-sig at python.org > Status: Active > Type: Process > Content-Type: text/x-rst > Created: 02-Jun-2016 > Python-Version: 3.8 > Post-History: > > > Abstract > ======== > > This PEP proposes to add a way for CPython extension methods to access > context such as > the state of the modules they are defined in. > > This will allow extension methods to use direct pointer dereferences > rather than PyState_FindModule for looking up module state, reducing > or eliminating the > performance cost of using module-scoped state over process global state. > > This fixes one of the remaining roadblocks for adoption of PEP 3121 > (Extension > module initialization and finalization) and PEP 489 > (Multi-phase extension module initialization). > > Additionaly, support for easier creation of immutable exception > classes is added. > I'm not a fan of using 'immutable' here, or in the API function name. I understand the types are to some extent immutable (apart from their refcount, I assume), but I think it's going to be too easy to confuse it with types whose *instances* are immutable. (We do occasionally say things like "tuples are an immutable type".) Since the point is that they behave like statically defined ones, perhaps 'Static' would be a reasonable replacement. This removes the need for keeping per-module state if it would only be used > for exception classes. > > While this PEP takes an additional step towards fully solving the > problems that PEP 3121 and PEP 489 started > tackling, it does not attempt to resolve *all* remaining concerns. In > particular, accessing the module state from slot methods (``nb_add``, > etc) remains slower than accessing that state from other extension > methods. > > > Terminology > =========== > > Process-Global State > -------------------- > > C-level static variables. Since this is very low-level > memory storage, it must be managed carefully. > > Per-module State > ---------------- > > State local to a module object, allocated dynamically as part of a > module object's initialization. This isolates the state from other > instances of the module (including those in other subinterpreters). > > Accessed by ``PyModule_GetState()``. > > > Static Type > ----------- > > A type object defined as a C-level static variable, i.e. a compiled-in > type object. > > A static type needs to be shared between module instances and has no > information of what module it belongs to. > Static types do not have ``__dict__`` (although their instances might). > > Heap Type > --------- > > A type object created at run time. > > > Rationale > ========= > > PEP 489 introduced a new way to initialize extension modules, which brings > several advantages to extensions that implement it: > > * The extension modules behave more like their Python counterparts. > * The extension modules can easily support loading into pre-existing > module objects, which paves the way for extension module support for > ``runpy`` or for systems that enable extension module reloading. > * Loading multiple modules from the same extension is possible, which > makes testing module isolation (a key feature for proper > sub-interpreter > support) possible from a single interpreter. > > The biggest hurdle for adoption of PEP 489 is allowing access to module > state > from methods of extension types. > Currently, the way to access this state from extension methods is by > looking up the module via > ``PyState_FindModule`` (in contrast to module level functions in > extension modules, which > receive a module reference as an argument). > However, ``PyState_FindModule`` queries the thread-local state, making > it relatively > costly compared to C level process global access and consequently > deterring module authors from using it. > > Also, ``PyState_FindModule`` relies on the assumption that in each > subinterpreter, there is at most one module corresponding to > a given ``PyModuleDef``. This does not align well with Python's import > machinery. Since PEP 489 aimed to fix that, the assumption does > not hold for modules that use multi-phase initialization, so > ``PyState_FindModule`` is unavailable for these modules. > > A faster, safer way of accessing module-level state from extension methods > is needed. > > > Immutable Exception Types > ------------------------- > > For isolated modules to work, any class whose methods touch module state > must be a heap type, so that each instance of a module can have its own > type object. With the changes proposed in this PEP, heap type instances > will > have access to module state without global registration. But, to create > instances of heap types, one will need the module state in order to > get the type object corresponding to the appropriate module. > In short, heap types are "viral" ? anything that ?touches? them must > itself be > a heap type. > > Curently, most exception types, apart from the ones in ``builtins``, are > heap types. This is likely simply because there is a convenient way > to create them: ``PyErr_NewException``. > Heap types generally have a mutable ``__dict__``. > In most cases, this mutability is harmful. For example, exception types > from the ``sqlite`` module are mutable and shared across subinterpreters. > This allows "smuggling" values to other subinterpreters via attributes of > ``sqlite3.Error``. > > Moreover, since raising exceptions is a common operation, and heap types > will be "viral", ``PyErr_NewException`` will tend to "infect" the module > with "heap type-ness" ? at least if the module decides play well with > subinterpreters/isolation. > Many modules could go without module state > entirely if the exception classes were immutable. > > To solve this problem, a new function for creating immutable exception > types > is proposed. > > > Background > =========== > > The implementation of a Python method may need access to one or more of > the following pieces of information: > > * The instance it is called on (``self``) > * The underlying function > * The class the method was defined in > * The corresponding module > * The module state > > In Python code, the Python-level equivalents may be retrieved as:: > > import sys > > def meth(self): > instance = self > module_globals = globals() > module_object = sys.modules[__name__] # (1) > underlying_function = Foo.meth # (1) > defining_class = Foo # (1) > defining_class = __class__ # (2) > > .. note:: > > The defining class is not ``type(self)``, since ``type(self)`` might > be a subclass of ``Foo``. > > The statements marked (1) implicitly rely on name-based lookup via the > function's ``__globals__``: > either the ``Foo`` attribute to access the defining class and Python > function object, or ``__name__`` to find the module object in > ``sys.modules``. > In Python code, this is feasible, as ``__globals__`` is set > appropriately when the function definition is executed, and > even if the namespace has been manipulated to return a different > object, at worst an exception will be raised. > > The ``__class__`` closure, (2), is a safer way to get the defining > class, but it still relies on ``__closure__`` being set appropriately. > > By contrast, extension methods are typically implemented as normal C > functions. > This means that they only have access to their arguments and C level > thread-local > and process-global states. Traditionally, many extension modules have > stored > their shared state in C-level process globals, causing problems when: > > * running multiple initialize/finalize cycles in the same process > * reloading modules (e.g. to test conditional imports) > * loading extension modules in subinterpreters > > PEP 3121 attempted to resolve this by offering the > ``PyState_FindModule`` API, but this still has significant problems > when it comes to extension methods (rather than module level > functions): > > * it is markedly slower than directly accessing C-level process-global > state > * there is still some inherent reliance on process global state > that means it still doesn't reliably handle module reloading > > It's also the case that when looking up a C-level struct such as > module state, supplying > an unexpected object layout can crash the interpreter, so it's > significantly more important to ensure that extension > methods receive the kind of object they expect. > > Proposal > ======== > > Currently, a bound extension method (``PyCFunction`` or > ``PyCFunctionWithKeywords``) receives only > ``self``, and (if applicable) the supplied positional and keyword > arguments. > > While module-level extension functions already receive access to the > defining module object via their > ``self`` argument, methods of extension types don't have that luxury: > they receive the bound instance > via ``self``, and hence have no direct access to the defining class or > the module level state. > > The additional module level context described above can be made > available with two changes. > Both additions are optional; extension authors need to opt in to start > using them: > > * Add a pointer to the module to heap type objects. > > * Pass the defining class to the underlying C function. > > The defining class is readily available at the time built-in > method object (``PyCFunctionObject``) is created, so it can be stored > in a new struct that extends ``PyCFunctionObject``. > > The module state can then be retrieved from the module object via > ``PyModule_GetState``. > > Note that this proposal implies that any type whose method needs to access > per-module state must be a heap type, rather than a static type. > > This is necessary to support loading multiple module objects from a single > extension: a static type, as a C-level global, has no information about > which module it belongs to. > > > Slot methods > ------------ > > The above changes don't cover slot methods, such as ``tp_iter`` or > ``nb_add``. > > The problem with slot methods is that their C API is fixed, so we can't > simply add a new argument to pass in the defining class. > Two possible solutions have been proposed to this problem: > > * Look up the class through walking the MRO. > This is potentially expensive, but will be useful if performance is > not > a problem (such as when raising a module-level exception). > * Storing a pointer to the defining class of each slot in a separate > table, > ``__typeslots__`` [#typeslots-mail]_. This is technically > feasible and fast, > but quite invasive. > > Due to the invasiveness of the latter approach, this PEP proposes > adding an MRO walking > helper for use in slot method implementations, deferring the more > complex alternative > as a potential future optimisation. Modules affected by this concern > also have the > option of using thread-local state or PEP 567 context variables, or > else defining their > own reload-friendly lookup caching scheme. > I do not believe walking the MRO is going to work without reworking the implementation of types, specifically how typeobject.c deals with slots of subclasses: in some cases copies the slots from the base class (see inherit_slots() and from where it's called). I believe this would cause problems if, for example, you define type X in module A, subclass it from type Y in module B without overriding the slot, and try to find the module object for A from the slot implementation. I don't think copying slots is a requirement for the desired semantics, but it's going to be fairly involved to rewrite it to do something else. There's also backward-compatibility to consider: third-party libraries can be inheriting from builtin types (e.g. numpy does this extensively) using the same copying-slot mechanism, which means those builtin types can't use the MRO walking to find their module without breaking compatibility with those third-party libraries. > > > Immutable Exception Types > ------------------------- > > To faciliate creating static exception classes, a new function is proposed: > ``PyErr_PrepareImmutableException``. It will work similarly to > ``PyErr_NewExceptionWithDoc`` > but will take a ``PyTypeObject **`` pointer, which points to a > ``PyTypeObject *`` that is > either ``NULL`` or an initialized ``PyTypeObject``. > This pointer may be declared in process-global state. The function will > then > allocate the object and will keep in mind that already existing exception > should not be overwritten. > > The extra indirection makes it possible to make > ``PyErr_PrepareImmutableException`` > part of the stable ABI by having the Python interpreter, rather than > extension code, > allocate the ``PyTypeObject``. > > > Specification > ============= > > Adding module references to heap types > -------------------------------------- > > The ``PyHeapTypeObject`` struct will get a new member, ``PyObject > *ht_module``, > that can store a pointer to the module object for which the type was > defined. > It will be ``NULL`` by default, and should not be modified after the type > object is created. > > A new factory method will be added for creating modules:: > > PyObject* PyType_FromModuleAndSpec(PyObject *module, > PyType_Spec *spec, > PyObject *bases) > > This acts the same as ``PyType_FromSpecWithBases``, and additionally sets > ``ht_module`` to the provided module object. > > Additionally, an accessor, ``PyObject * PyType_GetModule(PyTypeObject *)`` > will be provided. > It will return the ``ht_module`` if a heap type with module pointer set > is passed in, otherwise it will set a SystemError and return NULL. > > Usually, creating a class with ``ht_module`` set will create a reference > cycle involving the class and the module. > This is not a problem, as tearing down modules is not a > performance-sensitive > operation (and module-level functions typically also create reference > cycles). > The existing "set all module globals to None" code that breaks function > cycles > through ``f_globals`` will also break the new cycles through ``ht_module``. > > > Passing the defining class to extension methods > ----------------------------------------------- > > A new style of C-level functions will be added to the current selection of > ``PyCFunction`` and ``PyCFunctionWithKeywords``:: > > PyObject *PyCMethod(PyObject *self, > PyTypeObject *defining_class, > PyObject *args, PyObject *kwargs) > > A new method object flag, ``METH_METHOD``, will be added to signal that > the underlying C function is ``PyCMethod``. > > To hold the extra information, a new structure extending > ``PyCFunctionObject`` > will be added:: > > typedef struct { > PyCFunctionObject func; > PyTypeObject *mm_class; /* Passed as 'defining_class' arg to > the C func */ > } PyCMethodObject; > > To allow passing the defining class to the underlying C function, a change > to private API is required, now ``_PyMethodDef_RawFastCallDict`` and > ``_PyMethodDef_RawFastCallKeywords`` will receive ``PyTypeObject *cls`` > as one of their arguments. > > A new macro ``PyCFunction_GET_CLASS(cls)`` will be added for easier > access to mm_class. > > Method construction and calling code and will be updated to honor > ``METH_METHOD``. > > > Argument Clinic > --------------- > > To support passing the defining class to methods using Argument Clinic, > a new converter will be added to clinic.py: ``defining_class``. > > Each method may only have one argument using this converter, and it must > appear after ``self``, or, if ``self`` is not used, as the first argument. > The argument will be of type ``PyTypeObject *``. > > When used, Argument Clinic will select ``METH_METHOD`` as the calling > convention. > The argument will not appear in ``__text_signature__``. > > This will be compatible with ``__init__`` and ``__new__`` methods, where an > MRO walker will be used to pass the defining class from clinic generated > code to the user's function. > > > Slot methods > ------------ > > To allow access to per-module state from slot methods, an MRO walker > will be implemented:: > > PyTypeObject *PyType_DefiningTypeFromSlotFunc(PyTypeObject *type, > int slot, void *func) > > The walker will go through bases of heap-allocated ``type`` > and search for class that defines ``func`` at its ``slot``. > > The ``func`` needs not to be inherited by ``type``, only requirement > for the walker to find the defining class is that the defining class > must be heap-allocated. > > On failure, exception is set and NULL is returned. > > > Static exceptions > ----------------- > > A new function will be added:: > > int PyErr_PrepareImmutableException(PyTypeObject **exc, > const char *name, > const char *doc, > PyObject *base) > > Creates an immutable exception type which can be shared > across multiple module objects. > How is this going to deal with type.__subclasses__()? Is re-using the static type object between reloads and sub-interpreters important enough to warrant the different behaviour? What if sub-interpreters end up wanting to disallow sharing objects between them? If the type already exists (determined by a process-global pointer, > ``*exc``), skip the initialization and only ``INCREF`` it. > > If ``*exc`` is NULL, the function will > allocate a new exception type and initialize it using given parameters > the same way ``PyType_FromSpecAndBases`` would. > The ``doc`` and ``base`` arguments may be ``NULL``, defaulting to a > missing docstring and ``PyExc_Exception`` base class, respectively. > The exception type's ``tp_flags`` will be set to values common to > built-in exceptions and the ``Py_TPFLAGS_HEAP_IMMUTABLE`` flag (see below) > will be set. > On failure, ``PyErr_PrepareImmutableException`` will set an exception > and return -1. > > If called with an initialized exception type (``*exc`` > is non-NULL), the function will do nothing but incref ``*exc``. > > A new flag, ``Py_TPFLAGS_HEAP_IMMUTABLE``, will be added to prevent > mutation of the type object. This makes it possible to > share the object safely between multiple interpreters. > This flag is checked in ``type_setattro`` and blocks > setting of attributes when set, similar to built-in types. > > A new pointer, ``ht_moduleptr``, will be added to heap types to store > ``exc``. > > On deinitialization of the exception type, ``*exc`` will be set to > ``NULL``. > This makes it safe for ``PyErr_PrepareImmutableException`` to check if > the exception was already initialized. > > PyType_offsets > -------------- > > Some extension types are using instances with ``__dict__`` or > ``__weakref__`` > allocated. Currently, there is no way of passing offsets of these through > ``PyType_Spec``. To allow this, a new structure and a spec slot are > proposed. > > A new structure, ``PyType_offsets``, will have two members containing the > offsets of ``__dict__`` and ``__weakref__``:: > > typedef struct { > Py_ssize_t dict; > Py_ssize_t weaklist; > } PyType_offsets; > > The new slot, ``Py_offsets``, will be used to pass a ``PyType_offsets *`` > structure containing the mentioned data. > > > Helpers > ------- > > Getting to per-module state from a heap type is a very common task. To > make this > easier, a helper will be added:: > > void *PyType_GetModuleState(PyObject *type) > > This function takes a heap type and on success, it returns pointer to > state of the > module that the heap type belongs to. > > On failure, two scenarios may occure. When a type without a module is > passed in, > ``SystemError`` is set and ``NULL`` returned. If the module is found, > pointer > to the state, which may be ``NULL``, is returned without setting any > exception. > > > Modules Converted in the Initial Implementation > ----------------------------------------------- > > To validate the approach, several modules will be modified during > the initial implementation: > > The ``zipimport``, ``_io``, ``_elementtree``, and ``_csv`` modules > will be ported to PEP 489 multiphase initialization. > zipimport currently caches things in C globals. Changing it to use PEP 489 multi-phase initialisation is very likely going to change semantics in subtle ways... Is it really worth the risk? > > > Summary of API Changes and Additions > ==================================== > > New functions: > > * PyType_GetModule > * PyType_DefiningTypeFromSlotFunc > * PyType_GetModuleState > * PyErr_PrepareImmutableException > > New macros: > > * PyCFunction_GET_CLASS > > New types: > > * PyCMethodObject > > New structures: > > * PyType_offsets > > Modified functions: > > * _PyMethodDef_RawFastCallDict now receives ``PyTypeObject *cls``. > * _PyMethodDef_RawFastCallKeywords now receives ``PyTypeObject *cls``. > > Modified structures: > > * _heaptypeobject - added ht_module and ht_moduleptr > > Other changes: > > * METH_METHOD call flag > * defining_class converter in clinic > * Py_TPFLAGS_HEAP_IMMUTABLE flag > * Py_offsets type spec slot > > > Backwards Compatibility > ======================= > > Two new pointers are added to all heap types. > All other changes are adding new functions, structures and a type flag. > > The new ``PyErr_PrepareImmutableException`` function changes encourages > modules to switch from using heap type Exception classes to immutable ones, > and a number of modules will be switched in the initial implementation. > This change will prevent adding class attributes to such types. > For example, the following will raise AttributeError:: > > sqlite.OperationalError.foo = None > > Instances and subclasses of such exceptions will not be affected. > > Implementation > ============== > > An initial implementation is available in a Github repository [#gh-repo]_; > a patchset is at [#gh-patch]_. > > > Possible Future Extensions > ========================== > > Easy creation of types with module references > --------------------------------------------- > > It would be possible to add a PEP 489 execution slot type to make > creating heap types significantly easier than calling > ``PyType_FromModuleAndSpec``. > This is left to a future PEP. > > > Optimization > ------------ > > CPython optimizes calls to methods that have restricted signatures, > such as not allowing keyword arguments. > > As proposed here, methods defined with the ``METH_METHOD`` flag do not > support > these optimizations. > > Optimized calls still have the option of accessing per-module state > the same way slot methods do. > > > References > ========== > > .. [#typeslots-mail] [Import-SIG] On singleton modules, heap types, > and subinterpreters > (https://mail.python.org/pipermail/import-sig/2015-July/001035.html) > > .. [#gh-repo] > https://github.com/Traceur759/cpython/commits/pep-c > > .. [#gh-patch] > https://github.com/Traceur759/cpython/compare/master... > Traceur759:pep-c.patch > > > Copyright > ========= > > This document has been placed in the public domain. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > thomas%40python.org > -- Thomas Wouters Hi! I'm an email virus! Think twice before sending your email to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Thu Apr 26 09:57:58 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 26 Apr 2018 23:57:58 +1000 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: On 25 April 2018 at 18:23, Chris Angelico wrote: >>>> x={print(2): print(1) for _ in [1]} > 1 > 2 > > Hmmmmmmmmm. One of these is not like the others... Huh, it looks like we missed checking dict comprehensions when we fixed dict displays to evaluate keys before the corresponding values. That would qualify as a reasonable request for improvement in Python 3.8 :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Thu Apr 26 10:31:57 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 27 Apr 2018 00:31:57 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <5AE16232.9030807@canterbury.ac.nz> <20180426053417.GE7400@ando.pearwood.info> Message-ID: On 26 April 2018 at 15:38, Chris Angelico wrote: > On Thu, Apr 26, 2018 at 3:34 PM, Steven D'Aprano wrote: >> Can you give an example of a Python expression, involving PEP 572 >> binding-expressions, that is not a tree but a more general DAG or that >> contains cycles? > > A DAG is a directed *acyclic* graph, so it still can't contain cycles. > But I have no idea what kind of expression isn't a tree as a > consequence of having an assignment in it. At a parsing level, the expression remains a tree, it's just that one of the nodes is a name lookup. At a logical level though, binding expressions do indeed mean that expressions involving binding expressions are at least arguably better modelled with a DAG rather than as a tree the way they are now: # The arithmetic expression is a tree including two nodes that look up "c". The DAG is only needed at a statement level. c = expr() a*c + b*c # Whereas this backref to "c" pulls the DAG down to the expression level a*(c := expr()) + b*c Historically, that kind of order-of-evaluation dependence in Python has only been a problem for functions with side effects, so the folks asking that this be seen as a major complexity increase for expression level semantics have an entirely valid point. The PEP aims to address that point by saying "Don't use binding expressions when the order of evaluation would be ambiguous", but that's as a response to a valid concern, not a dismissal of it. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Thu Apr 26 10:37:55 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 27 Apr 2018 00:37:55 +1000 Subject: [Python-Dev] PEP 573 -- Module State Access from C Extension Methods In-Reply-To: <5AE0CCEA.2030806@UGent.be> References: <2854d9cdad73400aa968a084f40908d7@xmail101.UGent.be> <5ADE202B.2060501@UGent.be> <43a667611aa04e60a66372d75c759fa6@xmail101.UGent.be> <5ADEE826.9070806@UGent.be> <9333b45af89e47b4a0b349f284894435@xmail101.UGent.be> <5ADF405C.4060607@UGent.be> <5ADF6582.5080301@UGent.be> <15262b8f09724d1d801cc81289aa4ba2@xmail101.UGent.be> <5AE0CCEA.2030806@UGent.be> Message-ID: On 26 April 2018 at 04:46, Jeroen Demeyer wrote: > On 2018-04-25 20:33, Petr Viktorin wrote: >> >> Perhaps "m_objclass" could point to the module in this case > > > That was exactly my idea also today. Instead of treating m_objclass as the > defining class, we should generalize it to be the "parent" of the function: > either the class or the module. PEP 487 refers to this as the "owner" of a descriptor (See the "__set_name__" example implementation in https://www.python.org/dev/peps/pep-0487/#trait-descriptors ). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From J.Demeyer at UGent.be Thu Apr 26 10:51:52 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Thu, 26 Apr 2018 16:51:52 +0200 Subject: [Python-Dev] PEP 573 -- Module State Access from C Extension Methods In-Reply-To: <5296260b94dc43c79d84459ff2cceaf4@xmail101.UGent.be> References: <2854d9cdad73400aa968a084f40908d7@xmail101.UGent.be> <5ADE202B.2060501@UGent.be> <43a667611aa04e60a66372d75c759fa6@xmail101.UGent.be> <5ADEE826.9070806@UGent.be> <9333b45af89e47b4a0b349f284894435@xmail101.UGent.be> <5ADF405C.4060607@UGent.be> <5ADF6582.5080301@UGent.be> <15262b8f09724d1d801cc81289aa4ba2@xmail101.UGent.be> <5AE0CCEA.2030806@UGent.be> <5296260b94dc43c79d84459ff2cceaf4@xmail101.UGent.be> Message-ID: <5AE1E788.8050103@UGent.be> On 2018-04-26 16:37, Nick Coghlan wrote: > PEP 487 refers to this as the "owner" of a descriptor That's just copied from the Python docs: https://docs.python.org/3.8/reference/datamodel.html#object.__get__ Anyway, I never liked the name "owner" there, "cls" would have been much clearer. For PEP 575, I thought of "owner" too, but I don't like it because it sounds too possessive. It sounds like the "owner" somehow has complete control over the function and I don't want that association. From guido at python.org Thu Apr 26 11:07:50 2018 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Apr 2018 08:07:50 -0700 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: Just as I feared. While currently fixing this is just fixing a bug (nobody's code is going to break except perhaps some tests), with PEP 572 fixing this would be mandatory. On Thu, Apr 26, 2018 at 6:57 AM, Nick Coghlan wrote: > On 25 April 2018 at 18:23, Chris Angelico wrote: > >>>> x={print(2): print(1) for _ in [1]} > > 1 > > 2 > > > > Hmmmmmmmmm. One of these is not like the others... > > Huh, it looks like we missed checking dict comprehensions when we > fixed dict displays to evaluate keys before the corresponding values. > That would qualify as a reasonable request for improvement in Python > 3.8 :) > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Thu Apr 26 11:18:08 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 27 Apr 2018 01:18:08 +1000 Subject: [Python-Dev] PEP 572: Why not := as standard assignment operator? In-Reply-To: References: Message-ID: On Thu, Apr 26, 2018 at 11:13 PM, Martin Teichmann wrote: > Hi list, > > when reading PEP 572 I actually liked it a lot - I think it's actually > a cool idea. I think it's actually that cool an idea that it should be > made the default way of doing an assignment, over time phasing out the > good ole =. > > This would have several benefits: > > - people wouldn't have to worry about two different options > - different things would have a different look: assignment is :=, > keyword args is =, while comparison is ==. Especially beginners would > benefit from this clarity. > > in this case, for sure, we should make it possible to chain :=s, for > example by making it bind right-to-left, so that a := b := 3 would be > a := (b := 3) > > I'm sorry if somebody brought that up already, but the discussion has > grown so huge that I couldn't read through it entirely. It has indeed grown huge. And in the interests of not growing it even huger, I'm not going to rehash the arguments against making := into the one and only operator, save to say one thing: there's no way that "x = 1" can be removed from the language any time soon, and by "soon" I mean even by the Yes Prime Minister definition, where "any day now", in strategic terms, meant "within the next half century". For further details, search the archives. Please don't reply to debate this point. :) ChrisA From rosuav at gmail.com Thu Apr 26 11:21:13 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 27 Apr 2018 01:21:13 +1000 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: On Fri, Apr 27, 2018 at 1:07 AM, Guido van Rossum wrote: > Just as I feared. While currently fixing this is just fixing a bug (nobody's > code is going to break except perhaps some tests), with PEP 572 fixing this > would be mandatory. > > On Thu, Apr 26, 2018 at 6:57 AM, Nick Coghlan wrote: >> >> On 25 April 2018 at 18:23, Chris Angelico wrote: >> >>>> x={print(2): print(1) for _ in [1]} >> > 1 >> > 2 >> > >> > Hmmmmmmmmm. One of these is not like the others... >> >> Huh, it looks like we missed checking dict comprehensions when we >> fixed dict displays to evaluate keys before the corresponding values. >> That would qualify as a reasonable request for improvement in Python >> 3.8 :) It's deliberate code in compile.c: case COMP_DICTCOMP: /* With 'd[k] = v', v is evaluated before k, so we do the same. */ VISIT(c, expr, val); VISIT(c, expr, elt); ADDOP_I(c, MAP_ADD, gen_index + 1); break; So if it's a bug, it's still a deliberate bug. :) ChrisA From gjcarneiro at gmail.com Thu Apr 26 11:38:33 2018 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Thu, 26 Apr 2018 16:38:33 +0100 Subject: [Python-Dev] PEP 572: Why not := as standard assignment operator? In-Reply-To: References: Message-ID: On 26 April 2018 at 16:18, Chris Angelico wrote: > On Thu, Apr 26, 2018 at 11:13 PM, Martin Teichmann > wrote: > > Hi list, > > > > when reading PEP 572 I actually liked it a lot - I think it's actually > > a cool idea. I think it's actually that cool an idea that it should be > > made the default way of doing an assignment, over time phasing out the > > good ole =. > > > > This would have several benefits: > > > > - people wouldn't have to worry about two different options > > - different things would have a different look: assignment is :=, > > keyword args is =, while comparison is ==. Especially beginners would > > benefit from this clarity. > > > > in this case, for sure, we should make it possible to chain :=s, for > > example by making it bind right-to-left, so that a := b := 3 would be > > a := (b := 3) > > > > I'm sorry if somebody brought that up already, but the discussion has > > grown so huge that I couldn't read through it entirely. > > It has indeed grown huge. And in the interests of not growing it even > huger, I'm not going to rehash the arguments against making := into > the one and only operator, save to say one thing: there's no way that > "x = 1" can be removed from the language any time soon, and by "soon" > I mean even by the Yes Prime Minister definition, where "any day now", > in strategic terms, meant "within the next half century". > In the interest of that, do you think := can be made illegal, by the grammar, if used outside an expression? a = 1 # legal a := 1 # Syntax error if a := 1: # legal Thanks in advance. -- Gustavo J. A. M. Carneiro Gambit Research "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Thu Apr 26 11:39:54 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 27 Apr 2018 01:39:54 +1000 Subject: [Python-Dev] PEP 572: Why not := as standard assignment operator? In-Reply-To: References: Message-ID: On Fri, Apr 27, 2018 at 1:38 AM, Gustavo Carneiro wrote: > > > On 26 April 2018 at 16:18, Chris Angelico wrote: >> >> On Thu, Apr 26, 2018 at 11:13 PM, Martin Teichmann >> wrote: >> > Hi list, >> > >> > when reading PEP 572 I actually liked it a lot - I think it's actually >> > a cool idea. I think it's actually that cool an idea that it should be >> > made the default way of doing an assignment, over time phasing out the >> > good ole =. >> > >> > This would have several benefits: >> > >> > - people wouldn't have to worry about two different options >> > - different things would have a different look: assignment is :=, >> > keyword args is =, while comparison is ==. Especially beginners would >> > benefit from this clarity. >> > >> > in this case, for sure, we should make it possible to chain :=s, for >> > example by making it bind right-to-left, so that a := b := 3 would be >> > a := (b := 3) >> > >> > I'm sorry if somebody brought that up already, but the discussion has >> > grown so huge that I couldn't read through it entirely. >> >> It has indeed grown huge. And in the interests of not growing it even >> huger, I'm not going to rehash the arguments against making := into >> the one and only operator, save to say one thing: there's no way that >> "x = 1" can be removed from the language any time soon, and by "soon" >> I mean even by the Yes Prime Minister definition, where "any day now", >> in strategic terms, meant "within the next half century". > > > In the interest of that, do you think := can be made illegal, by the > grammar, if used outside an expression? > > a = 1 # legal > a := 1 # Syntax error > if a := 1: # legal > No. Any expression may be used as a statement, so this isn't "outside an expression". ChrisA From lukasz at langa.pl Thu Apr 26 11:48:12 2018 From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=) Date: Thu, 26 Apr 2018 08:48:12 -0700 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <20180426061000.GF7400@ando.pearwood.info> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <20180426061000.GF7400@ando.pearwood.info> Message-ID: <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> > On Apr 25, 2018, at 11:10 PM, Steven D'Aprano wrote: > Criticising binding- > expressions for that reason, especially implying that we must always use > parens, is simply FUD. The PEP has more examples with parentheses than without. >> As soon as we have to wrap a part of an expression in parentheses, >> parsing the entire thing becomes more complex. > > Unless it becomes less complex to read and understand. You're ignoring the context of the discussion. The new parentheses are there because there's a new assignment there. That's more complex. >> Often enough it will >> cause the expression to exceed whatever line length limit the codebase >> pledged not to exceed, causing one line to become three. > > Just how often are your lines within two characters of the maximum > column so that adding a pair of brackets () will "often enough" put it > over the limit? Seems pretty unlikely to me. Again, you're ignoring the context of the discussion. Assuming := will have spaces around it in PEP 8, this is adding *at least* 7 characters if the name of your choice is a single character. > This sounds really like > picking at straws. The entire point of the PEP is to make things "nicer". It doesn't fundamentally enable programmers to do anything they couldn't do before. If you think demonstrating cases where the end result won't be an improvement is picking at straws, then maybe the improvement of PEP 572 is as well. -- ? From gvanrossum at gmail.com Thu Apr 26 12:02:58 2018 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu, 26 Apr 2018 16:02:58 +0000 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: Maybe the order for d[k] = v should also be reconsidered? On Thu, Apr 26, 2018, 08:23 Chris Angelico wrote: > On Fri, Apr 27, 2018 at 1:07 AM, Guido van Rossum > wrote: > > Just as I feared. While currently fixing this is just fixing a bug > (nobody's > > code is going to break except perhaps some tests), with PEP 572 fixing > this > > would be mandatory. > > > > On Thu, Apr 26, 2018 at 6:57 AM, Nick Coghlan > wrote: > >> > >> On 25 April 2018 at 18:23, Chris Angelico wrote: > >> >>>> x={print(2): print(1) for _ in [1]} > >> > 1 > >> > 2 > >> > > >> > Hmmmmmmmmm. One of these is not like the others... > >> > >> Huh, it looks like we missed checking dict comprehensions when we > >> fixed dict displays to evaluate keys before the corresponding values. > >> That would qualify as a reasonable request for improvement in Python > >> 3.8 :) > > It's deliberate code in compile.c: > > case COMP_DICTCOMP: > /* With 'd[k] = v', v is evaluated before k, so we do > the same. */ > VISIT(c, expr, val); > VISIT(c, expr, elt); > ADDOP_I(c, MAP_ADD, gen_index + 1); > break; > > So if it's a bug, it's still a deliberate bug. :) > > ChrisA > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Thu Apr 26 12:06:33 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 27 Apr 2018 02:06:33 +1000 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: On Fri, Apr 27, 2018 at 2:02 AM, Guido van Rossum wrote: > Maybe the order for d[k] = v should also be reconsidered? If so, it would be something like: 1) Evaluate d 2) Evaluate k 3) Evaluate v 4) Call d.__setitem__(k, v), via slots etc In a vacuum, I don't have a problem with that. But I suspect that it'd break more code than the comprehensions changes do. ChrisA From d.f.hilliard at gmail.com Thu Apr 26 12:19:43 2018 From: d.f.hilliard at gmail.com (Jim F.Hilliard) Date: Thu, 26 Apr 2018 19:19:43 +0300 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: FWIW There's an existing issue (bpo 29652 ) for the order of evaluation in dict-comps. Best Regards, Jim Fasarakis Hilliard On Thu, Apr 26, 2018 at 7:06 PM, Chris Angelico wrote: > On Fri, Apr 27, 2018 at 2:02 AM, Guido van Rossum > wrote: > > Maybe the order for d[k] = v should also be reconsidered? > > If so, it would be something like: > > 1) Evaluate d > 2) Evaluate k > 3) Evaluate v > 4) Call d.__setitem__(k, v), via slots etc > > In a vacuum, I don't have a problem with that. But I suspect that it'd > break more code than the comprehensions changes do. > > ChrisA > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/d. > f.hilliard%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ericsnowcurrently at gmail.com Thu Apr 26 12:25:57 2018 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Thu, 26 Apr 2018 10:25:57 -0600 Subject: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 Message-ID: In pondering our approach to future Python major releases, I found myself considering the experience we've had with Python 3. The whole Py3k effort predates my involvement in the community so I missed a bunch of context about the motivations, decisions, and challenges. While I've pieced some of that together over the years now since I've been around, I've certainly seen much of the aftermath. For me, at least, it would be helpful to have a bit more insight into the history. :) With that in mind, it would be worth having an informational PEP with an authoritative retrospective on the lessons learned from the Python 3 effort (and transition). Consider it a sort of autobiography, "memoirs on the python-dev change to Python 3". :) At this point the transition has settled in enough that we should be able to present a relatively objective (and consistent) view, while we're not so far removed that we've forgotten anything important. :) If such a document already exists then I'd love a pointer to it. The document would benefit (among others): * python-dev (by giving us a clear viewpoint to inform decisions about future releases) * new-comers to Python that want more insight into the language * folks transitioning from 2 to 3 * communities that have (or think they have) problems similar to those we faced in Python 2 The PEP doesn't even have to be done all at once, nor by one person. In fact, there are many viewpoints that would add value to the document. Hence it would probably make sense to encourage broad participation and then have a single editor to effect a single voice in the document. The contents of the retrospective document should probably cover a broad range of topics, since there's so much to learn from the move to Python 3. To give an indication of what I mean, I've included a rough outline at the bottom of this message. So...I typically strongly avoid making proposals that I'm not willing to execute. However, in this case I simply do not have enough experience in the history to feel comfortable doing a good job of it in a reasonable amount of time (which matters due to the tendency of valuable info to fade away). :/ I have no expectation that someone will pick this up, though I do hope since the benefit would be significant. My apologies in advance if this wasted anyone's time. -eric ++++++++++++++++++++++++++++++++ I'd hope to see something along the lines of (at least) the following, in rough order: * a concise summary of the document at the top (very meta, I know :) ) + what were we solving? + what was the solution? + why do it that way? + what went right? + what went wrong? + impact on the community + impact on core dev contribution * timeline * key players (and level of involvement) + old guard core devs + new guard + folks brought on for Py3k (e.g. IIRC a swarm of Googlers dove in) + non-core-devs * motivations * expectations (e.g. time frames, community reaction) * corresponding results * a summary of what we did * alternative approaches * what went right (and was it on purpose :) ) * what went wrong (e.g. io) and why * how the Py3k project differed from normal python-dev workflow (e.g. pace, decision-making, communications) * lasting impact of python-dev * key things that would have been better if done differently * key decisions/planning (mostly a priori to the release work) + scope of backward compatibility + process (using PEPs with PEPs 30xx guiding) + schedule + specific changes (i.e. PEPs 31xx) + what was left out (and why) + plans to help library and app authors transition (e.g. 2to3) + feature/schedule overlap with Python 2 (i.e. 2.6 and 2.7) + the language moratorium * things that got missed and why + unicode/bytes in some stdlib modules (and builtins?) * things that were overdone (and how that got missed) + unicode/bytes in some stdlib modules (and builtins?) * (last but not least) challenges faced by folks working to transition their exiting code to Python 3 From ericsnowcurrently at gmail.com Thu Apr 26 12:28:10 2018 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Thu, 26 Apr 2018 10:28:10 -0600 Subject: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 In-Reply-To: References: Message-ID: On Thu, Apr 26, 2018 at 10:25 AM, Eric Snow wrote: > In pondering our approach to future Python major releases, I found > myself considering the experience we've had with Python 3. The whole > Py3k effort predates my involvement in the community so I missed a > bunch of context about the motivations, decisions, and challenges. > While I've pieced some of that together over the years now since I've > been around, I've certainly seen much of the aftermath. For me, at > least, it would be helpful to have a bit more insight into the > history. :) As to motivation, I suppose (since I wasn't involved yet) that it was roughly: let's bite the bullet and fix unicode with a backward-incompatible release which led to: well, we're already going to break backward compatibility so we might as well get any breakage we're planning over with now so folks only have to fix their code for this release and then to: oh, and while we're at it let's clean up a bunch of cruft that's built up over the years At least that's my impression. :) -eric From python-dev at mgmiller.net Thu Apr 26 12:36:48 2018 From: python-dev at mgmiller.net (Mike Miller) Date: Thu, 26 Apr 2018 09:36:48 -0700 Subject: [Python-Dev] Is PEP 572 really the most effective way to solve the problems it's targeting? In-Reply-To: <162ffcf2f98.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> References: <162ffcf2f98.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> Message-ID: <39fe9f53-843b-3394-b7fc-c51a400a1409@mgmiller.net> On 2018-04-25 19:36, Ryan Gonzalez wrote: By now we've searched over the current proposal in excruciating detail. However, there were two good questions in this message which I haven't seen addressed yet: - How are other modern languages solving this issue? - How does this new construct intersect with typing functionality? -Mike From tim.peters at gmail.com Thu Apr 26 12:39:47 2018 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 26 Apr 2018 11:39:47 -0500 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: [Guido] > Maybe the order for d[k] = v should also be reconsidered? There's certainly code in the wild relying on the order "v, then d, then k", because that's just how assignment statements have always worked (dicts or not). I'd rather change the dict comprehension code, because I think the analogy to what `d[k] = v` does is weak. I expect: {f(): g() for _ in [1]| to do the same as {f(): g()} The former currently evaluates g() first; the latter f(). Since I also expect both of those to do the same as dict((f(), g()) for _ in [1]) f() has to win ;-) From ericsnowcurrently at gmail.com Thu Apr 26 12:42:52 2018 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Thu, 26 Apr 2018 10:42:52 -0600 Subject: [Python-Dev] Every Release Can Be a Mini "Python 4000", Within Reason (was (name := expression) doesn't fit the narrative of PEP 20) Message-ID: On Thu, Apr 26, 2018 at 12:52 AM, Greg Ewing wrote: > [snip] > here we would be *creating* one (two different assignment > operators with overlapping use cases) that we won't be > able to get rid of without a Python 4000 (that Guido has > promised won't happen). The edict about "Python 4000" is more about not repeating what happened with Python 3 than strictly prohibiting breaking backward compatibility. [1] The way I understand it, the problem with Py3k was that so many things changed in a backward-incompatible way. Folks could have coped with the unicode change as the big one for Python 2.7 (or 2.8 or 3.0 or whatever it ended up being). However, so many other things changed all at once that the burden to move to Python 3 became daunting. This included a number of back-compat-breaking syntax changes. Unless I've missed something, there's no prohibition against deprecating things (and then later removing them) or other breaks in backward compatibility. We certainly avoid it, with good reason. However, when we do break compatibility, the thing we want to avoid is introducing too many such changes all at once (and to keep in mind that such changes can add up to the same detriment when viewed in aggregate across multiple releases). That said, syntax is definitely a trickier target when it comes to breaking backward compatibility. So we have to be especially careful about adding it in the first place. I suppose that's a big part of the reason for the strong reaction to the "binding expressions" proposal. -eric [1] I'm hopeful we can consolidate a retrospective on Python 3 in a PEP: https://mail.python.org/pipermail/python-dev/2018-April/153131.html From kirillbalunov at gmail.com Thu Apr 26 09:27:18 2018 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Thu, 26 Apr 2018 16:27:18 +0300 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: 2018-04-24 18:31 GMT+03:00 Chris Angelico : > > Recommended use-cases > ===================== > > [...] > > # Capturing regular expression match objects > # See, for instance, Lib/pydoc.py, which uses a multiline spelling > # of this effect > if match := re.search(pat, text): > print("Found:", match.group(0)) > > Not sure, but if additional motivating examples are required, there is a common pattern for dynamic attribute lookup (snippet from `copy.py`): reductor = dispatch_table.get(cls) if reductor: rv = reductor(x) else: reductor = getattr(x, "__reduce_ex__", None) if reductor: rv = reductor(4) else: reductor = getattr(x, "__reduce__", None) if reductor: rv = reductor() else: raise Error("un(shallow)copyable object of type %s" % cls) which can with the current `binding expression` syntax simplified to: if reductor := dispatch_table.get(cls): rv = reductor(x) elif reductor := getattr(x, "__reduce_ex__", None): rv = reductor(4) elif reductor := getattr(x, "__reduce__", None): rv = reductor() else: raise Error("un(shallow)copyable object of type %s" % cls) which becomes much clearer, at least in my opinion. With kind regards, -gdg -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.peters at gmail.com Thu Apr 26 12:58:32 2018 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 26 Apr 2018 11:58:32 -0500 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: [Kirill Balunov ] > Not sure, but if additional motivating examples are required, there is a > common pattern for dynamic attribute lookup (snippet from `copy.py`): > > reductor = dispatch_table.get(cls) > if reductor: > rv = reductor(x) > else: > reductor = getattr(x, "__reduce_ex__", None) > if reductor: > rv = reductor(4) > else: > reductor = getattr(x, "__reduce__", None) > if reductor: > rv = reductor() > else: > raise Error("un(shallow)copyable object of type %s" % cls) > > which can with the current `binding expression` syntax simplified to: > > if reductor := dispatch_table.get(cls): > rv = reductor(x) > elif reductor := getattr(x, "__reduce_ex__", None): > rv = reductor(4) > elif reductor := getattr(x, "__reduce__", None): > rv = reductor() > else: > raise Error("un(shallow)copyable object of type %s" % cls) > > which becomes much clearer, at least in my opinion. > > With kind regards, > -gdg Thanks for sharing that! While nobody else seems to, I absolutely love real code ;-) This is effectively an instance of Guido's "if/elif/elif/elif/..." example template, where binding expressions shine. But actual real code can make a point viscerally that "consider stuff akin to the following semi-abstract pattern" can't. From barry at python.org Thu Apr 26 13:18:20 2018 From: barry at python.org (Barry Warsaw) Date: Thu, 26 Apr 2018 10:18:20 -0700 Subject: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 In-Reply-To: References: Message-ID: On Apr 26, 2018, at 09:28, Eric Snow wrote: > > On Thu, Apr 26, 2018 at 10:25 AM, Eric Snow wrote: >> In pondering our approach to future Python major releases, I found >> myself considering the experience we've had with Python 3. The whole >> Py3k effort predates my involvement in the community so I missed a >> bunch of context about the motivations, decisions, and challenges. >> While I've pieced some of that together over the years now since I've >> been around, I've certainly seen much of the aftermath. For me, at >> least, it would be helpful to have a bit more insight into the >> history. :) It would certainly be an interesting document, but I suspect you?ll get a bit of the old ?ask 3 lawyers and get 5 opinions? kind of response. ;) As I remember it, there was definitely a feeling like, this would be our only chance to clean up some annoying cruft, and rectify some (in hindsight) incorrect design decisions made over the years, couple with a healthy dose of ?we have no idea how to do the bytes/str split in a backward compatible way". There was probably a sense that the Python community was just small enough to be able to handle such a disruptive change, but wouldn?t ever be so again. The latter is definitely true today, even if the former was overly optimistic. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From srkunze at mail.de Thu Apr 26 13:33:48 2018 From: srkunze at mail.de (Sven R. Kunze) Date: Thu, 26 Apr 2018 19:33:48 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <20180424231911.GN11616@ando.pearwood.info> References: <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <83e528a7-9758-fcbf-eb91-b6f1763b5e2e@mail.de> <20180424231911.GN11616@ando.pearwood.info> Message-ID: On 25.04.2018 01:19, Steven D'Aprano wrote: > Sorry, gcd(diff, n) is not the "perfect name", and I will tell you that > sometimes g is better. [...] We were talking about the real-world code snippet of Tim (as a justification of := ) and alternative rewritings of it without resorting to new syntax. Not about "sometimes", or in some toy examples, or code without unknown life-time and intention (interactive, library, etc). > You might know that, but how does somebody reading the code know > which functions are pure and which are not? How does the compiler know? There are invisible optimizations in CPython already. And for the most obvious examples, humans don't need to know. It's like wondering if constant folding really works. It's the same with all optimizations: if it works, it's fine. If it cannot be made working, the original code still works and you can optimize by hand anyway. The two obvious ways: the compiler might have a predefined list of pure functions OR the code tells him with an annotation of any sort. Both ways work, the latter somehow feels cleaner. > It's easy to say that you recognise gcd as a pure function. How about > len(document.chapter[this_chapter].pages()), is that a pure function? What exactly do you want me to answer? An algorithm? A trained neural network? Usually, one starts with the low-hanging fruits and extends if possible and needed. So that would be way out of scope. From srkunze at mail.de Thu Apr 26 13:57:39 2018 From: srkunze at mail.de (Sven R. Kunze) Date: Thu, 26 Apr 2018 19:57:39 +0200 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: References: Message-ID: On 24.04.2018 11:21, Victor Stinner wrote: > I have been asked to express myself on the PEP 572. +1 I knew about the relationship between read and write. But your stance on debugging just makes it a thing. Thanks a lot! From rosuav at gmail.com Thu Apr 26 14:00:35 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 27 Apr 2018 04:00:35 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <83e528a7-9758-fcbf-eb91-b6f1763b5e2e@mail.de> <20180424231911.GN11616@ando.pearwood.info> Message-ID: On Fri, Apr 27, 2018 at 3:33 AM, Sven R. Kunze wrote: > On 25.04.2018 01:19, Steven D'Aprano wrote: >> >> Sorry, gcd(diff, n) is not the "perfect name", and I will tell you that >> sometimes g is better. [...] > > > We were talking about the real-world code snippet of Tim (as a justification > of := ) and alternative rewritings of it without resorting to new syntax. > Not about "sometimes", or in some toy examples, or code without unknown > life-time and intention (interactive, library, etc). You're asking for some massive changes in semantics, though. If you want to discuss that, it's nothing at all to do with PEP 572, and is a completely new proposal. >> You might know that, but how does somebody reading the code know >> which functions are pure and which are not? How does the compiler know? > > > There are invisible optimizations in CPython already. > And for the most obvious examples, humans don't need to know. It's like > wondering if constant folding really works. > It's the same with all optimizations: if it works, it's fine. If it cannot > be made working, the original code still works and you can optimize by hand > anyway. > The two obvious ways: the compiler might have a predefined list of pure > functions OR the code tells him with an annotation of any sort. > Both ways work, the latter somehow feels cleaner. An annotation? You mean like functools.lru_cache? That's technically nothing to do with pure functions, but it's about as much as you're likely to get. The function still has to be called, and the result has to be cached somewhere. Ultimately, "pure function optimization" is just another form of cache. Please, take this to python-ideas, or even better, to python-list. It's not a counter-argument to PEP 572. ChrisA From larry at hastings.org Thu Apr 26 14:07:05 2018 From: larry at hastings.org (Larry Hastings) Date: Thu, 26 Apr 2018 11:07:05 -0700 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: On 04/26/2018 06:27 AM, Kirill Balunov wrote: > Not sure, but if additional motivating examples are required, there is > a common pattern for dynamic attribute lookup (snippet from `copy.py`): > > ??? reductor = dispatch_table.get(cls) > ??? if reductor: > ??????? rv = reductor(x) > ??? else: > ??????? reductor = getattr(x, "__reduce_ex__", None) > ??????? if reductor: > ??????????? rv = reductor(4) > ??????? else: > ??????????? reductor = getattr(x, "__reduce__", None) > ??????????? if reductor: > ??????????????? rv = reductor() > ??????????? else: > ??????????????? raise Error("un(shallow)copyable object of type %s" % cls) > > which can with the current `binding expression` syntax simplified to: > > ??? if reductor := dispatch_table.get(cls): > ??????? rv = reductor(x) > ??? elif reductor := getattr(x, "__reduce_ex__", None): > ??????? rv = reductor(4) > ??? elif reductor := getattr(x, "__reduce__", None): > ??????? rv = reductor() > ??? else: > ??????? raise Error("un(shallow)copyable object of type %s" % cls) > > which becomes much clearer, at least in my opinion. I hate to be pedantic--there's enough of that going on in this thread--but I can't agree with the word "simplifed" above.? I agree that the code using binding expressions is /shorter./? But considering that emit the two code examples implement the exact same algorithm, to the point where their bytecode would look nearly* identical, ISTM that the two code examples are of identical /complexity./ Comparing the two, the code using the binding expressions obviates four newlines, three uses of the identifier "reductor", and allows folding two "else / if"s into "elif"s.? In exchange, it adds three extra colons, and the density of complexity per line has shot up. Whether or not that's an improvement is in the eye of the beholder, and I gather the battle lines around this PEP have already been clearly drawn.? But it cannot be said that this transformation has made the code simpler. Personally, I've already voted -1; I don't see the version using the binding expressions as an improvement.? Expressing the same code using slightly fewer characters doesn't justify adding syntax to the language in my opinion. //arry/ * I assume that--if we don't already have one--we'll add a new DUP_AND_STORE_FAST opcode for binding expressions.? On the other hand, "STORE_FAST n" followed by "LOAD_FAST n" is a common enough bytecode idiom that the peephole optimizer could notice it and pretty easily swap it for DUP_AND_STORE_FAST. -------------- next part -------------- An HTML attachment was scrubbed... URL: From famopy at gmail.com Thu Apr 26 14:00:55 2018 From: famopy at gmail.com (Fatty Morgan) Date: Thu, 26 Apr 2018 20:00:55 +0200 Subject: [Python-Dev] Reserve ':=' for type-inferred variable initialization (was PEP 572) Message-ID: I would like to urge you to reconsider the use of the token ':=' for assignment expressions. The natural interpretation of 'name := expr' is a PEP 526 type-annotated variable initialization 'name : T = expr' with the type annotation T omitted, the tokens ':' and '=' coalesced, and the implied type T inferred as 'type(expr)'. There is an ongoing tendency to introduce possibilities for static analysis into python, perhaps eventually resulting in a statically typed variant of python were (optional) type annotations are enforced (and used for optimizations), and it would be a pity if this specific piece of obvious syntax had been wasted on a comparatively unimportant feature. The following should probably be discussed in python-ideas and would only be relevant for a more distant future, but while I have your attention, here are a couple of thoughts: Distinguishing in a rather unobtrusive way (no 'var' or 'let') the initialization of a previously unbound variable 'name := expr' and the re-assignment of a previously bound variable 'name = expr' would be beneficial in itself (this has been discussed before but AFAIK not with this syntax). For example, 'global' and 'nonlocal' would be redundant and could be deprecated. The use of variable initializations with explicit type annotations as in PEP 526 would only be required in situations where the declared type is deliberately different from the inferred type, e.g. 'any_var: object = 42; any_var = "foo"; #OK, no TypeError'. In the majority of cases a simple 'name := expr' would be sufficient and type-safe. Destructuring initialization, e.g., 'a,b,c: int = range(3)' or 'a,b,c := range(3)', is possible (the annotated or inferred type refers not to the type of the iterable but to the generated type). If it were forbidden (i.e., only a single name-variable allowed) PEP 572 with ':=' could be simply implemented by parsing 'name := expr' as an expression rather than a statement (assuming it is OK to introduce 'name' into the current scope) but destructuring initialization appears to be the more important feature and would make assignment expressions with ':=' ambiguous. Implementation would be straightforward if 'name := expr' were lowered to 'name: auto = expr' where the newly introduced abstract type 'auto' simply acts as a token informing compile-time and/or run-time what to do. An explicit type annotation with type 'auto' could be allowed and would help to teach the feature. From lukasz at langa.pl Thu Apr 26 14:16:12 2018 From: lukasz at langa.pl (Lukasz Langa) Date: Thu, 26 Apr 2018 11:16:12 -0700 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> Message-ID: [Uncle T] > So, to match your sarcasm, here's mine: try using a feature for what > it's good at instead of for what it's bad at ;-) Yes, this is the fundamental wisdom. Judging which is which is left as an exercise to the programmer. With this, I'm leaving the discussion. With Guido and you on board for PEP 572, I feel that Chris' streak is indeed about to break. Some parting hair-splitting follows. > [Uncle T] >>> One language feature conspicuous by absence in newbie >>> confusions was, consistently, assignment expressions. Read any book >>> or tutorial for such a language, and you'll find very little space >>> devoted to them too. > > [?ukasz Langa ] >> Well, you have an entire code style built around this feature called Yoda >> conditions. You teach people on Day 1 to never ever confuse == with =. Some >> compilers even warn about this because so many people did it wrong. > [Uncle T] > Sorry, I couldn't follow that. You implied that newbies don't have to even know about assignments in expressions. I wanted to demonstrate that this isn't really the case because mistaking `=` for `==` is a relatively common occurence for newbies. If you want to argue that it isn't, I'd like to point out that the WordPress code style *requires* Yoda conditions because it was enough of a hindrance. ESLint (a JavaScript linter) also has a warning about assignment in a conditional. [Uncle T]] > In languages like C that use easily > confused operator symbols, sure, people are forever typing "=" when > they mean "==". That's nothing to do with whether they _understand_ > what the different operators do, though. What you're saying is true. But for it to be true, newbies *have to* learn the distinction, and the fact that yes, sometimes the programmer indeed meant to put a single `=` sign in the conditional. That's why we'll end up with the Pascal assignment operator. And that *is* a thing that you will have to explain to newbies when they encounter it for the first time. Sadly, googling for a colon followed by an equal sign isn't trivial if you don't know what you're looking for. [?ukasz] >> Well, you can also use it as a statement. But don't! > [Uncle T]] > Why not? _Every_ expression in Python can be used as a statement. > Nothing forbids it, and that's even (very!) useful at an interactive > prompt. Because it suggests different intent, because it's limited, because it's slower at runtime, and because PEP 572 says so itself. > At this point I think you must have a lower opinion of Python > programmers than I have ;-) If adding even a dozen characters to a > line makes it exceed a reasonable line-length guide, the code was > almost certainly too confusingly dense to begin with. Around 5% of if and elif statements in the standard library don't fit a single line *as is*. Sure, that's a low percentage but that's over 1,000 statements. If you're putting an `if` statement in a method, you are already starting out with 71 characters left on the line. Four of those are going to be taken by "if", a space, and the colon. Adding a parenthesized assignment expression takes at least 10% of that available space. The silver lining for me is that this makes the environment riper for auto-formatting. -- ? From rosuav at gmail.com Thu Apr 26 14:18:43 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 27 Apr 2018 04:18:43 +1000 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: On Fri, Apr 27, 2018 at 4:07 AM, Larry Hastings wrote: > * I assume that--if we don't already have one--we'll add a new > DUP_AND_STORE_FAST opcode for binding expressions. On the other hand, > "STORE_FAST n" followed by "LOAD_FAST n" is a common enough bytecode idiom > that the peephole optimizer could notice it and pretty easily swap it for > DUP_AND_STORE_FAST. In the reference implementation, it's just DUP_TOP followed by STORE_FAST (well, technically by "whatever the assignment compiles to", as it could be affected by global/nonlocal, closures, etc). Is there much advantage to creating a new opcode? OTOH, I noticed a lot of STORE_FAST followed by LOAD_FAST in list comprehensions. For instance, [x+1 for x in items] translates to: 1 0 BUILD_LIST 0 2 LOAD_FAST 0 (.0) >> 4 FOR_ITER 12 (to 18) 6 STORE_FAST 1 (x) 8 LOAD_FAST 1 (x) 10 LOAD_CONST 0 (1) 12 BINARY_ADD 14 LIST_APPEND 2 16 JUMP_ABSOLUTE 4 >> 18 RETURN_VALUE In theory, STORE_FAST followed by LOAD_FAST could be transformed into DUP_TOP followed by STORE_FAST, and then if there is any variable that is stored fast and never loaded, it could be optimized out. That'd leave this with just a DUP_TOP and no variable named 'x'. No idea whether it'd be worth the effort though. ChrisA From brett at python.org Thu Apr 26 14:20:58 2018 From: brett at python.org (Brett Cannon) Date: Thu, 26 Apr 2018 18:20:58 +0000 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: With the new name restriction on the LHS, I'm now -0 on this. While I don't think the benefits outweigh the overhead cost of pushing Python closer to not fitting in my brain, I would admittedly use this if provided to me. (I also put this in the bucket of consenting adult features; ripe for abuse, but common sense should prevail much like with every other feature we have added that could get over-used, e.g. decorators.) On Tue, 24 Apr 2018 at 08:36 Chris Angelico wrote: > The most notable change since last posting is that the assignment > target is no longer as flexible as with the statement form of > assignment, but is restricted to a simple name. > > Note that the reference implementation has not been updated. > > ChrisA > > > PEP: 572 > Title: Assignment Expressions > Author: Chris Angelico > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 28-Feb-2018 > Python-Version: 3.8 > Post-History: 28-Feb-2018, 02-Mar-2018, 23-Mar-2018, 04-Apr-2018, > 17-Apr-2018, > 25-Apr-2018 > > > Abstract > ======== > > This is a proposal for creating a way to assign to variables within an > expression. Additionally, the precise scope of comprehensions is adjusted, > to > maintain consistency and follow expectations. > > > Rationale > ========= > > Naming the result of an expression is an important part of programming, > allowing a descriptive name to be used in place of a longer expression, > and permitting reuse. Currently, this feature is available only in > statement form, making it unavailable in list comprehensions and other > expression contexts. Merely introducing a way to assign as an expression > would create bizarre edge cases around comprehensions, though, and to avoid > the worst of the confusions, we change the definition of comprehensions, > causing some edge cases to be interpreted differently, but maintaining the > existing behaviour in the majority of situations. > > > Syntax and semantics > ==================== > > In any context where arbitrary Python expressions can be used, a **named > expression** can appear. This is of the form ``name := expr`` where > ``expr`` is any valid Python expression, and ``name`` is an identifier. > > The value of such a named expression is the same as the incorporated > expression, with the additional side-effect that the target is assigned > that value:: > > # Handle a matched regex > if (match := pattern.search(data)) is not None: > ... > > # A more explicit alternative to the 2-arg form of iter() invocation > while (value := read_next_item()) is not None: > ... > > # Share a subexpression between a comprehension filter clause and its > output > filtered_data = [y for x in data if (y := f(x)) is not None] > > > Differences from regular assignment statements > ---------------------------------------------- > > Most importantly, since ``:=`` is an expression, it can be used in contexts > where statements are illegal, including lambda functions and > comprehensions. > > An assignment statement can assign to multiple targets, left-to-right:: > > x = y = z = 0 > > The equivalent assignment expression is parsed as separate binary > operators, > and is therefore processed right-to-left, as if it were spelled thus:: > > assert 0 == (x := (y := (z := 0))) > > Statement assignment can include annotations. This would be syntactically > noisy in expressions, and is of minor importance. An annotation can be > given separately from the assignment if needed:: > > x:str = "" # works > (x:str := "") # SyntaxError > x:str # possibly before a loop > (x := "") # fine > > Augmented assignment is not supported in expression form:: > > >>> x +:= 1 > File "", line 1 > x +:= 1 > ^ > SyntaxError: invalid syntax > > Statement assignment is able to set attributes and subscripts, but > expression assignment is restricted to names. (This restriction may be > relaxed in a future version of Python.) > > Otherwise, the semantics of assignment are identical in statement and > expression forms. > > > Alterations to comprehensions > ----------------------------- > > The current behaviour of list/set/dict comprehensions and generator > expressions has some edge cases that would behave strangely if an > assignment > expression were to be used. Therefore the proposed semantics are changed, > removing the current edge cases, and instead altering their behaviour > *only* > in a class scope. > > As of Python 3.7, the outermost iterable of any comprehension is evaluated > in the surrounding context, and then passed as an argument to the implicit > function that evaluates the comprehension. > > Under this proposal, the entire body of the comprehension is evaluated in > its implicit function. Names not assigned to within the comprehension are > located in the surrounding scopes, as with normal lookups. As one special > case, a comprehension at class scope will **eagerly bind** any name which > is already defined in the class scope. > > A list comprehension can be unrolled into an equivalent function. With > Python 3.7 semantics:: > > numbers = [x + y for x in range(3) for y in range(4)] > # Is approximately equivalent to > def (iterator): > result = [] > for x in iterator: > for y in range(4): > result.append(x + y) > return result > numbers = (iter(range(3))) > > Under the new semantics, this would instead be equivalent to:: > > def (): > result = [] > for x in range(3): > for y in range(4): > result.append(x + y) > return result > numbers = () > > When a class scope is involved, a naive transformation into a function > would > prevent name lookups (as the function would behave like a method):: > > class X: > names = ["Fred", "Barney", "Joe"] > prefix = "> " > prefixed_names = [prefix + name for name in names] > > With Python 3.7 semantics, this will evaluate the outermost iterable at > class > scope, which will succeed; but it will evaluate everything else in a > function:: > > class X: > names = ["Fred", "Barney", "Joe"] > prefix = "> " > def (iterator): > result = [] > for name in iterator: > result.append(prefix + name) > return result > prefixed_names = (iter(names)) > > The name ``prefix`` is thus searched for at global scope, ignoring the > class > name. Under the proposed semantics, this name will be eagerly bound; and > the > same early binding then handles the outermost iterable as well. The list > comprehension is thus approximately equivalent to:: > > class X: > names = ["Fred", "Barney", "Joe"] > prefix = "> " > def (names=names, prefix=prefix): > result = [] > for name in names: > result.append(prefix + name) > return result > prefixed_names = () > > With list comprehensions, this is unlikely to cause any confusion. With > generator expressions, this has the potential to affect behaviour, as the > eager binding means that the name could be rebound between the creation of > the genexp and the first call to ``next()``. It is, however, more closely > aligned to normal expectations. The effect is ONLY seen with names that > are looked up from class scope; global names (eg ``range()``) will still > be late-bound as usual. > > One consequence of this change is that certain bugs in genexps will not > be detected until the first call to ``next()``, where today they would be > caught upon creation of the generator. > > > Recommended use-cases > ===================== > > Simplifying list comprehensions > ------------------------------- > > A list comprehension can map and filter efficiently by capturing > the condition:: > > results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0] > > Similarly, a subexpression can be reused within the main expression, by > giving it a name on first use:: > > stuff = [[y := f(x), x/y] for x in range(5)] > > # There are a number of less obvious ways to spell this in current > # versions of Python, such as: > > # Inline helper function > stuff = [(lambda y: [y,x/y])(f(x)) for x in range(5)] > > # Extra 'for' loop - potentially could be optimized internally > stuff = [[y, x/y] for x in range(5) for y in [f(x)]] > > # Using a mutable cache object (various forms possible) > c = {} > stuff = [[c.update(y=f(x)) or c['y'], x/c['y']] for x in range(5)] > > In all cases, the name is local to the comprehension; like iteration > variables, > it cannot leak out into the surrounding context. > > > Capturing condition values > -------------------------- > > Assignment expressions can be used to good effect in the header of > an ``if`` or ``while`` statement:: > > # Proposed syntax > while (command := input("> ")) != "quit": > print("You entered:", command) > > # Capturing regular expression match objects > # See, for instance, Lib/pydoc.py, which uses a multiline spelling > # of this effect > if match := re.search(pat, text): > print("Found:", match.group(0)) > > # Reading socket data until an empty string is returned > while data := sock.read(): > print("Received data:", data) > > # Equivalent in current Python, not caring about function return value > while input("> ") != "quit": > print("You entered a command.") > > # To capture the return value in current Python demands a four-line > # loop header. > while True: > command = input("> "); > if command == "quit": > break > print("You entered:", command) > > Particularly with the ``while`` loop, this can remove the need to have an > infinite loop, an assignment, and a condition. It also creates a smooth > parallel between a loop which simply uses a function call as its condition, > and one which uses that as its condition but also uses the actual value. > > > Rejected alternative proposals > ============================== > > Proposals broadly similar to this one have come up frequently on > python-ideas. > Below are a number of alternative syntaxes, some of them specific to > comprehensions, which have been rejected in favour of the one given above. > > > Alternative spellings > --------------------- > > Broadly the same semantics as the current proposal, but spelled > differently. > > 1. ``EXPR as NAME``:: > > stuff = [[f(x) as y, x/y] for x in range(5)] > > Since ``EXPR as NAME`` already has meaning in ``except`` and ``with`` > statements (with different semantics), this would create unnecessary > confusion or require special-casing (eg to forbid assignment within the > headers of these statements). > > 2. ``EXPR -> NAME``:: > > stuff = [[f(x) -> y, x/y] for x in range(5)] > > This syntax is inspired by languages such as R and Haskell, and some > programmable calculators. (Note that a left-facing arrow ``y <- f(x)`` > is > not possible in Python, as it would be interpreted as less-than and > unary > minus.) This syntax has a slight advantage over 'as' in that it does not > conflict with ``with`` and ``except`` statements, but otherwise is > equivalent. > > 3. Adorning statement-local names with a leading dot:: > > stuff = [[(f(x) as .y), x/.y] for x in range(5)] # with "as" > stuff = [[(.y := f(x)), x/.y] for x in range(5)] # with ":=" > > This has the advantage that leaked usage can be readily detected, > removing > some forms of syntactic ambiguity. However, this would be the only > place > in Python where a variable's scope is encoded into its name, making > refactoring harder. > > 4. Adding a ``where:`` to any statement to create local name bindings:: > > value = x**2 + 2*x where: > x = spam(1, 4, 7, q) > > Execution order is inverted (the indented body is performed first, > followed > by the "header"). This requires a new keyword, unless an existing > keyword > is repurposed (most likely ``with:``). See PEP 3150 for prior > discussion > on this subject (with the proposed keyword being ``given:``). > > 5. ``TARGET from EXPR``:: > > stuff = [[y from f(x), x/y] for x in range(5)] > > This syntax has fewer conflicts than ``as`` does (conflicting only with > the > ``raise Exc from Exc`` notation), but is otherwise comparable to it. > Instead > of paralleling ``with expr as target:`` (which can be useful but can > also be > confusing), this has no parallels, but is evocative. > > > Special-casing conditional statements > ------------------------------------- > > One of the most popular use-cases is ``if`` and ``while`` statements. > Instead > of a more general solution, this proposal enhances the syntax of these two > statements to add a means of capturing the compared value:: > > if re.search(pat, text) as match: > print("Found:", match.group(0)) > > This works beautifully if and ONLY if the desired condition is based on the > truthiness of the captured value. It is thus effective for specific > use-cases (regex matches, socket reads that return `''` when done), and > completely useless in more complicated cases (eg where the condition is > ``f(x) < 0`` and you want to capture the value of ``f(x)``). It also has > no benefit to list comprehensions. > > Advantages: No syntactic ambiguities. Disadvantages: Answers only a > fraction > of possible use-cases, even in ``if``/``while`` statements. > > > Special-casing comprehensions > ----------------------------- > > Another common use-case is comprehensions (list/set/dict, and genexps). As > above, proposals have been made for comprehension-specific solutions. > > 1. ``where``, ``let``, or ``given``:: > > stuff = [(y, x/y) where y = f(x) for x in range(5)] > stuff = [(y, x/y) let y = f(x) for x in range(5)] > stuff = [(y, x/y) given y = f(x) for x in range(5)] > > This brings the subexpression to a location in between the 'for' loop > and > the expression. It introduces an additional language keyword, which > creates > conflicts. Of the three, ``where`` reads the most cleanly, but also has > the > greatest potential for conflict (eg SQLAlchemy and numpy have ``where`` > methods, as does ``tkinter.dnd.Icon`` in the standard library). > > 2. ``with NAME = EXPR``:: > > stuff = [(y, x/y) with y = f(x) for x in range(5)] > > As above, but reusing the `with` keyword. Doesn't read too badly, and > needs > no additional language keyword. Is restricted to comprehensions, though, > and cannot as easily be transformed into "longhand" for-loop syntax. Has > the C problem that an equals sign in an expression can now create a name > binding, rather than performing a comparison. Would raise the question > of > why "with NAME = EXPR:" cannot be used as a statement on its own. > > 3. ``with EXPR as NAME``:: > > stuff = [(y, x/y) with f(x) as y for x in range(5)] > > As per option 2, but using ``as`` rather than an equals sign. Aligns > syntactically with other uses of ``as`` for name binding, but a simple > transformation to for-loop longhand would create drastically different > semantics; the meaning of ``with`` inside a comprehension would be > completely different from the meaning as a stand-alone statement, while > retaining identical syntax. > > Regardless of the spelling chosen, this introduces a stark difference > between > comprehensions and the equivalent unrolled long-hand form of the loop. It > is > no longer possible to unwrap the loop into statement form without reworking > any name bindings. The only keyword that can be repurposed to this task is > ``with``, thus giving it sneakily different semantics in a comprehension > than > in a statement; alternatively, a new keyword is needed, with all the costs > therein. > > > Lowering operator precedence > ---------------------------- > > There are two logical precedences for the ``:=`` operator. Either it should > bind as loosely as possible, as does statement-assignment; or it should > bind > more tightly than comparison operators. Placing its precedence between the > comparison and arithmetic operators (to be precise: just lower than bitwise > OR) allows most uses inside ``while`` and ``if`` conditions to be spelled > without parentheses, as it is most likely that you wish to capture the > value > of something, then perform a comparison on it:: > > pos = -1 > while pos := buffer.find(search_term, pos + 1) >= 0: > ... > > Once find() returns -1, the loop terminates. If ``:=`` binds as loosely as > ``=`` does, this would capture the result of the comparison (generally > either > ``True`` or ``False``), which is less useful. > > While this behaviour would be convenient in many situations, it is also > harder > to explain than "the := operator behaves just like the assignment > statement", > and as such, the precedence for ``:=`` has been made as close as possible > to > that of ``=``. > > > Migration path > ============== > > The semantic changes to list/set/dict comprehensions, and more so to > generator > expressions, may potentially require migration of code. In many cases, the > changes simply make legal what used to raise an exception, but there are > some > edge cases that were previously legal and now are not, and a few corner > cases > with altered semantics. > > > The Outermost Iterable > ---------------------- > > As of Python 3.7, the outermost iterable in a comprehension is special: it > is > evaluated in the surrounding context, instead of inside the comprehension. > Thus it is permitted to contain a ``yield`` expression, to use a name also > used elsewhere, and to reference names from class scope. Also, in a genexp, > the outermost iterable is pre-evaluated, but the rest of the code is not > touched until the genexp is first iterated over. Class scope is now handled > more generally (see above), but if other changes require the old behaviour, > the iterable must be explicitly elevated from the comprehension:: > > # Python 3.7 > def f(x): > return [x for x in x if x] > def g(): > return [x for x in [(yield 1)]] > # With PEP 572 > def f(x): > return [y for y in x if y] > def g(): > sent_item = (yield 1) > return [x for x in [sent_item]] > > This more clearly shows that it is g(), not the comprehension, which is > able > to yield values (and is thus a generator function). The entire > comprehension > is consistently in a single scope. > > The following expressions would, in Python 3.7, raise exceptions > immediately. > With the removal of the outermost iterable's special casing, they are now > equivalent to the most obvious longhand form:: > > gen = (x for x in rage(10)) # NameError > gen = (x for x in 10) # TypeError (not iterable) > gen = (x for x in range(1/0)) # ZeroDivisionError > > def (): > for x in rage(10): > yield x > gen = () # No exception yet > tng = next(gen) # NameError > > > Open questions > ============== > > Importing names into comprehensions > ----------------------------------- > > A list comprehension can use and update local names, and they will retain > their values from one iteration to another. It would be convenient to use > this feature to create rolling or self-effecting data streams:: > > progressive_sums = [total := total + value for value in data] > > This will fail with UnboundLocalError due to ``total`` not being > initalized. > Simply initializing it outside of the comprehension is insufficient - > unless > the comprehension is in class scope:: > > class X: > total = 0 > progressive_sums = [total := total + value for value in data] > > At other scopes, it may be beneficial to have a way to fetch a value from > the > surrounding scope. Should this be automatic? Should it be controlled with a > keyword? Hypothetically (and using no new keywords), this could be > written:: > > total = 0 > progressive_sums = [total := total + value > import nonlocal total > for value in data] > > Translated into longhand, this would become:: > > total = 0 > def (total=total): > result = [] > for value in data: > result.append(total := total + value) > return result > progressive_sums = () > > ie utilizing the same early-binding technique that is used at class scope. > > > Frequently Raised Objections > ============================ > > Why not just turn existing assignment into an expression? > --------------------------------------------------------- > > C and its derivatives define the ``=`` operator as an expression, rather > than > a statement as is Python's way. This allows assignments in more contexts, > including contexts where comparisons are more common. The syntactic > similarity > between ``if (x == y)`` and ``if (x = y)`` belies their drastically > different > semantics. Thus this proposal uses ``:=`` to clarify the distinction. > > > This could be used to create ugly code! > --------------------------------------- > > So can anything else. This is a tool, and it is up to the programmer to > use it > where it makes sense, and not use it where superior constructs can be used. > > > With assignment expressions, why bother with assignment statements? > ------------------------------------------------------------------- > > The two forms have different flexibilities. The ``:=`` operator can be > used > inside a larger expression; the ``=`` statement can be augmented to ``+=`` > and > its friends, can be chained, and can assign to attributes and subscripts. > > > Why not use a sublocal scope and prevent namespace pollution? > ------------------------------------------------------------- > > Previous revisions of this proposal involved sublocal scope (restricted to > a > single statement), preventing name leakage and namespace pollution. While > a > definite advantage in a number of situations, this increases complexity in > many others, and the costs are not justified by the benefits. In the > interests > of language simplicity, the name bindings created here are exactly > equivalent > to any other name bindings, including that usage at class or module scope > will > create externally-visible names. This is no different from ``for`` loops > or > other constructs, and can be solved the same way: ``del`` the name once it > is > no longer needed, or prefix it with an underscore. > > Names bound within a comprehension are local to that comprehension, even in > the outermost iterable, and can thus be used freely without polluting the > surrounding namespace. > > (The author wishes to thank Guido van Rossum and Christoph Groth for their > suggestions to move the proposal in this direction. [2]_) > > > Style guide recommendations > =========================== > > As expression assignments can sometimes be used equivalently to statement > assignments, the question of which should be preferred will arise. For the > benefit of style guides such as PEP 8, two recommendations are suggested. > > 1. If either assignment statements or assignment expressions can be > used, prefer statements; they are a clear declaration of intent. > > 2. If using assignment expressions would lead to ambiguity about > execution order, restructure it to use statements instead. > > > Acknowledgements > ================ > > The author wishes to thank Guido van Rossum and Nick Coghlan for their > considerable contributions to this proposal, and to members of the > core-mentorship mailing list for assistance with implementation. > > > References > ========== > > .. [1] Proof of concept / reference implementation > (https://github.com/Rosuav/cpython/tree/assignment-expressions) > .. [2] Pivotal post regarding inline assignment semantics > (https://mail.python.org/pipermail/python-ideas/2018-March/049409.html) > > > Copyright > ========= > > This document has been placed in the public domain. > > > .. > Local Variables: > mode: indented-text > indent-tabs-mode: nil > sentence-end-double-space: t > fill-column: 70 > coding: utf-8 > End: > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukasz at langa.pl Thu Apr 26 14:28:37 2018 From: lukasz at langa.pl (Lukasz Langa) Date: Thu, 26 Apr 2018 11:28:37 -0700 Subject: [Python-Dev] Reserve ':=' for type-inferred variable initialization (was PEP 572) In-Reply-To: References: Message-ID: > On Apr 26, 2018, at 11:00 AM, Fatty Morgan wrote: > > I would like to urge you to reconsider the use of the token ':=' > for assignment expressions. > > The natural interpretation of 'name := expr' is a PEP 526 > type-annotated variable initialization 'name : T = expr' with the > type annotation T omitted, the tokens ':' and '=' coalesced, and > the implied type T inferred as 'type(expr)'. PEP 484 type checkers *always* infer the type of an assignment based on the right-hand side's value. The main reason we require PEP 526 annotations at all is when your initial assignment is not useful for this inference. The two most common examples of this are `None` and empty containers. In those cases the type checker is unable to infer what the *intended* value should be, so it urges the programmer to explicitly state it. In this sense, `a := []` or `a := None` doesn't provide any additional information. And `a = 1` is already enough to determine what the expected type is. > The following should probably be discussed in python-ideas and > would only be relevant for a more distant future, but while I > have your attention, here are a couple of thoughts: Discussing this here will add to noise for Chris and the BDFL. Consider re-posting in python-ideas. -- ? From larry at hastings.org Thu Apr 26 14:33:04 2018 From: larry at hastings.org (Larry Hastings) Date: Thu, 26 Apr 2018 11:33:04 -0700 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: <538aea0d-11d2-8542-4d18-bbcb34d9848c@hastings.org> On 04/26/2018 11:18 AM, Chris Angelico wrote: > In the reference implementation, it's just DUP_TOP followed by > STORE_FAST (well, technically by "whatever the assignment compiles > to", as it could be affected by global/nonlocal, closures, etc). Is > there much advantage to creating a new opcode? Probably not much, but I thought we now lived in an age of wonders where common sequences of opcodes were getting mashed together into new more-complicated-but-redundant bytecodes (e.g. BUILD_MAP_UNPACK_WITH_CALL) just to save on dispatch overhead. You're right, it'd be a micro-optimization, and its value would be debatable. //arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From storchaka at gmail.com Thu Apr 26 14:37:10 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Thu, 26 Apr 2018 21:37:10 +0300 Subject: [Python-Dev] Boundaries between numbers and identifiers Message-ID: In Python 2.5 `0or[]` was accepted by the Python parser. It became an error in 2.6 because "0o" became recognizing as an incomplete octal number. `1or[]` still is accepted. On other hand, `1if 2else 3` is accepted despites the fact that "2e" can be recognized as an incomplete floating point number. In this case the tokenizer pushes "e" back and returns "2". Shouldn't it do the same with "0o"? It is possible to make `0or[]` be parseable again. Python implementation is able to tokenize this example: $ echo '0or[]' | ./python -m tokenize 1,0-1,1: NUMBER '0' 1,1-1,3: NAME 'or' 1,3-1,4: OP '[' 1,4-1,5: OP ']' 1,5-1,6: NEWLINE '\n' 2,0-2,0: ENDMARKER '' On other hand, all these examples look weird. There is an assymmetry: `1or 2` is a valid syntax, but `1 or2` is not. It is hard to recognize visually the boundary between a number and the following identifier or keyword, especially if numbers can contain letters ("b", "e", "j", "o", "x") and underscores, and identifiers can contain digits. On both sides of the boundary can be letters, digits, and underscores. I propose to change the Python syntax by adding a requirement that there should be a whitespace or delimiter between a numeric literal and the following keyword. From tjreedy at udel.edu Thu Apr 26 14:54:20 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 26 Apr 2018 14:54:20 -0400 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <20180426075628.GH7400@ando.pearwood.info> Message-ID: On 4/26/2018 6:20 AM, Steve Holden wrote: > On Thu, Apr 26, 2018 at 8:56 AM, Steven D'Aprano > wrote: > > On Thu, Apr 26, 2018 at 03:31:13AM -0400, Terry Reedy wrote: > > On 4/25/2018 8:20 PM, Chris Angelico wrote: > > >On Thu, Apr 26, 2018 at 10:11 AM, Yury Selivanov > > >> wrote: > > >>Just yesterday this snippet was used on python-dev to show how great the > > >>new syntax is: > > >> > > >>? ? ? ? ? ?my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf)) > > > > What strikes me as awful about this example is that len(buf) is > > get_size(), so the wrong value is being named and saved. > > 'size=len(buf)' is, in a sense, backwards. > > Terry is absolutely right, and I'm to blame for that atrocity. Mea > culpa. > > ?Perhaps a better spelling would be > > ? ? my_func(arg, buffer=[None]*(buflen := get_size()), size=buflen) That is exactly what I wrote in the continuation that Steven snipped. -- Terry Jan Reedy From encukou at gmail.com Thu Apr 26 14:56:49 2018 From: encukou at gmail.com (Petr Viktorin) Date: Thu, 26 Apr 2018 14:56:49 -0400 Subject: [Python-Dev] PEP 573 -- Module State Access from C Extension Methods In-Reply-To: References: Message-ID: <055c9eb7-d53c-bfd0-7599-46f5a43569af@gmail.com> On 04/26/18 09:32, Thomas Wouters wrote: > > Thanks for working on this, Marcel (and Petr). This looks like an > ambitious intern project :) Couple of questions and comments in-line. Indeed, but also a multi-year one. Sometimes it seems like Python is moving too fast under us! Specifically, PEP 573 (Unifying function/method classes) andto a smaller extent PEP 567 (Context Variables) seem to be solving similar problems as this PEP. I'm inclined to put this PEP temporarily on hold, help 575 get accepted, and then rebase on top of that. > On Mon, Apr 23, 2018 at 12:36 PM, Marcel Plch > wrote: > PEP: 573 [...] > > Abstract > ======== > [...] > > Additionaly, support for easier creation of immutable exception > classes is added. > > > I'm not a fan of using 'immutable' here, or in the API function name. I > understand the types are to some extent immutable (apart from their > refcount, I assume), but I think it's going to be too easy to confuse it > with types whose *instances* are immutable. (We do occasionally say > things like "tuples are an immutable type".) Since the point is that > they behave like statically defined ones, perhaps 'Static' would be a > reasonable replacement. That was the first naming choice, but then on the other hand, "static heap type" sounds like an oxymoron -- in C, static things don't live on the heap. Naming is hard :( [...] > Slot methods > ------------ > > The above changes don't cover slot methods, such as ``tp_iter`` or > ``nb_add``. > > The problem with slot methods is that their C API is fixed, so we can't > simply add a new argument to pass in the defining class. > Two possible solutions have been proposed to this problem: > > ? ? * Look up the class through walking the MRO. > ? ? ? This is potentially expensive, but will be useful if > performance is not > ? ? ? a problem (such as when raising a module-level exception). > ? ? * Storing a pointer to the defining class of each slot in a > separate table, > ? ? ? ``__typeslots__`` [#typeslots-mail]_.? This is technically > feasible and fast, > ? ? ? but quite invasive. > > Due to the invasiveness of the latter approach, this PEP proposes > adding an MRO walking > helper for use in slot method implementations, deferring the more > complex alternative > as a potential future optimisation. Modules affected by this concern > also have the > option of using thread-local state or PEP 567 context variables, or > else defining their > own reload-friendly lookup caching scheme. > > > I do not believe walking the MRO is going to work without reworking the > implementation of types, specifically how typeobject.c deals with slots > of subclasses: in some cases copies the slots from the base class (see > inherit_slots() and from where it's called). I believe this would cause > problems if, for example, you define type X in module A, subclass it > from type Y in module B without overriding the slot, and try to find the > module object for A from the slot implementation. I don't think copying > slots is a requirement for the desired semantics, but it's going to be > fairly involved to rewrite it to do something else. There's also > backward-compatibility to consider: third-party libraries can be > inheriting from builtin types (e.g. numpy does this extensively) using > the same copying-slot mechanism, which means those builtin types can't > use the MRO walking to find their module without breaking compatibility > with those third-party libraries. The MRO walking code needs to deal with copied slots. It's not straigtforward, but I do think it's possible. [...] > Static exceptions > ----------------- > > A new function will be added:: > > ? ? int PyErr_PrepareImmutableException(PyTypeObject **exc, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?const char *name, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?const char *doc, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?PyObject *base) > > Creates an immutable exception type which can be shared > across multiple module objects. > > > How is this going to deal with type.__subclasses__()? Is re-using the > static type object between reloads and sub-interpreters important enough > to warrant the different behaviour? What if sub-interpreters end up > wanting to disallow sharing objects between them? Argh. Yes, subclasses seem to be the obvious shortcoming that a fresh pair of eyes is bound to find. Thanks! This sends the proposal back to the drawing board. [...] > Modules Converted in the Initial Implementation > ----------------------------------------------- > > To validate the approach, several modules will be modified during > the initial implementation: > > The ``zipimport``, ``_io``, ``_elementtree``, and ``_csv`` modules > will be ported to PEP 489 multiphase initialization. > > > zipimport currently caches things in C globals. Changing it to use PEP > 489 multi-phase initialisation is very likely going to change semantics > in subtle ways... Is it really worth the risk? The whole excercise is to show that existing modules *can* be ported to the new approach, without unacceptable changes to semantics -- and in cases where they can't, to improve the situation. So even if changes to zipimport ultimately aren't accepted, it makes sense to attempt this. From brett at python.org Thu Apr 26 14:28:53 2018 From: brett at python.org (Brett Cannon) Date: Thu, 26 Apr 2018 18:28:53 +0000 Subject: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 In-Reply-To: References: Message-ID: On Thu, 26 Apr 2018 at 10:19 Barry Warsaw wrote: > On Apr 26, 2018, at 09:28, Eric Snow wrote: > > > > On Thu, Apr 26, 2018 at 10:25 AM, Eric Snow > wrote: > >> In pondering our approach to future Python major releases, I found > >> myself considering the experience we've had with Python 3. The whole > >> Py3k effort predates my involvement in the community so I missed a > >> bunch of context about the motivations, decisions, and challenges. > >> While I've pieced some of that together over the years now since I've > >> been around, I've certainly seen much of the aftermath. For me, at > >> least, it would be helpful to have a bit more insight into the > >> history. :) > > It would certainly be an interesting document, but I suspect you?ll get a > bit of the old ?ask 3 lawyers and get 5 opinions? kind of response. ;) > > As I remember it, there was definitely a feeling like, this would be our > only chance to clean up some annoying cruft, and rectify some (in > hindsight) incorrect design decisions made over the years, couple with a > healthy dose of ?we have no idea how to do the bytes/str split in a > backward compatible way". There was probably a sense that the Python > community was just small enough to be able to handle such a disruptive > change, but wouldn?t ever be so again. The latter is definitely true > today, even if the former was overly optimistic. > I agree with everything Barry said. There are some lessons in hindsight of how we could have handled bytes/str, but it was more of a decision of "really long transition versus a short one" -- jokes on us for what "short" became ;) -- which we simply won't make ever again. -------------- next part -------------- An HTML attachment was scrubbed... URL: From schesis at gmail.com Thu Apr 26 15:00:46 2018 From: schesis at gmail.com (Zero Piraeus) Date: Thu, 26 Apr 2018 20:00:46 +0100 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: Message-ID: : On 25 April 2018 at 21:28, Guido van Rossum wrote: > A very emotional appeal, you don't seem to grasp the usability improvements > this will give. I hear you but at this point appeals to Python's "Zen" don't > help you. I have to admit, in half-following this discussion I've swung between thinking "there's no way this is actually going to happen" and "what, this might actually happen?" Since it now looks like it really *does* have a decent chance, and maybe another -1 has a small chance of tipping the balance: my reaction to the proposal is also emotional. Visceral, in fact, to the extent that I'd aim to read and write less Python if it became commonplace. I don't have arguments. I just have an instinctive "ugh, no". -[]z. From lukasz at langa.pl Thu Apr 26 15:02:22 2018 From: lukasz at langa.pl (Lukasz Langa) Date: Thu, 26 Apr 2018 12:02:22 -0700 Subject: [Python-Dev] Boundaries between numbers and identifiers In-Reply-To: References: Message-ID: > On Apr 26, 2018, at 11:37 AM, Serhiy Storchaka wrote: > > I propose to change the Python syntax by adding a requirement that there should be a whitespace or delimiter between a numeric literal and the following keyword. -1 This would make Python 3.8 reject code due to stylistic preference. Code that it actually can unambiguously parse today. I agree that a formatting style that omits whitespace between numerals and other tokens is terrible. However, if you start downright rejecting it, you will likely punish the wrong people. Users of third-party libraries will be met with random parsing errors in files they have no control over. This is not helpful. And given BPO-33338 the standard library tokenizer would have to keep parsing those things as is. Making 0or[] working again is also not worth it since that's been broken since Python 2.6 and hopefully nobody is running Python 2.5-only code anymore. What we should instead is to make the standard library tokenizer reflect the behavior of Python 2.6+. -- ? From tjreedy at udel.edu Thu Apr 26 15:05:23 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 26 Apr 2018 15:05:23 -0400 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <5AE1AB35.6090609@canterbury.ac.nz> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <5AE17CCC.5070707@canterbury.ac.nz> <20180426072917.GG7400@ando.pearwood.info> <20180426103059.0fdbfc4e@fsol> <5AE1AB35.6090609@canterbury.ac.nz> Message-ID: On 4/26/2018 6:34 AM, Greg Ewing wrote: > Antoine Pitrou wrote: >> Well, how do languages where assignment is an expression returning the >> assigned value make their REPLs work?? I'm sure they don't inflict that >> on their users, so it's certainly a solvable problem. > > I can't think of any such language that has a REPL > offhand, but here's a possible solution: > > > ?? x := expr # doesn't print anything Ugh! The only reason I would bother typing the : in a top level assignment would be to get the print without having to retype the name, as in >>> x = expr >>> x I consider echoing top-level interactive assignments to be a feature of the proposal. -- Terry Jan Reedy From tim.peters at gmail.com Thu Apr 26 15:12:21 2018 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 26 Apr 2018 14:12:21 -0500 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: [Kirill Balunov] > Not sure, but if additional motivating examples are required, there is a > common pattern for dynamic attribute lookup (snippet from `copy.py`): > > reductor = dispatch_table.get(cls) > if reductor: > rv = reductor(x) > else: > reductor = getattr(x, "__reduce_ex__", None) > if reductor: > rv = reductor(4) > else: > reductor = getattr(x, "__reduce__", None) > if reductor: > rv = reductor() > else: > raise Error("un(shallow)copyable object of type %s" % cls) > > which can with the current `binding expression` syntax simplified to: > > if reductor := dispatch_table.get(cls): > rv = reductor(x) > elif reductor := getattr(x, "__reduce_ex__", None): > rv = reductor(4) > elif reductor := getattr(x, "__reduce__", None): > rv = reductor() > else: > raise Error("un(shallow)copyable object of type %s" % cls) > > which becomes much clearer, at least in my opinion. [Larry Hastings ] > I hate to be pedantic--there's enough of that going on in this thread--but I > can't agree with the word "simplifed" above. I agree that the code using > binding expressions is shorter. But considering that emit the two code > examples implement the exact same algorithm, to the point where their > bytecode would look nearly* identical, ISTM that the two code examples are > of identical complexity. In the absence of defining an objectively computable complexity measure, I expect you're doomed to arguing taste. For example, argue that both spellings have the same formal "cyclomatic complexity" measure (which they do). By other formal measures (e.g., total number of identifier instances), the latter spelling is "objectively simpler". By yet others (e.g., total number of non-whitespace characters divided by total number of lines), the former spelling is "objectively simpler". But that all kinda misses the point to me: the latter spelling is "obviously simpler" in a way that _actually matters_, for the same reason, e.g., a case statement with N cases is "obviously simpler" than the semantically equivalent spelling using N nested if/else if/else if/else if/else ... blocks. The latter spelling above is indeed visually very much like a case statement: all the tests are at the same indentation level, and all the conditional actions are too. It's obvious _at a glance_ in the latter that exactly one of the action blocks will be performed. That's how if/elif/elif/else always works. It's not at all obvious at a glance (certainly not to me) in the original spelling. > Comparing the two, the code using the binding expressions obviates four > newlines, three uses of the identifier "reductor", and allows folding two > "else / if"s into "elif"s. In exchange, it adds three extra colons, If it weren't for that you hate being pedantic, I'd add that you're overlooking the piles of leading whitespace characters also saved in the latter ;-) The number of those saved grows quadratically in the number of uselessly indented blocks shifted left. > and the density of complexity per line has shot up. Average non-whitespace character count per line has certainly shot up, but I don't actually know what you mean by "density of complexity" there. Just FYI, when I write long if/elif/elif/... chains, I typically put a blank line before each elif, to give better visual separation of the peer (both semantically and visually) test-action blocks. Which has nothing to do with any formal notion of complexity, because I don't much care about that - readability is what I value, and that's not the same as any formal notion of complexity I've ever seen. > ... From storchaka at gmail.com Thu Apr 26 15:25:09 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Thu, 26 Apr 2018 22:25:09 +0300 Subject: [Python-Dev] Order of positional and keyword arguments Message-ID: There is an inconsistence in passing arguments to functions. Explicit positional arguments should precede keyword arguments (both explicit and variable), but variable positional arguments can follow explicit keyword arguments and precede variable keyword arguments. For example, for function def f(a=None, b=None): return a, b the following is valid syntax: f(1, b=2) f(1, **{'b': 2}) f(*[1], b=2) f(*[1], **{'b': 2}) f(b=2, *[1]) but the following is an error: f(b=2, 1) f(**{'b': 2}, 1) f(**{'b': 2}, *[1]) f(b=2, *[1]) is surprised in two ways: 1. Argument values are passed not in order. The first value is assigned to the second parameter, and the second value is assigned to the first parameter. 2. Argument values are evaluated not from left to right. This contradicts the general rule that expressions are evaluated from left to right (with few known exceptions). I never seen the form `f(b=2, *[1])` in practice (though the language reference contains an explicit example for it), and it looks weird to me. I don't see reasons of writing `f(b=2, *[1])` instead of more natural `f(*[1], b=2)`. I propose to disallow it. This will also make the grammar simpler. Current grammar: argument_list: `positional_arguments` ["," `starred_and_keywords`] : ["," `keywords_arguments`] : | `starred_and_keywords` ["," `keywords_arguments`] : | `keywords_arguments` positional_arguments: ["*"] `expression` ("," ["*"] `expression`)* starred_and_keywords: ("*" `expression` | `keyword_item`) : ("," "*" `expression` | "," `keyword_item`)* keywords_arguments: (`keyword_item` | "**" `expression`) : ("," `keyword_item` | "," "**" `expression`)* keyword_item: `identifier` "=" `expression` Proposed grammar: argument_list: `positional_arguments` ["," `keywords_arguments`] : | `keywords_arguments` positional_arguments: ["*"] `expression` ("," ["*"] `expression`)* keywords_arguments: `keyword_argument` ("," `keyword_argument`)* keyword_argument: `identifier` "=" `expression` | "**" `expression` From p.f.moore at gmail.com Thu Apr 26 15:52:18 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 26 Apr 2018 20:52:18 +0100 Subject: [Python-Dev] Order of positional and keyword arguments In-Reply-To: References: Message-ID: I see no practical benefit to making such a restriction, and there's a risk of breaking existing code. While it's not something I've ever used myself in real code, def wrapper(*args, **kw): return wrapped_fn(some_arg=1, *args, **kw) seems like a perfectly reasonable way to write a wrapper to me. I'm against breaking this for no better reason than "to simplify the grammar" and to exclude cases that some people might find confusing (I actually found your example perfectly comprehensible, if convoluted). -1 from me. Paul On 26 April 2018 at 20:25, Serhiy Storchaka wrote: > There is an inconsistence in passing arguments to functions. > > Explicit positional arguments should precede keyword arguments (both > explicit and variable), but variable positional arguments can follow > explicit keyword arguments and precede variable keyword arguments. > > For example, for function > > def f(a=None, b=None): > return a, b > > the following is valid syntax: > > f(1, b=2) > f(1, **{'b': 2}) > f(*[1], b=2) > f(*[1], **{'b': 2}) > f(b=2, *[1]) > > but the following is an error: > > f(b=2, 1) > f(**{'b': 2}, 1) > f(**{'b': 2}, *[1]) > > f(b=2, *[1]) is surprised in two ways: > > 1. Argument values are passed not in order. The first value is assigned to > the second parameter, and the second value is assigned to the first > parameter. > > 2. Argument values are evaluated not from left to right. This contradicts > the general rule that expressions are evaluated from left to right (with few > known exceptions). > > I never seen the form `f(b=2, *[1])` in practice (though the language > reference contains an explicit example for it), and it looks weird to me. I > don't see reasons of writing `f(b=2, *[1])` instead of more natural `f(*[1], > b=2)`. I propose to disallow it. > > This will also make the grammar simpler. Current grammar: > > argument_list: `positional_arguments` ["," `starred_and_keywords`] > : ["," `keywords_arguments`] > : | `starred_and_keywords` ["," `keywords_arguments`] > : | `keywords_arguments` > positional_arguments: ["*"] `expression` ("," ["*"] `expression`)* > starred_and_keywords: ("*" `expression` | `keyword_item`) > : ("," "*" `expression` | "," `keyword_item`)* > keywords_arguments: (`keyword_item` | "**" `expression`) > : ("," `keyword_item` | "," "**" `expression`)* > keyword_item: `identifier` "=" `expression` > > Proposed grammar: > > argument_list: `positional_arguments` ["," `keywords_arguments`] > : | `keywords_arguments` > positional_arguments: ["*"] `expression` ("," ["*"] `expression`)* > keywords_arguments: `keyword_argument` ("," `keyword_argument`)* > keyword_argument: `identifier` "=" `expression` | "**" `expression` > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/p.f.moore%40gmail.com From storchaka at gmail.com Thu Apr 26 15:53:00 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Thu, 26 Apr 2018 22:53:00 +0300 Subject: [Python-Dev] Boundaries between numbers and identifiers In-Reply-To: References: Message-ID: <2aa2f899-8f79-2950-f461-76c63a807b43@gmail.com> 26.04.18 22:02, Lukasz Langa ????: >> On Apr 26, 2018, at 11:37 AM, Serhiy Storchaka wrote: >> >> I propose to change the Python syntax by adding a requirement that there should be a whitespace or delimiter between a numeric literal and the following keyword. > -1 > > This would make Python 3.8 reject code due to stylistic preference. Code that it actually can unambiguously parse today. Of course I don't propose to make it a syntax error in 3.8. It should first emit a SyntaxWarning and be converted into an error only in 3.10. Or maybe first add a rule for this in PEP 8 and make it a syntax error in distant future, after all style checkers include it. > I agree that a formatting style that omits whitespace between numerals and other tokens is terrible. However, if you start downright rejecting it, you will likely punish the wrong people. Users of third-party libraries will be met with random parsing errors in files they have no control over. This is not helpful. > > And given BPO-33338 the standard library tokenizer would have to keep parsing those things as is. > > Making 0or[] working again is also not worth it since that's been broken since Python 2.6 and hopefully nobody is running Python 2.5-only code anymore. > > What we should instead is to make the standard library tokenizer reflect the behavior of Python 2.6+. The behavior of the standard library tokenizer doesn't contradict rules. It is the most natural behavior of regex-based tokenizer. Actually the behavior of the building tokenizer can be incorrect. In any case accepting `1if 2else 3` and rejecting `0or[]` looks weird. They should use the same rule. "0or" and "2else" should be considered ambiguous or unambiguous in the same way. From tjreedy at udel.edu Thu Apr 26 16:00:38 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 26 Apr 2018 16:00:38 -0400 Subject: [Python-Dev] PEP 572: Why not := as standard assignment operator? In-Reply-To: References: Message-ID: On 4/26/2018 11:38 AM, Gustavo Carneiro wrote: > On Thu, Apr 26, 2018 at 11:13 PM, Martin Teichmann > > wrote: > > when reading PEP 572 I actually liked it a lot - I think it's actually > > a cool idea. I think it's actually that cool an idea that it should be > > made the default way of doing an assignment, over time phasing out the > > good ole =. Please no, I really, really appreciate not having to see add ':' all the time. Plus, it already has two other uses: end of header and dict key-item separator. Plus, see below. > In the interest of that, do you think := can be made illegal, by the > grammar, if used outside an expression? > > a = 1? # legal > a := 1? # Syntax error > if a := 1:? # legal Please no. PEP 572 does not discuss top-level interactive assignments. But it should, because having two forms of assignment that lets one choose to echo or not is a plus. Sometimes one wants to see the result of an assignment. After >>> a, b = 14387, 344 it would be nice to be able to write >>> diff := a - b 14343 instead of >>> diff = a - b >>> diff 14343 I consider this a significant positive feature of the proposal. On the other hand, sometimes one does not want the see the result. A long result is often useless to view and may scroll previous results out of the window. A very long result may scroll previous results out of a finite console buffer altogether. (For Windows Command Prompt, the default is 300 lines, with a max of 9999.) And sometimes the result must not be printed. Tk text widgets (and hence IDLE's Shell) do not have a particular limit on the number of lines. On the other hands, long lines drasticly slow down a text widget, and a long enough line will freeze it. (This is supposed to be fixed in Tcl/Tk 8.7, now in a1 stage.) For instance, ">>> [None]*1000000" in IDLE freezes the process. -- Terry Jan Reedy From jimjjewett at gmail.com Thu Apr 26 16:18:21 2018 From: jimjjewett at gmail.com (Jim J. Jewett) Date: Thu, 26 Apr 2018 16:18:21 -0400 Subject: [Python-Dev] Nickname Binding (PEP 572) Message-ID: I think part of the disconnect is that this enhancement could very easily be abused, and it seems likely that it will be, because the problems aren't visible while writing the code -- only when reading it later. I therefore suggest making it very clear in the PEP -- and probably in PEP 8 -- how these expressions should be limited. Simply renaming them to "nickname binding" would be start, but here is a rough draft for wording. When scanning code by eye, it is helpful that assignments are (almost) always at the start of a line. Even def and class statements can cause confusion if the reader didn't realize that the name referred to a class, rather than an instance. Moving assignments to the middle of a line will make it harder for someone else to read your code -- so don't do that. A nickname is just a regular name, except that it also suggests an intimate environment. If the name is purely for documentation, or will be used only later in the same expression (or, possibly, the same block or just after), then a nickname may be appropriate. But * If you are wondering what to do about type hints, then the expression is probably too complex to leave inline. Separate it out into a regular assignment statement; nicknames do not support type hints. * If you will be saving the value -- even as an attribute on self -- there is a chance it will be retrieved in a very different context. Use a regular assignment statement; nicknames are just simple names, not attributes or keys. * If you will be using the value somewhere else in your code, use a regular assignment statement. This makes it easier to find, and warns people that the value may be used again later. -jJ From tim.peters at gmail.com Thu Apr 26 16:33:24 2018 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 26 Apr 2018 15:33:24 -0500 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> Message-ID: [Tim] >> So, to match your sarcasm, here's mine: try using a feature for what >> it's good at instead of for what it's bad at ;-) [Lukasz Langa ] > Yes, this is the fundamental wisdom. Judging which is which is left as an > exercise to the programmer. > > With this, I'm leaving the discussion. With Guido and you on board for PEP > 572, I feel that Chris' streak is indeed about to break. I still expect it could go either way, but do wish people didn't believe it will be a major loss if "the other side wins". I'll be fine regardless - and so will everyone else. Guido rarely makes language design mistakes. In this case he's seeing serious opposition from several core developers, and you shouldn't believe either that he just dismisses that. [?ukasz Langa] >>> Well, you have an entire code style built around this feature called Yoda >>> conditions. You teach people on Day 1 to never ever confuse == with =. Some >>> compilers even warn about this because so many people did it wrong. >> Sorry, I couldn't follow that. Part of the problem here is that I had never seen "Yoda conditions" before, and had no idea what it meant. Some later Googling suggests it's "a thing" youngsters say at times ;-) > You implied that newbies don't have to even know about assignments in > expressions. I wanted to demonstrate that this isn't really the case because > mistaking `=` for `==` is a relatively common occurence for newbies. If you > want to argue that it isn't, I'd like to point out that the WordPress code > style *requires* Yoda conditions because it was enough of a hindrance. ESLint > (a JavaScript linter) also has a warning about assignment in a conditional. What does that have to do with Python? If they try to use "=" in an expression now, they get a SyntaxError. The PEP doesn't change anything about that. Indeed, that's why it uses ":=" instead. I have experience in other languages with embedded assignments that also use ":=", and it's _never_ the case that people type ":=" when they intend "equality test" in those. The horrid "I typed = when I meant ==" mistakes are unique to languages that mindlessly copied C. The mistakes aren't primarily due to embedded assignments, they're due to that even highly experienced programmers sometimes type "=" when they're _thinking_ "equals". Nobody types ":=" when they're thinking "equals". > ... > What you're saying is true. But for it to be true, newbies *have to* learn the > distinction, and the fact that yes, sometimes the programmer indeed meant to > put a single `=` sign in the conditional. Again, the PEP is about Python: a single "=" in a conditional is, and will remain, a SyntaxError. So nobody can sanely intend to put a single "=" in a condition _in Python_ unless they're writing a test intending to provoke a syntax error. > That's why we'll end up with the Pascal assignment operator. ":=" is already in the PEP. > And that *is* a thing that you will have to explain to newbies when they encounter > it for the first time. Sure. That doesn't frighten me, though. It's easy to explain what it does - although it may be hard to explain when it's _desirable_ to use it. > Sadly, googling for a colon followed by an equal sign isn't trivial if you don't > know what you're looking for. To judge from Stackoverflow volume, the single most misunderstood of all Python operators - by far - is "is" - try Googling for that ;-) In far second and third places are "and" and "or", for which searches are also useless. Regardless, I'm not concerned about one-time tiny learning curves. Don't know what ":=" means already? Ask someone. If you know what "=" means, you're already close to done. Given that you already understand what "binding a name" means, ":=" may well be the simplest of all Python's operators (there's no computation _to_ be understood, and no possibility either of a dunder method changing its meaning depending on operand type(s)). >>> Well, you can also use it as a statement. But don't! >> Why not? _Every_ expression in Python can be used as a statement. >> Nothing forbids it, and that's even (very!) useful at an interactive >> prompt. > Because it suggests different intent, because it's limited, because it's slower > at runtime, and because PEP 572 says so itself. I didn't say you're _required_ to use it as a statement. Regardless of what PEPs say, people will do what they find most useful. I trust people to figure this out quickly for themselves. >> At this point I think you must have a lower opinion of Python >> programmers than I have ;-) If adding even a dozen characters to a >> line makes it exceed a reasonable line-length guide, the code was >> almost certainly too confusingly dense to begin with. > Around 5% of if and elif statements in the standard library don't fit a single > line *as is*. Sure, that's a low percentage but that's over 1,000 statements. > If you're putting an `if` statement in a method, you are already starting out > with 71 characters left on the line. Four of those are going to be taken by > "if", a space, and the colon. Adding a parenthesized assignment expression > takes at least 10% of that available space. Confirming that you do have a lower opinion of them ;-) Are you picturing people stampeding to introduce ":=" in every place they possibly could? I may be wrong, but I don't expect that at all. I expect a vast majority of uses in real life will replace: name = expression if name: by if name := expression: and while True: name = expression if name comparison expression2: break by while (name := expression) inverted_comparison expression2: _provided that_ the latter spelling doesn't make the line uncomfortably long. In all the code of mine I've seen a good use for it, there's a whole lot of empty horizontal screen space to spare, even after recoding. In places where I already had "long lines", I didn't even check to see whether a binding operation could be used too - why would I? I don't feel _compelled_ to use it - I'm only looking to reduce redundancy where it's an obvious win. > The silver lining for me is that this makes the environment riper for > auto-formatting. See? It's win-win for you too no matter how this turns out ;-) From kirillbalunov at gmail.com Thu Apr 26 14:24:45 2018 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Thu, 26 Apr 2018 21:24:45 +0300 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <20180426075628.GH7400@ando.pearwood.info> Message-ID: 2018-04-26 13:20 GMT+03:00 Steve Holden : > On Thu, Apr 26, 2018 at 8:56 AM, Steven D'Aprano > wrote: > >> On Thu, Apr 26, 2018 at 03:31:13AM -0400, Terry Reedy wrote: >> > On 4/25/2018 8:20 PM, Chris Angelico wrote: >> > >On Thu, Apr 26, 2018 at 10:11 AM, Yury Selivanov >> > > wrote: >> > >>Just yesterday this snippet was used on python-dev to show how great >> the >> > >>new syntax is: >> > >> >> > >> my_func(arg, buffer=(buf := [None]*get_size()), >> size=len(buf)) >> > >> > What strikes me as awful about this example is that len(buf) is >> > get_size(), so the wrong value is being named and saved. >> > 'size=len(buf)' is, in a sense, backwards. >> >> Terry is absolutely right, and I'm to blame for that atrocity. Mea >> culpa. >> >> ?Perhaps a better spelling would be > > my_func(arg, buffer=[None]*(buflen := get_size()), size=buflen) > > I know it is non productive and spamy (I promise, this is the last) since `as` syntax is dead. In many cases, there is not much difference in perception between `:=` and `as`. But in several situations, like this one and as Ethan pointed up-thread - the expression first syntax makes obvious the intent and linearly readable: my_func(arg, buffer=[None]*get_size() as buf, size=buf) In any case, it is rather an anti-pattern than a good example to follow. p.s.: as Victor Stinner wrote on twitter that previously, there was a similar PEP in spirit - "PEP 379 -- Adding an Assignment Expression", which was withdrawn. May be it is worth to make a link to it in the current PEP. With kind regards, -gdg -------------- next part -------------- An HTML attachment was scrubbed... URL: From gvanrossum at gmail.com Thu Apr 26 18:11:59 2018 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu, 26 Apr 2018 22:11:59 +0000 Subject: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 In-Reply-To: References: Message-ID: Also see my talk at PyCascades and Victor's upcoming talk at PyCon. On Thu, Apr 26, 2018, 12:02 Brett Cannon wrote: > > > On Thu, 26 Apr 2018 at 10:19 Barry Warsaw wrote: > >> On Apr 26, 2018, at 09:28, Eric Snow wrote: >> > >> > On Thu, Apr 26, 2018 at 10:25 AM, Eric Snow < >> ericsnowcurrently at gmail.com> wrote: >> >> In pondering our approach to future Python major releases, I found >> >> myself considering the experience we've had with Python 3. The whole >> >> Py3k effort predates my involvement in the community so I missed a >> >> bunch of context about the motivations, decisions, and challenges. >> >> While I've pieced some of that together over the years now since I've >> >> been around, I've certainly seen much of the aftermath. For me, at >> >> least, it would be helpful to have a bit more insight into the >> >> history. :) >> >> It would certainly be an interesting document, but I suspect you?ll get a >> bit of the old ?ask 3 lawyers and get 5 opinions? kind of response. ;) >> >> As I remember it, there was definitely a feeling like, this would be our >> only chance to clean up some annoying cruft, and rectify some (in >> hindsight) incorrect design decisions made over the years, couple with a >> healthy dose of ?we have no idea how to do the bytes/str split in a >> backward compatible way". There was probably a sense that the Python >> community was just small enough to be able to handle such a disruptive >> change, but wouldn?t ever be so again. The latter is definitely true >> today, even if the former was overly optimistic. >> > > I agree with everything Barry said. There are some lessons in hindsight of > how we could have handled bytes/str, but it was more of a decision of > "really long transition versus a short one" -- jokes on us for what "short" > became ;) -- which we simply won't make ever again. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Thu Apr 26 19:16:57 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 27 Apr 2018 11:16:57 +1200 Subject: [Python-Dev] PEP 572: Why not := as standard assignment operator? In-Reply-To: References: Message-ID: <5AE25DE9.2010103@canterbury.ac.nz> Chris Angelico wrote: > No. Any expression may be used as a statement, so this isn't "outside > an expression". It could be detected as a special case and rejected some time later in the compilation process. The question is whether it's worth making the rules more complicated just to forbid something that some peple think is bad style. -- Greg From ben+python at benfinney.id.au Thu Apr 26 19:21:30 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 27 Apr 2018 09:21:30 +1000 Subject: [Python-Dev] PEP 394 update proposal: Allow changing the `python` command in some cases References: <4c4ec942-e17b-c4f4-4b2f-459fed2c32aa@gmail.com> Message-ID: <85wowttuyt.fsf@benfinney.id.au> Petr Viktorin writes: > In Fedora, I found that PEP 394's strict recommendation that `python` > points to `python2` is holding us back. I have read the message, but I don't see how you draw the link that PEP 394 is holding you back. > The problems are: > - For developers that are not following the language's development, > the fact that `python` invokes `python2` sends a strong signal that 2 > is somehow the preferred version, and it's OK to start new projects in > it. I agree with the statement you make later in the message: > [?] we feel that the only way to *enforce* that guidelines is to > provide environments where the `python` command does not work (unless > explicitly installed). Yes. The ?python? command is confusing, for the reasons you say. There should be ?python2? and ?python3? commands for Python 2 and Python 3 respectively, and no ?python? command should be installed by the operating system. The fact that ?/usr/bin/python? exists is an historical accident, and I agree with the proposal you state: the best way to correct the confusion is to bar the confusing command from being installed by packages. > - Users and sysadmins that *do* want to ?live in the future? are > switching the symlink to `python3` themselves. We would like to give > them a supported, documented way to do so -- and make surer they're > aware of the caveats. The supported, documented way to add a command pointing to a different command already exists, and there is no need to make a Python-specific special case. Users who want to make a ?python? alias can do so in their shell; this is supported and documented. Users who want to add a new command file can add a suitable directory (e.g. ?$HOME/bin?) to their ?PATH? variable, and put a symlink in there named ?python?. This is supported and documented. Sysadmins who want to create a system-wide command ?python? can put a symlink at ?/usr/local/bin/python?. This is supported and documented. I disagree with making some special-case extra way; that would be both cunfusing and superfluous. > - The `python` command is still not available out-of-the box on macOS, > so it didn't completely live up to the expectation of being the > cross-platform way to launch 2/3 source compatile scripts. That is one of the minor ways which macOS fails to conform to community-agreed conventions. We should not let that intransigence distort our discussion of best practices. > To help solve these, I would like to relax recommendations on the Unix > ``python -> python2`` symlink in these cases: For the above reasons, I disagree that PEP 394 is limiting what you want to do on free-software operating systems. For non-free operating systems, I don't think the already-discussed PEP 394 should be weakened if the operating system vendor fails to conform. > - Users and administrators can, by a deliberate action, change > ``python`` to invoke Python 3. Yes. That is well-known and long-standardised on Unix operating systems, and is much more broadly understood than any Python-specific special case would be. So I don't see how anyone is being held back. I trust that PEP 394 will not be weakened in its effect, and I wish you well with using the already-supported, already-documented, PEP-394 compatible means to add local customisations for a ?python? command. -- \ ?Pity the meek, for they shall inherit the earth.? ?Donald | `\ Robert Perry Marquis | _o__) | Ben Finney From greg.ewing at canterbury.ac.nz Thu Apr 26 20:00:50 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 27 Apr 2018 12:00:50 +1200 Subject: [Python-Dev] Is PEP 572 really the most effective way to solve the problems it's targeting? In-Reply-To: <39fe9f53-843b-3394-b7fc-c51a400a1409@mgmiller.net> References: <162ffcf2f98.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> <39fe9f53-843b-3394-b7fc-c51a400a1409@mgmiller.net> Message-ID: <5AE26832.2090303@canterbury.ac.nz> Mike Miller wrote: > - How are other modern languages solving this issue? In all the languages I can think of that allow assignments in expressions, there is only one assignment operator -- a stand alone assignment is just a bare assignment expression. But those languages were all designed that way from the start. I'm not aware of any that began by forbidding assignment in expressions and then added it later. -- Greg From larry at hastings.org Thu Apr 26 20:26:55 2018 From: larry at hastings.org (Larry Hastings) Date: Thu, 26 Apr 2018 17:26:55 -0700 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: <2167625f-7479-5c95-a51a-66c3787bf9bc@hastings.org> On 04/26/2018 12:12 PM, Tim Peters wrote: > [Larry Hastings ] >> I hate to be pedantic--there's enough of that going on in this thread--but I >> can't agree with the word "simplifed" above. I agree that the code using >> binding expressions is shorter. But considering that emit the two code >> examples implement the exact same algorithm, to the point where their >> bytecode would look nearly* identical, ISTM that the two code examples are >> of identical complexity. > In the absence of defining an objectively computable complexity > measure, I expect you're doomed to arguing taste. As are you!? I haven't seen any arguments that binding expressions allow us to express programs that were inexpressible in Python before.? I'm not even sure that binding expressions fall under the heading of "syntactic sugar", given their negligible semantics (and, imo, negligible benefit).? What else is left, on /both/ sides of the debate, if not a debate over aesthetics? > For example, argue > that both spellings have the same formal "cyclomatic complexity" > measure (which they do). By other formal measures (e.g., total number > of identifier instances), the latter spelling is "objectively > simpler". By yet others (e.g., total number of non-whitespace > characters divided by total number of lines), the former spelling is > "objectively simpler". What is this "objective simplicity" measurement you cite?? I understand that the code example cited had fewer identifiers, so when measuring "number of identifiers used" in isolation, the code example using binding expressions had fewer of them.? But this is so narrow as to be almost meaningless. Perhaps I'm misunderstanding you, but I read this as saying that there's a larger, well-established concept called "objective simplicity", of which this measurement is a part.? Can you tell me more about it?? Google was no help here. > But that all kinda misses the point to me: the latter spelling is > "obviously simpler" in a way that _actually matters_, for the same > reason, e.g., a case statement with N cases is "obviously simpler" > than the semantically equivalent spelling using N nested if/else > if/else if/else if/else ... blocks. As I already mentioned, the with-binding-expressions code expresses the same code, the same concept, and likely results in the same bytecode, as the without-binding-expressions code.? In contrast, a switch statement /is/ simpler than a series of nested if statements.? It's a different code construct, it has different (and comparatively restricted) semantics, and it results in simpler (and faster) code.? Whereas the with-binding-expressions code is equivalent to the without-binding-expressions code, semantically, bytecode-ly, etc.? So comparing the with-binding-expressions version of the code to the simplification afforded by a switch statement isn't an apples-to-apples comparison. In other words: you're really only arguing taste here.? You find it "obviously simpler", but this an aesthetic call on your part and not an objective measurement.? Me, my tastes are different--I find it "needlessly complicated" and prefer the without-binding-expressions version. > If it weren't for that you hate being pedantic, I'd add that you're > overlooking the piles of leading whitespace characters also saved in > the latter ;-) The number of those saved grows quadratically in the > number of uselessly indented blocks shifted left. Surely my dislike of being pedantic is irrelevant to your being pedantic.? It seems it's not something you mind doing! ;-) The nightmare scenario of quadratically right-shifted code is of course solvable in other ways.? Off the top of my head, one could rewrite the code example as follows, without any new syntax: def r(value): nonlocal reductor reductor = value return value if r(dispatch_table.get(cls)): rv = reductor(x) elif r(getattr(x, "__reduce_ex__", None)): rv = reductor(4) elif r(getattr(x, "__reduce__", None)): rv = reductor() else: raise Error("un(shallow)copyable object of type %s" % cls) It makes this particular code example longer--however it avoids the indenting you seem to dislike.? However: given that "r()" is shorter than "reductor := ", the more elifs you add, the more overall /characters/ you'll save!? With another three or four elifs this version would probably use fewer characters overall. I'm not seriously arguing that people rewrite their code in this way; I think it's less clear this way.? I suppose my point is, "we already have so many ways To Do It, I'm not in favor of adding syntax to the language to add one more way". >> and the density of complexity per line has shot up. > Average non-whitespace character count per line has certainly shot up, > but I don't actually know what you mean by "density of complexity" > there. What I meant was, stuff-accomplished-per-line.? I guess we could measure that objectively as bytecodes / linecount.? But I meant it more as a subjective measurement, considering factors like "how many unrelated operations occur on this line?" and "how many operations on this line have side-effects?".? I suggest that the with-binding-expressions version has more side effects and more unrelated operations per line, because several lines now fold assignment (side effect) into lines that were previously flow control. //arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Thu Apr 26 20:36:05 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 27 Apr 2018 10:36:05 +1000 Subject: [Python-Dev] Reserve ':=' for type-inferred variable initialization (was PEP 572) In-Reply-To: References: Message-ID: <20180427003605.GK7400@ando.pearwood.info> Hi Fatty, and welcome! On Thu, Apr 26, 2018 at 08:00:55PM +0200, Fatty Morgan wrote: > The natural interpretation of 'name := expr' is a PEP 526 > type-annotated variable initialization 'name : T = expr' with the > type annotation T omitted, the tokens ':' and '=' coalesced, and > the implied type T inferred as 'type(expr)'. I'm not sure why you say that is the "natural" interpretation, unless you're saying that Guido, Chris, myself and dozens of other people taking part of this conversation are unnatural, since none of us thought of that interpretation *smiles* The := token is the second most common assignment operator in programming languages, behind only = single equals sign. For those of us who were raised on Pascal, it is entirely natural to use = for equality tests and := for assignment, and languages that use == for equality are the ones which are weird. Since type-annotations are still only used by a small proportion of Python code and Python developers, I doubt that they will jump to the interpretation of "explicit type hint with no type given". If the type-checker can infer the type of the expression, there's no need to use the colon at all. name := expression # can infer type here name = expression # why not just infer the type here? So using : Type without the type is entirely unnecessary. The colon is only needed when you have to specify a type manually. Your comments about some entirely hypothetical "Python with enforced static typing" are interesting but so blue-sky that I honestly doubt that there's any point in discussing them now. We're focused on Python 3.8 and 3.9, not Python 5 or Python 6. -- Steve From tim.peters at gmail.com Thu Apr 26 20:46:02 2018 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 26 Apr 2018 19:46:02 -0500 Subject: [Python-Dev] Is PEP 572 really the most effective way to solve the problems it's targeting? In-Reply-To: <5AE26832.2090303@canterbury.ac.nz> References: <162ffcf2f98.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> <39fe9f53-843b-3394-b7fc-c51a400a1409@mgmiller.net> <5AE26832.2090303@canterbury.ac.nz> Message-ID: [Mike Miller] >> - How are other modern languages solving this issue? [Greg Ewing ] > In all the languages I can think of that allow assignments in > expressions, there is only one assignment operator -- a stand > alone assignment is just a bare assignment expression. Pretty much so, but I don't know what "modern" means to Mike. The R language may set a record for, umm, innovation here: """ There are three different assignment operators: two of them have leftwards and rightwards forms.[1] """ So there are 5 assignment operator spellings in R: = <- -> <<- ->> Note that the link doesn't tell the whole story either; e.g., they don't all have the same precedence level. And, in addition to the 5 infix spellings shown above, there are also prefix (looks like a 2-argument function call) spellings. Back on Earth ;-) , I think it's worth it to point out that only languages (with assignment expressions) aping C use "=" for assignment and "==" for equality. That was a Really Bad Idea that all other (not aping C) languages I know of avoided. But I'm not sure any of this is relevant to what Mike meant by "this issue". > But those languages were all designed that way from the start. > I'm not aware of any that began by forbidding assignment in > expressions and then added it later. Me neither. It's certainly the case that Guido would not have designed a language that aped C's poor decision here. At its very start, Python used "=" for both assignment and equality testing (and == was a syntax error). So I think it's evident that, at the time, he didn't envision ever adding assignment expressions. [1] https://www.rdocumentation.org/packages/base/versions/3.5.0/topics/assignOps From steve at pearwood.info Thu Apr 26 20:58:16 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 27 Apr 2018 10:58:16 +1000 Subject: [Python-Dev] Is PEP 572 really the most effective way to solve the problems it's targeting? In-Reply-To: <39fe9f53-843b-3394-b7fc-c51a400a1409@mgmiller.net> References: <162ffcf2f98.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> <39fe9f53-843b-3394-b7fc-c51a400a1409@mgmiller.net> Message-ID: <20180427005816.GL7400@ando.pearwood.info> On Thu, Apr 26, 2018 at 09:36:48AM -0700, Mike Miller wrote: > However, there were two good questions in this message which I haven't seen > addressed yet: > > - How are other modern languages solving this issue? > - How does this new construct intersect with typing functionality? What counts as a modern language? Less than five years old? Less than fifty years old? Are Javascript, Ruby and R modern? They all support assignment as expressions. I think Koitlin, Rust and Go prohibit assignment as expressions. Swift assignment evaluates as Void (equivalent to None in Python, I guess), so you can use assignment in an expression but it returns nothing and only operates by side-effect. As far as type hints go, I think that if you need explicit type hints in the middle of an expression, it's a bad idea and you ought to pull it out as a separate statement. That applies regardless of whether that expression involves binding or not. -- Steve From steve at pearwood.info Thu Apr 26 21:12:56 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 27 Apr 2018 11:12:56 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: Message-ID: <20180427011256.GM7400@ando.pearwood.info> On Thu, Apr 26, 2018 at 08:00:46PM +0100, Zero Piraeus wrote: > Since it now looks like it really *does* have a decent chance, and > maybe another -1 has a small chance of tipping the balance: my > reaction to the proposal is also emotional. Visceral, in fact, to the > extent that I'd aim to read and write less Python if it became > commonplace. Funnily enough, that's what some people said about decorator syntax, ternary if, type annotations and list comprehensions. All of them have become great additions to the language. I hated the idea of aping C and adding += operators and swore I'd never use them. That lasted, well, about a month. Just sayin'. -- Steve From mikhailwas at gmail.com Thu Apr 26 21:33:43 2018 From: mikhailwas at gmail.com (Mikhail V) Date: Fri, 27 Apr 2018 04:33:43 +0300 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: References: Message-ID: On Thu, Apr 26, 2018 at 4:27 PM, Kirill Balunov wrote: > > > 2018-04-24 18:31 GMT+03:00 Chris Angelico : >> >> >> > > Not sure, but if additional motivating examples are required, there is a > common pattern for dynamic attribute lookup (snippet from `copy.py`): > > reductor = dispatch_table.get(cls) > if reductor: > rv = reductor(x) > else: > reductor = getattr(x, "__reduce_ex__", None) > if reductor: > rv = reductor(4) > else: > reductor = getattr(x, "__reduce__", None) > if reductor: > rv = reductor() > else: > raise Error("un(shallow)copyable object of type %s" % cls) > > which can with the current `binding expression` syntax simplified to: > > if reductor := dispatch_table.get(cls): > rv = reductor(x) > elif reductor := getattr(x, "__reduce_ex__", None): > rv = reductor(4) > elif reductor := getattr(x, "__reduce__", None): > rv = reductor() > else: > raise Error("un(shallow)copyable object of type %s" % cls) > To me this looks more like a motivating example for adding GOTO statement to Python... Anybody writing a PEP? :) And should not the above be wrapped in a function? (a workaround in abscence of GOTO) Also, the ELIF is merely an edge case: once you have something like: if value : rv = f(x) else : temp = ... value = f1(temp) print (value) if value : rv = f2(x) ... then it does not wrap into anything anymore. Mikhail From paul at ganssle.io Thu Apr 26 21:23:47 2018 From: paul at ganssle.io (Paul G) Date: Fri, 27 Apr 2018 01:23:47 +0000 Subject: [Python-Dev] Is PEP 572 really the most effective way to solve the problems it's targeting? In-Reply-To: <20180427005816.GL7400@ando.pearwood.info> References: <162ffcf2f98.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> <39fe9f53-843b-3394-b7fc-c51a400a1409@mgmiller.net> <20180427005816.GL7400@ando.pearwood.info> Message-ID: <76D52ACB-0D41-4880-8835-E6A02C065557@ganssle.io> Rust has a few syntactic ways to accomplish the same thing, though. I think match expressions are used for the equivalent of conditionals that carry the condition value into the body of the expression, and all blocks return the result of the last statement, so you can do things like: let mut x; while { x = foo(); x } { bar(x); } I don't know if that's idiomatic Rust (and I wrote it on a phone and didn't check to be sure it compiles), but it does more or less solve the problem of assignment in a control flow condition. On April 27, 2018 12:58:16 AM UTC, Steven D'Aprano wrote: >On Thu, Apr 26, 2018 at 09:36:48AM -0700, Mike Miller wrote: > >> However, there were two good questions in this message which I >haven't seen >> addressed yet: >> >> - How are other modern languages solving this issue? >> - How does this new construct intersect with typing >functionality? > >What counts as a modern language? Less than five years old? Less than >fifty years old? Are Javascript, Ruby and R modern? They all support >assignment as expressions. > >I think Koitlin, Rust and Go prohibit assignment as expressions. > >Swift assignment evaluates as Void (equivalent to None in Python, I >guess), so you can use assignment in an expression but it returns >nothing and only operates by side-effect. > >As far as type hints go, I think that if you need explicit type hints >in >the middle of an expression, it's a bad idea and you ought to pull it >out as a separate statement. That applies regardless of whether that >expression involves binding or not. > > >-- >Steve >_______________________________________________ >Python-Dev mailing list >Python-Dev at python.org >https://mail.python.org/mailman/listinfo/python-dev >Unsubscribe: >https://mail.python.org/mailman/options/python-dev/paul%40ganssle.io -------------- next part -------------- An HTML attachment was scrubbed... URL: From songofacandy at gmail.com Thu Apr 26 22:15:20 2018 From: songofacandy at gmail.com (INADA Naoki) Date: Fri, 27 Apr 2018 02:15:20 +0000 Subject: [Python-Dev] Is PEP 572 really the most effective way to solve the problems it's targeting? In-Reply-To: <76D52ACB-0D41-4880-8835-E6A02C065557@ganssle.io> References: <162ffcf2f98.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> <39fe9f53-843b-3394-b7fc-c51a400a1409@mgmiller.net> <20180427005816.GL7400@ando.pearwood.info> <76D52ACB-0D41-4880-8835-E6A02C065557@ganssle.io> Message-ID: On Fri, Apr 27, 2018 at 10:52 AM Paul G wrote: > Rust has a few syntactic ways to accomplish the same thing, though. I think match expressions are used for the equivalent of conditionals that carry the condition value into the body of the expression, and all blocks return the result of the last statement, so you can do things like: > let mut x; > while { x = foo(); x } { > bar(x); > } Go is similar to Python; it's doesn't allow assignment in expression. And Go has similar syntax like above; for x := foo(); x { bar(x) } if err := baz(); err != nil { return err } I like Go and I think this syntax can be ported to Python. But it help only if/while statements. It doesn't help list comprehension. And Go doesn't have list comprehension. From tim.peters at gmail.com Thu Apr 26 22:49:21 2018 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 26 Apr 2018 21:49:21 -0500 Subject: [Python-Dev] The new and improved PEP 572, same great taste with 75% less complexity! In-Reply-To: <2167625f-7479-5c95-a51a-66c3787bf9bc@hastings.org> References: <2167625f-7479-5c95-a51a-66c3787bf9bc@hastings.org> Message-ID: [Larry Hastings ] >>> I hate to be pedantic--there's enough of that going on in this thread--but I >>> can't agree with the word "simplifed" above. I agree that the code using >>> binding expressions is shorter. But considering that emit the two code >>> examples implement the exact same algorithm, to the point where their >>> bytecode would look nearly* identical, ISTM that the two code examples are >>> of identical complexity. [Tim] >> In the absence of defining an objectively computable complexity >> measure, I expect you're doomed to arguing taste. [Larry] > As are you! I didn't claim otherwise. > I haven't seen any arguments that binding expressions allow us > to express programs that were inexpressible in Python before. They don't. > I'm not even sure that binding expressions fall under the heading > of "syntactic sugar", given their negligible semantics (and, imo, > negligible benefit). What else is left, on both sides of the debate, > if not a debate over aesthetics? I prefer to look at effects on real code. Other people prefer to philosophize. >> For example, argue that both spellings have the same formal >> "cyclomatic complexity" measure (which they do). By other formal >> measures (e.g., total number of identifier instances), the latter >> spelling is "objectively simpler". By yet others (e.g., total number >> of non-whitespace characters divided by total number of lines), the >> former spelling is "objectively simpler". > What is this "objective simplicity" measurement you cite? There are many ways you can (and various programs do) attempt to define, quantitatively, what "program complexity" means. Under any such objectively defined measure, two pieces of code can be "objectively compared". I use scare quotes with their ordinary meaning: that it's "objective" only if you're silly enough to believe that _whatever_ numbers you're computing are going to settle the issue ;-) > I understand that the code example cited had fewer identifiers, so when > measuring "number of identifiers used" in isolation, the code example using > binding expressions had fewer of them. Then you necessarily agree that _if_ our objective definition of complexity is "total number of identifier instances", the binding-expression version is "objectively simpler". It's been reduced, by definition, to a question of determining which of two integers is smaller. > But this is so narrow as to be almost meaningless. Of course! As is your original claim that "the two code examples are of identical complexity". "because" "their bytecode would look nearly identical". Well, sure, _if_ that's how we define program complexity, the conclusion follows. But there's no reason I can see to accept that definition to begin with either. I suspect _you_ like it primarily because you found it supported the conclusion you had already reached ;-) > Perhaps I'm misunderstanding you, but I read this as saying that there's a > larger, well-established concept called "objective simplicity", of which > this measurement is a part. Can you tell me more about it? Google was no > help here. The metrics I mentioned are used by a number of programs that claim to quantify program complexity. For example, among many other things, this program computes cyclomatic complexity, and uses N_2 for "total number of operands" (which I called "identifiers" instead to specialize it to the specific example) under the widely used "Halstead Metrics": http://radon.readthedocs.io/en/latest/intro.html My favorite part is where the numerator of the "Maintainability Index" adds in 50 * sin(sqrt(2.4 * C)) where "C is the percent of comment lines (important: converted to radians)". WTF?! ;-) But they're not joking: some people take this stuff very seriously. >> But that all kinda misses the point to me: the latter spelling is >> "obviously simpler" in a way that _actually matters_, for the same >> reason, e.g., a case statement with N cases is "obviously simpler" >> than the semantically equivalent spelling using N nested if/else >> if/else if/else if/else ... blocks. > As I already mentioned, the with-binding-expressions code expresses the same > code, the same concept, and likely results in the same bytecode, as the > without-binding-expressions code. And as I already explained in some detail, while I agree with (almost) all that, it leaves me cold as a dead fish. The test-action pairs in the code are _semantically_ peers, not a nesting of subordinates. It's _clearer_ to human eyes if the syntactic structure of the code reflects the peer relationship directly. I couldn't care less that the byte code turns out being nearly the same. I'm not a PVM - I need to _reason_ about the code I read. In failing to visually reflect the peer relationship, the original code obscures a key simplicity. > In contrast, a switch statement is simpler than a series of nested if > statements. It's a different code construct, it has different (and > comparatively restricted) semantics, and it results in simpler (and faster) code. That all depends on the language you're using. If I had intended C, I would have said "switch statement" instead - but I certainly could have been clearer about that. For example, in the Icon language, its "case" control structure supports arbitrary expressions as selectors, including what Python calls generators. It in fact compiles to "Icon code" identical to what you'd get from tediously writing out a pile of nested if/else if/else if/else blocks instead. Its case selectors can work with any kinds of Icon objects (not just "little integers"), and invoke arbitrarily complex code to generate the values to test against. Nevertheless, every Icon programmer in existence would agree it's _clearer_ to write a long string of "I want to execute the first block of code that matches one of these test conditions, and no other blocks of code associated with other test conditions" as a "case" than as a tedious mountain of nested if/else blocks Surely you understand that? When "exactly one of these conditions" is of primary interest, a case structure says that _directly_ the instant you see the word "case". This has nothing to do with generated code, or efficiency. In Python, it's not _as_ obvious, but follows immediately if you see that the first words on the control structure lines are "if/elif/elif/../elif/else" indented at the same level. It doesn't matter to that what the tests or, or what the "action blocks" do. If it's a mass of `if` and `else` statements indented all over the place, it requires actual work to deduce it. Which the Python compiler does - but which people "shouldn't" need to. > Whereas the with-binding-expressions code is equivalent to the > without-binding-expressions code, semantically, bytecode-ly, etc. > So comparing the with-binding-expressions version of the > code to the simplification afforded by a switch statement isn't an > apples-to-apples comparison. Except you snipped the actual comparison I made: The latter spelling above is indeed visually very much like a case statement: all the tests are at the same indentation level, and all the conditional actions are too. It's obvious _at a glance_ in the latter that exactly one of the action blocks will be performed. That wasn't at all about final semantics, generated code, etc - it was about the visual appearance of the code, or as explained in even more tedious detail ;-) above, about the visual structure _highlighting_ rather than _obscuring_ the peer relationship among the test-action pairs. > In other words: you're really only arguing taste here. You find it > "obviously simpler", but this an aesthetic call on your part and not an > objective measurement. Me, my tastes are different--I find it "needlessly > complicated" and prefer the without-binding-expressions version. I have no objection to ending this by agreeing my taste is excellent ;-) And I make no pretensions at all to being able to quantify the extent to which the visual appearance of code reflects key features of the code's semantics. Which is the only part _I_ care much about here. In particular, the generated bytecode has no bearing on what I care about. Think about it: in what possible world _could_ the generated bytecode have any bearing on the readability of Python source code? For example, def f(): return 1 and def f2(): return 1 - 2 + 3 - 4 + 5 - 2 compile to nearly identical bytecode too (the only difference is the offset on the LOAD_CONST loading "1"), but I bet you'd agree the first "is simpler". Same thing to me in the example: you have to turn me into "a compiler" in the original example to _deduce_ the key aspects of the code that are obvious by inspection in the rewritten version. > ... > Surely my dislike of being pedantic is irrelevant to your being pedantic. > It seems it's not something you mind doing! ;-) Nope, not a bit! > The nightmare scenario of quadratically right-shifted code is of course > solvable in other ways. Of course it can. But as Guido pointed out at the start, and as Kirill illustrated with verbatim code from Python's standard library, binding expressions allow solving it with minimal, easy edits to the code exactly as he found it. The hardest part is getting rid of all the semantically misleading indentation. And it seems all but certain that had binding expressions been in the language at the time that code had been written, it would have been written in the visually transparent way from the start. > ... > I'm not seriously arguing that people rewrite their code in this way; I > think it's less clear this way. Which is why I snipped it :-) > ... From tim.peters at gmail.com Thu Apr 26 23:11:59 2018 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 26 Apr 2018 22:11:59 -0500 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <20180427011256.GM7400@ando.pearwood.info> References: <20180427011256.GM7400@ando.pearwood.info> Message-ID: [Zero Piraeus] >> Since it now looks like it really *does* have a decent chance, and >> maybe another -1 has a small chance of tipping the balance: my >> reaction to the proposal is also emotional. Visceral, in fact, to the >> extent that I'd aim to read and write less Python if it became >> commonplace. [Steven D'Aprano ] > Funnily enough, that's what some people said about decorator syntax, > ternary if, type annotations and list comprehensions. > > All of them have become great additions to the language. > > I hated the idea of aping C and adding += operators and swore I'd never > use them. That lasted, well, about a month. > > Just sayin'. Well - I've come to respect your opinion, so ... OK, I'll give += a try. Frankly, I've grown tired of editing it out of all the packages I download anyway ;-) From tritium-list at sdamon.com Fri Apr 27 00:02:14 2018 From: tritium-list at sdamon.com (Alex Walters) Date: Fri, 27 Apr 2018 00:02:14 -0400 Subject: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 In-Reply-To: References: Message-ID: <1088c501d3dddc$875785c0$96069140$@sdamon.com> http://pyvideo.org/pycascades-2018/bdfl-python-3-retrospective.html link to Guido?s talk, for your convenience From: Python-Dev On Behalf Of Guido van Rossum Sent: Thursday, April 26, 2018 6:12 PM To: Brett Cannon Cc: Barry Warsaw ; Python-Dev Subject: Re: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 Also see my talk at PyCascades and Victor's upcoming talk at PyCon. On Thu, Apr 26, 2018, 12:02 Brett Cannon > wrote: On Thu, 26 Apr 2018 at 10:19 Barry Warsaw > wrote: On Apr 26, 2018, at 09:28, Eric Snow > wrote: > > On Thu, Apr 26, 2018 at 10:25 AM, Eric Snow > wrote: >> In pondering our approach to future Python major releases, I found >> myself considering the experience we've had with Python 3. The whole >> Py3k effort predates my involvement in the community so I missed a >> bunch of context about the motivations, decisions, and challenges. >> While I've pieced some of that together over the years now since I've >> been around, I've certainly seen much of the aftermath. For me, at >> least, it would be helpful to have a bit more insight into the >> history. :) It would certainly be an interesting document, but I suspect you?ll get a bit of the old ?ask 3 lawyers and get 5 opinions? kind of response. ;) As I remember it, there was definitely a feeling like, this would be our only chance to clean up some annoying cruft, and rectify some (in hindsight) incorrect design decisions made over the years, couple with a healthy dose of ?we have no idea how to do the bytes/str split in a backward compatible way". There was probably a sense that the Python community was just small enough to be able to handle such a disruptive change, but wouldn?t ever be so again. The latter is definitely true today, even if the former was overly optimistic. I agree with everything Barry said. There are some lessons in hindsight of how we could have handled bytes/str, but it was more of a decision of "really long transition versus a short one" -- jokes on us for what "short" became ;) -- which we simply won't make ever again. _______________________________________________ Python-Dev mailing list Python-Dev at python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From python-dev at mgmiller.net Fri Apr 27 01:14:48 2018 From: python-dev at mgmiller.net (Mike Miller) Date: Thu, 26 Apr 2018 22:14:48 -0700 Subject: [Python-Dev] Is PEP 572 really the most effective way to solve the problems it's targeting? In-Reply-To: <20180427005816.GL7400@ando.pearwood.info> References: <162ffcf2f98.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> <39fe9f53-843b-3394-b7fc-c51a400a1409@mgmiller.net> <20180427005816.GL7400@ando.pearwood.info> Message-ID: Sorry all, wasn't specific enough. By "modern" I mean the last decade perhaps. New languages that have had a chance to look at the older generations and choose their best ideas, while leaving behind the rest. Personally I thought of Swift (Ryan mentioned), Kotlin, Rust, and perhaps Go, though Go wasn't focused on breaking new ground outside of ease of concurrency. I don't know R or Felix at all, but sound interesting. Nim is another I'm vaguely aware of. They surely have given some thought to the issue. One thing that jumped out at me is that most replies here jumped to the question of whether they supported assignment-expressions, but that is only one potential solution. To be more clear, I wondered how did they solve "the problem itself?" Was their solution different? Ryan somewhat alluded to that, but I'd like to dig in a bit on that part. In contrast, in many of the other threads I heard, "C, C++, C#, Java, etc do assignment-expressions, they're useful and not so hard to learn." Ok that's reasonable, but where is the industry headed? Python deferred long enough that we don't necessarily have to choose a classic solution. So, it sounds like many of the new generation of languages are not embracing these expressions everywhere but rather letting folks do an assignment right in the statement where their use case applies, if, while, maybe comprehensions. Is that accurate? Looks like I've got some homework to do, haha. -Mike On 2018-04-26 17:58, Steven D'Aprano wrote: > What counts as a modern language? Less than five years old? Less than > fifty years old? Are Javascript, Ruby and R modern? They all support > assignment as expressions. > > I think Koitlin, Rust and Go prohibit assignment as expressions. > > Swift assignment evaluates as Void (equivalent to None in Python, I > guess), so you can use assignment in an expression but it returns > nothing and only operates by side-effect. > > As far as type hints go, I think that if you need explicit type hints in > the middle of an expression, it's a bad idea and you ought to pull it > out as a separate statement. That applies regardless of whether that > expression involves binding or not. From steve at pearwood.info Fri Apr 27 01:38:37 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 27 Apr 2018 15:38:37 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <20180426061000.GF7400@ando.pearwood.info> <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> Message-ID: <20180427053837.GN7400@ando.pearwood.info> On Thu, Apr 26, 2018 at 08:48:12AM -0700, ?ukasz Langa wrote: > > > On Apr 25, 2018, at 11:10 PM, Steven D'Aprano wrote: > > Criticising binding- > > expressions for that reason, especially implying that we must always use > > parens, is simply FUD. > > The PEP has more examples with parentheses than without. Yes? Parens aren't mandatory, and my point that other operators also sometimes needs parens still holds. > >> As soon as we have to wrap a part of an expression in parentheses, > >> parsing the entire thing becomes more complex. > > > > Unless it becomes less complex to read and understand. > > You're ignoring the context of the discussion. The new parentheses are > there because there's a new assignment there. That's more complex. I'm not ignoring the context of the discussion. I'm comparing binding- expression with and without parens. That's what I thought you were doing. If that wasn't your intended meaning, then I apologise but please understand why I answered the way I did. I still stand by my argument: parens are not always needed, and even when they are not needed, adding them can sometimes make things *easier* and *less complex* to read. [...] > If you think demonstrating cases where the end result won't be an > improvement is picking at straws, then maybe the improvement of PEP > 572 is as well. Any feature can have cases where the end result is worse than not using the feature. That *alone* isn't a strong argument against a feature. Do you have much existing code using binding expressions? Of course not. Will you be adding them to code that already exists? Probably not -- you can't do so until you are using 3.8 at minimum, and if your code needs to be backwards compatible, you can't use it until you've dropped support for 3.7 and older. That might not be for five or ten years. So it is likely that for most people only new code will use this feature. It is not reasonable to say that if I have existing code like this: spam = expression if long_condition_that_takes_up_most_of_the_line == spam or spam: ... that I'm going to immediately change it to a one-liner: if long_condition_that_takes_up_most_of_the_line == (spam := expression) or spam: ... and push it over the maximum line width. With or without parentheses. Why would I do something so silly? Using binding expressions isn't mandatory and most coders don't intentionally do things that make their code worse. And if I wouldn't change existing code and push it over the limit, why would I write new code that does it? Especially when there are much better alternatives: if (long_condition_that_takes_up_most_of_the_line == (spam:=expression) or spam): ... We have a history of adding features that can be abused, but aren't. People hardly ever abuse list comps with overly long and complex multiple-loop comprehensions: [... for a in sequence for b in something for c in another for d in something_else] I'm sure we've all seen one or two of those. But it doesn't happen enough to matter. Same with if...elif...else chains. People didn't immediately run out and replace every single if chain into nested ternary if expressions, pushing their lines out to beyond the maximum line width: expression if condition else (expression if condition else (expression if condition else (expression if condition else expression))) Real people don't abuse comprehensions or ternary if enough for us to regret adding them to the language. I'm sure that they won't abuse this feature either. The Python community simply doesn't have the culture of abusing syntax in this way and writing overly dense one-liners, and I don't think it is reasonable to say this feature will tip the balance. It is reasonable to say that *some* code will be made worse by this, because there's always *someone* who will abuse syntax. There are those who insist on writing list comprehensions for their side-effects: # return and throw away a list of Nones [print(item) for item in bunch_of_stuff] but I don't think this happens enough to make us regret adding comprehensions to the language. -- Steve From ben+python at benfinney.id.au Fri Apr 27 02:03:17 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 27 Apr 2018 16:03:17 +1000 Subject: [Python-Dev] PEP 394 update proposal: Allow changing the `python` command in some cases References: <4c4ec942-e17b-c4f4-4b2f-459fed2c32aa@gmail.com> <85wowttuyt.fsf@benfinney.id.au> Message-ID: <85muxpp4nu.fsf@benfinney.id.au> Ben Finney writes: > Petr Viktorin writes: > > > [?] we feel that the only way to *enforce* that guidelines is to > > provide environments where the `python` command does not work > > (unless explicitly installed). > > Yes. The ?python? command is confusing, for the reasons you say. There > should be ?python2? and ?python3? commands for Python 2 and Python 3 > respectively, and no ?python? command should be installed by the > operating system. > > The fact that ?/usr/bin/python? exists is an historical accident, and I > agree with the proposal you state: the best way to correct the confusion > is to bar the confusing command from being installed by packages. Because the above is ambiguous, I'll clarify: I am not calling for, and PEP 394 does not call for, the banishment of the ?python? command. What I'm saying is that muddying the rules further on what ?python? may or may not mean is *worse than* banishing the ?python? command entirely. So, short of banishing ?python? entirely, I think PEP 394 is already a good clear way to address the issue. Existing, documented and supported means to locally modify a ?python? command already exist and should be sufficient. > I trust that PEP 394 will not be weakened in its effect, and I wish you > well with using the already-supported, already-documented, PEP-394 > compatible means to add local customisations for a ?python? command. -- \ ?Try to learn something about everything and everything about | `\ something.? ?Thomas Henry Huxley | _o__) | Ben Finney From ethan at stoneleaf.us Fri Apr 27 02:15:01 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 26 Apr 2018 23:15:01 -0700 Subject: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 In-Reply-To: <1088c501d3dddc$875785c0$96069140$@sdamon.com> References: <1088c501d3dddc$875785c0$96069140$@sdamon.com> Message-ID: <5AE2BFE5.4000600@stoneleaf.us> On 04/26/2018 09:02 PM, Alex Walters wrote: > http://pyvideo.org/pycascades-2018/bdfl-python-3-retrospective.html link to Guido?s talk, for your convenience Many thanks! -- ~Ethan~ From agriff at tin.it Fri Apr 27 02:13:20 2018 From: agriff at tin.it (Andrea Griffini) Date: Fri, 27 Apr 2018 08:13:20 +0200 Subject: [Python-Dev] Reserve ':=' for type-inferred variable initialization (was PEP 572) In-Reply-To: <20180427003605.GK7400@ando.pearwood.info> References: <20180427003605.GK7400@ando.pearwood.info> Message-ID: >> We're focused on Python 3.8 and 3.9, not Python 5 or Python 6. Hmmm... When I was hearing the repeated belated saying that Python will never ever jump on the statically typed ship on each and every static type annotation discussion I started to worry this wasn't indeed the case (why the urge of repeating it so much otherwise?). Now we got standard library features requiring type annotation and a little shift towards a "not now" position. I'm just wondering... I'm NOT saying this would be bad (or good). On Fri, Apr 27, 2018 at 2:36 AM, Steven D'Aprano wrote: > Hi Fatty, and welcome! > > On Thu, Apr 26, 2018 at 08:00:55PM +0200, Fatty Morgan wrote: > > > The natural interpretation of 'name := expr' is a PEP 526 > > type-annotated variable initialization 'name : T = expr' with the > > type annotation T omitted, the tokens ':' and '=' coalesced, and > > the implied type T inferred as 'type(expr)'. > > I'm not sure why you say that is the "natural" interpretation, > unless you're saying that Guido, Chris, myself and dozens of other > people taking part of this conversation are unnatural, since none of us > thought of that interpretation *smiles* > > The := token is the second most common assignment operator in > programming languages, behind only = single equals sign. For those of us > who were raised on Pascal, it is entirely natural to use = for equality > tests and := for assignment, and languages that use == for equality are > the ones which are weird. > > Since type-annotations are still only used by a small proportion of > Python code and Python developers, I doubt that they will jump to the > interpretation of "explicit type hint with no type given". > > If the type-checker can infer the type of the expression, there's no > need to use the colon at all. > > name := expression # can infer type here > name = expression # why not just infer the type here? > > So using : Type without the type is entirely unnecessary. The colon is > only needed when you have to specify a type manually. > > Your comments about some entirely hypothetical "Python with enforced > static typing" are interesting but so blue-sky that I honestly doubt > that there's any point in discussing them now. We're focused on Python > 3.8 and 3.9, not Python 5 or Python 6. > > > -- > Steve > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > agriff%40tin.it > -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Fri Apr 27 03:38:22 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 27 Apr 2018 19:38:22 +1200 Subject: [Python-Dev] Every Release Can Be a Mini "Python 4000", Within Reason (was (name := expression) doesn't fit the narrative of PEP 20) In-Reply-To: References: Message-ID: <5AE2D36E.7060801@canterbury.ac.nz> Eric Snow wrote: > Unless I've missed something, there's no prohibition against > deprecating things (and then later removing them) or other breaks in > backward compatibility. I suppose that's true, but even so, changing "=" is going to feel like a really big change in the style of the language, bigger even than making "print" no longer a statement. It seems like there should be more justification for it than "well, it became redundant with :=". How would you complete the following sentence? "The ':=' symbol is a much better symbol for assignment than '=', because..." -- Greg From greg.ewing at canterbury.ac.nz Fri Apr 27 03:38:33 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 27 Apr 2018 19:38:33 +1200 Subject: [Python-Dev] Reserve ':=' for type-inferred variable initialization (was PEP 572) In-Reply-To: References: Message-ID: <5AE2D379.6050408@canterbury.ac.nz> Fatty Morgan wrote: > Distinguishing in a rather unobtrusive way (no 'var' or 'let') > the initialization of a previously unbound variable 'name := expr' > and the re-assignment of a previously bound variable 'name = expr' That would be a massively confusing change. It's a fine idea for a new language, but I can't see a smooth way to get there from existing Python. -- Greg From chris.jerdonek at gmail.com Fri Apr 27 05:46:27 2018 From: chris.jerdonek at gmail.com (Chris Jerdonek) Date: Fri, 27 Apr 2018 02:46:27 -0700 Subject: [Python-Dev] Order of positional and keyword arguments In-Reply-To: References: Message-ID: On Thu, Apr 26, 2018 at 12:25 PM, Serhiy Storchaka wrote: > f(b=2, *[1]) is surprised in two ways: > > 1. Argument values are passed not in order. The first value is assigned to > the second parameter, and the second value is assigned to the first > parameter. > > 2. Argument values are evaluated not from left to right. This contradicts > the general rule that expressions are evaluated from left to right (with few > known exceptions). > > I never seen the form `f(b=2, *[1])` in practice (though the language > reference contains an explicit example for it), and it looks weird to me. I > don't see reasons of writing `f(b=2, *[1])` instead of more natural `f(*[1], > b=2)`. I propose to disallow it. Coincidentally, I recently came across and reviewed a PR to Django that proposed exactly this, or at least something very similar. They proposed changing-- def create_cursor(self, name=None): to-- def create_cursor(self, name=None, *args, **kwargs): https://github.com/django/django/pull/9674/files#diff-53fcf3ac0535307033e0cfabb85c5301R173 --Chris > > This will also make the grammar simpler. Current grammar: > > argument_list: `positional_arguments` ["," `starred_and_keywords`] > : ["," `keywords_arguments`] > : | `starred_and_keywords` ["," `keywords_arguments`] > : | `keywords_arguments` > positional_arguments: ["*"] `expression` ("," ["*"] `expression`)* > starred_and_keywords: ("*" `expression` | `keyword_item`) > : ("," "*" `expression` | "," `keyword_item`)* > keywords_arguments: (`keyword_item` | "**" `expression`) > : ("," `keyword_item` | "," "**" `expression`)* > keyword_item: `identifier` "=" `expression` > > Proposed grammar: > > argument_list: `positional_arguments` ["," `keywords_arguments`] > : | `keywords_arguments` > positional_arguments: ["*"] `expression` ("," ["*"] `expression`)* > keywords_arguments: `keyword_argument` ("," `keyword_argument`)* > keyword_argument: `identifier` "=" `expression` | "**" `expression` > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/chris.jerdonek%40gmail.com From solipsis at pitrou.net Fri Apr 27 05:59:38 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 27 Apr 2018 11:59:38 +0200 Subject: [Python-Dev] Order of positional and keyword arguments References: Message-ID: <20180427115938.7b6bca45@fsol> On Thu, 26 Apr 2018 22:25:09 +0300 Serhiy Storchaka wrote: > > f(b=2, *[1]) is surprised in two ways: > > 1. Argument values are passed not in order. The first value is assigned > to the second parameter, and the second value is assigned to the first > parameter. I don't find it that surprising. If you write f(b=2, a=1), you are also passing arguments "not in order", but it still looks ok to me. > 2. Argument values are evaluated not from left to right. This > contradicts the general rule that expressions are evaluated from left to > right (with few known exceptions). Well... If you have code that relies on that rule, I would agree it's brittle code and should be rewritten differently (perhaps by assigning to temporary variables explicitly). Regards Antoine. From wes.turner at gmail.com Fri Apr 27 06:18:36 2018 From: wes.turner at gmail.com (Wes Turner) Date: Fri, 27 Apr 2018 06:18:36 -0400 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <20180427053837.GN7400@ando.pearwood.info> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <20180426061000.GF7400@ando.pearwood.info> <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> <20180427053837.GN7400@ando.pearwood.info> Message-ID: So, the style guidelines for this new feature -- and also ternary expressions and comprehension -- would need to mention that: - debuggers have no idea what to do with all of this on one line - left-to-right doesn't apply to comprehensions results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0] - left-to-right doesn't apply to ternary expressions if (y := func(x)) if (x := 3) else 0: while (y := func(x)) if (x := 3) else 0: - left-to-right does apply to everything else? - *these* are discouraged: if (x := 3) or (y := func(x)): if (3) or (func(3)): if ((x := 3) if 1 else (y := func(x))): IDK, I could just be resistant to change, but this seems like something that will decrease readability -- and slow down code review -- without any real performance gain. So, while this would be useful for golfed-down (!) one-liners with pyline, I'm -1 on PEP 572. How do I step through this simple example with a debugger? if re.search(pat, text) as match: print("Found:", match.group(0)) How do I explain what ':=' is when teaching Python? AFAIU, semantically: Python = ('equals') indicates a statement. What you are proposing is adding an ':=' ('colon equals') assignment operator which defines a variable which is limited in scope only in list, dict, and generator comprehensions. >From https://en.wikipedia.org/wiki/Assignment_(computer_science) : > In some languages the symbol used is regarded as an operator (meaning that the assignment has a value) while others define the assignment as a statement (meaning that it cannot be used in an expression). PEP 572 -- Assignment Expressions PEP 572 -- Assignment Operator (:=) and Assignment Expressions On Friday, April 27, 2018, Steven D'Aprano wrote: > On Thu, Apr 26, 2018 at 08:48:12AM -0700, ?ukasz Langa wrote: > > > > > On Apr 25, 2018, at 11:10 PM, Steven D'Aprano > wrote: > > > Criticising binding- > > > expressions for that reason, especially implying that we must always > use > > > parens, is simply FUD. > > > > The PEP has more examples with parentheses than without. > > Yes? Parens aren't mandatory, and my point that other operators also > sometimes needs parens still holds. > > > > >> As soon as we have to wrap a part of an expression in parentheses, > > >> parsing the entire thing becomes more complex. > > > > > > Unless it becomes less complex to read and understand. > > > > You're ignoring the context of the discussion. The new parentheses are > > there because there's a new assignment there. That's more complex. > > I'm not ignoring the context of the discussion. I'm comparing binding- > expression with and without parens. That's what I thought you were > doing. > > If that wasn't your intended meaning, then I apologise but please > understand why I answered the way I did. > > I still stand by my argument: parens are not always needed, and even > when they are not needed, adding them can sometimes make things > *easier* and *less complex* to read. > > > [...] > > If you think demonstrating cases where the end result won't be an > > improvement is picking at straws, then maybe the improvement of PEP > > 572 is as well. > > Any feature can have cases where the end result is worse than not using > the feature. That *alone* isn't a strong argument against a feature. > > Do you have much existing code using binding expressions? Of course not. > Will you be adding them to code that already exists? Probably not -- you > can't do so until you are using 3.8 at minimum, and if your code needs > to be backwards compatible, you can't use it until you've dropped > support for 3.7 and older. That might not be for five or ten years. > > So it is likely that for most people only new code will use this > feature. It is not reasonable to say that if I have existing code like > this: > > spam = expression > if long_condition_that_takes_up_most_of_the_line == spam or spam: > ... > > that I'm going to immediately change it to a one-liner: > > if long_condition_that_takes_up_most_of_the_line == (spam := > expression) or spam: > ... > > and push it over the maximum line width. With or without parentheses. > Why would I do something so silly? Using binding expressions isn't > mandatory and most coders don't intentionally do things that make their > code worse. > > And if I wouldn't change existing code and push it over the limit, why > would I write new code that does it? Especially when there are much > better alternatives: > > if (long_condition_that_takes_up_most_of_the_line > == (spam:=expression) > or spam): > ... > > > We have a history of adding features that can be abused, but aren't. > People hardly ever abuse list comps with overly long and complex > multiple-loop comprehensions: > > [... for a in sequence for b in something for c in another for d in > something_else] > > I'm sure we've all seen one or two of those. But it doesn't happen > enough to matter. Same with if...elif...else chains. People didn't > immediately run out and replace every single if chain into nested > ternary if expressions, pushing their lines out to beyond the maximum > line width: > > expression if condition else (expression if condition else (expression > if condition else (expression if condition else expression))) > > Real people don't abuse comprehensions or ternary if enough for us to > regret adding them to the language. I'm sure that they won't abuse this > feature either. The Python community simply doesn't have the culture of > abusing syntax in this way and writing overly dense one-liners, and I > don't think it is reasonable to say this feature will tip the balance. > > It is reasonable to say that *some* code will be made worse by this, > because there's always *someone* who will abuse syntax. There are those > who insist on writing list comprehensions for their side-effects: > > # return and throw away a list of Nones > [print(item) for item in bunch_of_stuff] > > but I don't think this happens enough to make us regret adding > comprehensions to the language. > > > -- > Steve > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > wes.turner%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Apr 27 06:25:24 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 27 Apr 2018 20:25:24 +1000 Subject: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 In-Reply-To: References: Message-ID: On 27 April 2018 at 03:18, Barry Warsaw wrote: > On Apr 26, 2018, at 09:28, Eric Snow wrote: >> >> On Thu, Apr 26, 2018 at 10:25 AM, Eric Snow wrote: >>> In pondering our approach to future Python major releases, I found >>> myself considering the experience we've had with Python 3. The whole >>> Py3k effort predates my involvement in the community so I missed a >>> bunch of context about the motivations, decisions, and challenges. >>> While I've pieced some of that together over the years now since I've >>> been around, I've certainly seen much of the aftermath. For me, at >>> least, it would be helpful to have a bit more insight into the >>> history. :) > > It would certainly be an interesting document, but I suspect you?ll get a bit of the old ?ask 3 lawyers and get 5 opinions? kind of response. ;) http://python-notes.curiousefficiency.org/en/latest/python3/questions_and_answers.html covers some of the questions Eric is asking (mostly from my PoV, but Guido corrected my answer to the initial "Why was Python 3 made incompatible with Python 2?" question shortly after I posted the first version of it). https://www.curiousefficiency.org/posts/2014/08/python-4000.html is a more retrospective-y article that looks more at the implications for Python 4. For the "What actually happened?" info, probably the 3 main documents to look at would be PEP 3000 (the process doc), PEP 3100 (accepted changes that didn't get their own PEPs), and PEP 3099 (explicitly rejected ideas that also didn't get their own PEPs). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From vstinner at redhat.com Fri Apr 27 06:30:00 2018 From: vstinner at redhat.com (Victor Stinner) Date: Fri, 27 Apr 2018 12:30:00 +0200 Subject: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 In-Reply-To: References: Message-ID: I gave the talk "Python 3: 10 years later" at FOSDEM and Pycon Italy and will give it again at Pycon US next month: https://fosdem.org/2018/schedule/event/python3/ My talk is focused on the migration path. How to "port" Python 2 code to Python 3, 2to3 tool, six module, things done to make the migration simpler, etc. Victor 2018-04-26 18:25 GMT+02:00 Eric Snow : > In pondering our approach to future Python major releases, I found > myself considering the experience we've had with Python 3. The whole > Py3k effort predates my involvement in the community so I missed a > bunch of context about the motivations, decisions, and challenges. > While I've pieced some of that together over the years now since I've > been around, I've certainly seen much of the aftermath. For me, at > least, it would be helpful to have a bit more insight into the > history. :) > > With that in mind, it would be worth having an informational PEP with > an authoritative retrospective on the lessons learned from the Python > 3 effort (and transition). Consider it a sort of autobiography, > "memoirs on the python-dev change to Python 3". :) At this point the > transition has settled in enough that we should be able to present a > relatively objective (and consistent) view, while we're not so far > removed that we've forgotten anything important. :) If such a > document already exists then I'd love a pointer to it. > > The document would benefit (among others): > > * python-dev (by giving us a clear viewpoint to inform decisions about > future releases) > * new-comers to Python that want more insight into the language > * folks transitioning from 2 to 3 > * communities that have (or think they have) problems similar to those > we faced in Python 2 > > The PEP doesn't even have to be done all at once, nor by one person. > In fact, there are many viewpoints that would add value to the > document. Hence it would probably make sense to encourage broad > participation and then have a single editor to effect a single voice > in the document. > > The contents of the retrospective document should probably cover a > broad range of topics, since there's so much to learn from the move to > Python 3. To give an indication of what I mean, I've included a rough > outline at the bottom of this message. > > So...I typically strongly avoid making proposals that I'm not willing > to execute. However, in this case I simply do not have enough > experience in the history to feel comfortable doing a good job of it > in a reasonable amount of time (which matters due to the tendency of > valuable info to fade away). :/ I have no expectation that someone > will pick this up, though I do hope since the benefit would be > significant. My apologies in advance if this wasted anyone's time. > > -eric > > > ++++++++++++++++++++++++++++++++ > > I'd hope to see something along the lines of (at least) the following, > in rough order: > > * a concise summary of the document at the top (very meta, I know :) ) > + what were we solving? > + what was the solution? > + why do it that way? > + what went right? > + what went wrong? > + impact on the community > + impact on core dev contribution > * timeline > * key players (and level of involvement) > + old guard core devs > + new guard > + folks brought on for Py3k (e.g. IIRC a swarm of Googlers dove in) > + non-core-devs > * motivations > * expectations (e.g. time frames, community reaction) > * corresponding results > * a summary of what we did > * alternative approaches > * what went right (and was it on purpose :) ) > * what went wrong (e.g. io) and why > * how the Py3k project differed from normal python-dev workflow (e.g. > pace, decision-making, communications) > * lasting impact of python-dev > * key things that would have been better if done differently > * key decisions/planning (mostly a priori to the release work) > + scope of backward compatibility > + process (using PEPs with PEPs 30xx guiding) > + schedule > + specific changes (i.e. PEPs 31xx) > + what was left out (and why) > + plans to help library and app authors transition (e.g. 2to3) > + feature/schedule overlap with Python 2 (i.e. 2.6 and 2.7) > + the language moratorium > * things that got missed and why > + unicode/bytes in some stdlib modules (and builtins?) > * things that were overdone (and how that got missed) > + unicode/bytes in some stdlib modules (and builtins?) > * (last but not least) challenges faced by folks working to transition > their exiting code to Python 3 > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com From wes.turner at gmail.com Fri Apr 27 07:03:42 2018 From: wes.turner at gmail.com (Wes Turner) Date: Fri, 27 Apr 2018 07:03:42 -0400 Subject: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 In-Reply-To: References: Message-ID: On Thursday, April 26, 2018, Eric Snow wrote: > In pondering our approach to future Python major releases, I found > myself considering the experience we've had with Python 3. The whole > Py3k effort predates my involvement in the community so I missed a > bunch of context about the motivations, decisions, and challenges. > While I've pieced some of that together over the years now since I've > been around, I've certainly seen much of the aftermath. For me, at > least, it would be helpful to have a bit more insight into the > history. :) > > With that in mind, it would be worth having an informational PEP with > an authoritative retrospective on the lessons learned from the Python > 3 effort (and transition). Consider it a sort of autobiography, > "memoirs on the python-dev change to Python 3". :) At this point the > transition has settled in enough that we should be able to present a > relatively objective (and consistent) view, while we're not so far > removed that we've forgotten anything important. :) If such a > document already exists then I'd love a pointer to it. > > The document would benefit (among others): > > * python-dev (by giving us a clear viewpoint to inform decisions about > future releases) > * new-comers to Python that want more insight into the language > * folks transitioning from 2 to 3 > * communities that have (or think they have) problems similar to those > we faced in Python 2 > > The PEP doesn't even have to be done all at once, nor by one person. > In fact, there are many viewpoints that would add value to the > document. Hence it would probably make sense to encourage broad > participation and then have a single editor to effect a single voice > in the document. > > The contents of the retrospective document should probably cover a > broad range of topics, since there's so much to learn from the move to > Python 3. To give an indication of what I mean, I've included a rough > outline at the bottom of this message. > > So...I typically strongly avoid making proposals that I'm not willing > to execute. However, in this case I simply do not have enough > experience in the history to feel comfortable doing a good job of it > in a reasonable amount of time (which matters due to the tendency of > valuable info to fade away). :/ I have no expectation that someone > will pick this up, though I do hope since the benefit would be > significant. My apologies in advance if this wasted anyone's time. > > -eric > > > ++++++++++++++++++++++++++++++++ > > I'd hope to see something along the lines of (at least) the following, > in rough order: > > * a concise summary of the document at the top (very meta, I know :) ) > + what were we solving? > + what was the solution? > + why do it that way? > + what went right? > + what went wrong? > + impact on the community > + impact on core dev contribution > * timeline > * key players (and level of involvement) > + old guard core devs > + new guard > + folks brought on for Py3k (e.g. IIRC a swarm of Googlers dove in) > + non-core-devs > * motivations > * expectations (e.g. time frames, community reaction) > * corresponding results > * a summary of what we did nine has a very concise, if incomplete in comparison to six and 2to3, code/namespace/functional summary: https://github.com/nandoflorestan/nine/blob/master/nine/__init__.py https://github.com/benjaminp/six/blob/master/six.py http://python-future.org/overview.html#automatic-conversion-to-py2-3-compatible-code > python-future comes with two scripts called futurize and pasteurize to aid in making Python 2 code or Python 3 code compatible with both platforms (Py2/3). It is based on 2to3 and uses fixers from lib2to3, lib3to2, and python-modernize, as well as custom fixers "Cheat Sheet: Writing Python 2-3 compatible code" http://python-future.org/compatible_idioms.html https://github.com/PythonCharmers/python-future > It provides future and past packages with backports and forward ports of features from Python 3 and 2. > * alternative approaches #!/usr/bin/env python # ~~?pythonver: > 3.3?~~ > * what went right (and was it on purpose :) ) > * what went wrong (e.g. io) and why > * how the Py3k project differed from normal python-dev workflow (e.g. > pace, decision-making, communications) > * lasting impact of python-dev > * key things that would have been better if done differently > * key decisions/planning (mostly a priori to the release work) > + scope of backward compatibility > + process (using PEPs with PEPs 30xx guiding) > + schedule > + specific changes (i.e. PEPs 31xx) > + what was left out (and why) > + plans to help library and app authors transition (e.g. 2to3) > + feature/schedule overlap with Python 2 (i.e. 2.6 and 2.7) > + the language moratorium > * things that got missed and why > + unicode/bytes in some stdlib modules (and builtins?) > * things that were overdone (and how that got missed) > + unicode/bytes in some stdlib modules (and builtins?) > * (last but not least) challenges faced by folks working to transition > their exiting code to Python 3 > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > wes.turner%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From storchaka at gmail.com Fri Apr 27 07:34:41 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Fri, 27 Apr 2018 14:34:41 +0300 Subject: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 In-Reply-To: References: Message-ID: 27.04.18 13:25, Nick Coghlan ????: > and PEP 3099 (explicitly > rejected ideas that also didn't get their own PEPs). "There will be no alternative binding operators such as :=." From steve at pearwood.info Fri Apr 27 08:28:46 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 27 Apr 2018 22:28:46 +1000 Subject: [Python-Dev] Reserve ':=' for type-inferred variable initialization (was PEP 572) In-Reply-To: References: <20180427003605.GK7400@ando.pearwood.info> Message-ID: <20180427122846.GQ7400@ando.pearwood.info> On Fri, Apr 27, 2018 at 08:13:20AM +0200, Andrea Griffini wrote: > >> We're focused on Python 3.8 and 3.9, not Python 5 or Python 6. > > Hmmm... When I was hearing the repeated belated saying that Python > will never ever jump on the statically typed ship on each and every > static type annotation discussion I started to worry this wasn't > indeed the case (why the urge of repeating it so much otherwise?). Only because people keep worrying about it. If we don't deny it, people will think our failure to deny it means it will happen. If we do deny it, they think that our repeated denials means it will happen. We're damned whatever we do. > Now we got standard library features requiring type annotation We do? Did I miss them? Which std lib features are you referring to? (That's not a rhetorical question -- maybe I have missed something.) > and a little shift towards a "not now" position. If you're referring to my comment above about "Python 5 or Python 6", perhaps I should have followed my first instinct and written "Python 5000". You know, the Python we get in the year 5000 :-) Guido has said that Python will never *require* type-annotations and static type-checking, and I see no reason to doubt that. But this doesn't rule out a hypothetical runtime option (hence, *optional*) to enforce static type-safety some time in the future. Personally I doubt this will happen: mypy is a non-trivial project itself, by my estimate about 200 .py files, 77000 lines of text, about 60 kloc, and I don't think Guido wants to build it into the reference Python interpreter. And why bother, when it is so easy to add a single dependency (mypy) and integrate it with your work-flow? My point was that reserving syntax for such a hypothetical future Python is a waste of time. Even if it happens, which it probably won't, it won't happen any time soon. By the time this hypothetical future rolls around, who knows what syntax we'll want? -- Steve From J.Demeyer at UGent.be Fri Apr 27 08:34:23 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Fri, 27 Apr 2018 14:34:23 +0200 Subject: [Python-Dev] PEP 575 (Unifying function/method classes) update Message-ID: <5AE318CF.9030403@UGent.be> Hello all, I have updated PEP 575 and its reference implementation. See https://www.python.org/dev/peps/pep-0575/ The main differences with respect to the previous version are: * METH_PASS_FUNCTION now passes the function *in addition* to self (previously, it was passed *instead* of self). * __objclass__ was generalized to __parent__ and stores either the defining class or the defining module of a built-in function/method. * Proposed two-phase implementation for better backwards compatibility (at the cost of added complexity). The first two items on the above list are meant to prepare for PEP 573 but are sufficiently useful by itself to add them to PEP 575. On this mailing list, there have been concerns about backwards compatibility. This PEP does indeed affect code not using duck typing, but using type checks or things like inspect.isbuiltin(). Note that "affect" != "break". I don't know how bad this presumed breakage is. Personally, I think it will be acceptable, but others may disagree. What I *do* know for sure is that very little breaks in the Python standard library. If anybody has a clever idea to estimate the breakage, I would love to know. Jeroen. From rymg19 at gmail.com Fri Apr 27 08:35:08 2018 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Fri, 27 Apr 2018 07:35:08 -0500 Subject: [Python-Dev] Is PEP 572 really the most effective way to solve the problems it's targeting? In-Reply-To: References: <162ffcf2f98.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> <39fe9f53-843b-3394-b7fc-c51a400a1409@mgmiller.net> <20180427005816.GL7400@ando.pearwood.info> Message-ID: <16307199860.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> On April 27, 2018 12:16:09 AM Mike Miller wrote: Sorry all, wasn't specific enough. By "modern" I mean the last decade perhaps. New languages that have had a chance to look at the older generations and choose their best ideas, while leaving behind the rest. Personally I thought of Swift (Ryan mentioned), Kotlin, Rust, and perhaps Go, though Go wasn't focused on breaking new ground outside of ease of concurrency. I don't know R or Felix at all, but sound interesting. Nim is another I'm vaguely aware of. They surely have given some thought to the issue. One thing that jumped out at me is that most replies here jumped to the question of whether they supported assignment-expressions, but that is only one potential solution. To be more clear, I wondered how did they solve "the problem itself?" Was their solution different? Ryan somewhat alluded to that, but I'd like to dig in a bit on that part. In contrast, in many of the other threads I heard, "C, C++, C#, Java, etc do assignment-expressions, they're useful and not so hard to learn." Ok that's reasonable, but where is the industry headed? Python deferred long enough that we don't necessarily have to choose a classic solution. So, it sounds like many of the new generation of languages are not embracing these expressions everywhere but rather letting folks do an assignment right in the statement where their use case applies, if, while, maybe comprehensions. Is that accurate? This is basically what I was trying to say, except far better worded... Looks like I've got some homework to do, haha. -Mike On 2018-04-26 17:58, Steven D'Aprano wrote: What counts as a modern language? Less than five years old? Less than fifty years old? Are Javascript, Ruby and R modern? They all support assignment as expressions. I think Koitlin, Rust and Go prohibit assignment as expressions. Swift assignment evaluates as Void (equivalent to None in Python, I guess), so you can use assignment in an expression but it returns nothing and only operates by side-effect. As far as type hints go, I think that if you need explicit type hints in the middle of an expression, it's a bad idea and you ought to pull it out as a separate statement. That applies regardless of whether that expression involves binding or not. _______________________________________________ Python-Dev mailing list Python-Dev at python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com -- Ryan (????) Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone else https://refi64.com/ From vstinner at redhat.com Fri Apr 27 08:13:32 2018 From: vstinner at redhat.com (Victor Stinner) Date: Fri, 27 Apr 2018 14:13:32 +0200 Subject: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 In-Reply-To: References: Message-ID: In my opinion, the largest failure of Python 3 is that we failed to provide a smooth and *slow* transition from Python 2 and Python 3. It can be explained by the long list of backward incompatible changes. My advice would be to restrict the number of backward incompatible changes per release, and always emit a warning (at runtime) in the previous release. For example, "async" and "await" have been marked as deprecated in Python 3.5, before becoming real keywords in Python 3.6. Then people complained that they didn't see the warning which is hidden by default, but that's another topic :-) (This issue is partially solved in Python 3.7 with Nick's PEP 565 and my -X dev option). I don't think that having Python X.Y which introduces backward incompatible changes is an issue by itself. We did it multiple times during the Python 3 cycle: my PEP 446 (non-inheritable file descriptors) and PEP 475 (retry syscalls interrupted by signals, PEP co-written with Charles-Fran?ois Natali) introduced backward incompatible changes in Python 3.4 and 3.5. I am aware that they broke a few applications, but it was possible to manage these issues because each release only introduced a few backward incompatible changes. The main issue is the deprecation process. Should we provide tools to automatic conversion? Should we only document the deprecation, or also emit a warning at runtime? Should the warning be displayed by default? It's also a matter of collaboration with the Python community. For example, help major Python projects to handle these changes. Nobody wants to see pip broken by the next Python release for example. It seems like we are already working closely with pip, Cython and numpy, for example. IMHO queuing backward incompatible changes until Python 4 is a very bad idea. Start to break things early :-) But always follow the deprecation process. If my vote counts, Python 4.0 should just be the version following the previous Python 3.x release, as the Linux kernel is now doing (Linux 3 and Linux 4 are just regular release, they don't break everything.) Note: Another option is to never introduce backward incompatible changes, no? ;-) Victor From ncoghlan at gmail.com Fri Apr 27 08:56:06 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 27 Apr 2018 22:56:06 +1000 Subject: [Python-Dev] Reserve ':=' for type-inferred variable initialization (was PEP 572) In-Reply-To: <20180427122846.GQ7400@ando.pearwood.info> References: <20180427003605.GK7400@ando.pearwood.info> <20180427122846.GQ7400@ando.pearwood.info> Message-ID: On 27 April 2018 at 22:28, Steven D'Aprano wrote: > On Fri, Apr 27, 2018 at 08:13:20AM +0200, Andrea Griffini wrote: > > Now we got standard library features requiring type annotation > > We do? Did I miss them? Which std lib features are you referring to? > > (That's not a rhetorical question -- maybe I have missed something.) > Data classes rely on the presence of annotations to spot field declarations (it mostly doesn't care what those annotations actually say, but it does need them to be present in order to create the list of field names). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Fri Apr 27 09:04:34 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 27 Apr 2018 23:04:34 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <20180426061000.GF7400@ando.pearwood.info> <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> <20180427053837.GN7400@ando.pearwood.info> Message-ID: On Fri, Apr 27, 2018 at 8:18 PM, Wes Turner wrote: > IDK, I could just be resistant to change, but this seems like something that > will decrease readability -- and slow down code review -- without any real > performance gain. So, while this would be useful for golfed-down (!) > one-liners with pyline, > I'm -1 on PEP 572. PEP 572 has never promised a performance gain, so "without any real performance gain" is hardly a criticism. > How do I step through this simple example with a debugger? > > if re.search(pat, text) as match: > print("Found:", match.group(0)) Step the 'if' statement. It will call re.search() and stash the result in 'match'. Then the cursor would be put either on the 'print' (if the RE matched) or on the next executable line (if it didn't). > From https://en.wikipedia.org/wiki/Assignment_(computer_science) : > >> In some languages the symbol used is regarded as an operator (meaning that >> the assignment has a value) while others define the assignment as a >> statement (meaning that it cannot be used in an expression). > > > PEP 572 -- Assignment Expressions > PEP 572 -- Assignment Operator (:=) and Assignment Expressions Huh? I don't get your point. ChrisA From eric at trueblade.com Fri Apr 27 08:58:06 2018 From: eric at trueblade.com (Eric V. Smith) Date: Fri, 27 Apr 2018 08:58:06 -0400 Subject: [Python-Dev] Reserve ':=' for type-inferred variable initialization (was PEP 572) In-Reply-To: <20180427122846.GQ7400@ando.pearwood.info> References: <20180427003605.GK7400@ando.pearwood.info> <20180427122846.GQ7400@ando.pearwood.info> Message-ID: <706c8b71-2157-3bcb-a5f3-fd190822fa23@trueblade.com> On 4/27/2018 8:28 AM, Steven D'Aprano wrote: > On Fri, Apr 27, 2018 at 08:13:20AM +0200, Andrea Griffini wrote: >> Now we got standard library features requiring type annotation > > We do? Did I miss them? Which std lib features are you referring to? > > (That's not a rhetorical question -- maybe I have missed something.) Presumably dataclasses and typing.NamedTuple. Eric From ericsnowcurrently at gmail.com Fri Apr 27 10:42:56 2018 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Fri, 27 Apr 2018 08:42:56 -0600 Subject: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 In-Reply-To: References: Message-ID: On Thu, Apr 26, 2018 at 10:25 AM, Eric Snow wrote: > In pondering our approach to future Python major releases, I found > myself considering the experience we've had with Python 3. The whole > Py3k effort predates my involvement in the community so I missed a > bunch of context about the motivations, decisions, and challenges. > While I've pieced some of that together over the years now since I've > been around, I've certainly seen much of the aftermath. For me, at > least, it would be helpful to have a bit more insight into the > history. :) Thanks, all, for the responses. :) -eric From encukou at gmail.com Fri Apr 27 11:25:05 2018 From: encukou at gmail.com (Petr Viktorin) Date: Fri, 27 Apr 2018 11:25:05 -0400 Subject: [Python-Dev] PEP 394 update proposal: Allow changing the `python` command in some cases In-Reply-To: <85muxpp4nu.fsf@benfinney.id.au> References: <4c4ec942-e17b-c4f4-4b2f-459fed2c32aa@gmail.com> <85wowttuyt.fsf@benfinney.id.au> <85muxpp4nu.fsf@benfinney.id.au> Message-ID: On 04/27/18 02:03, Ben Finney wrote: > Ben Finney writes: > >> Petr Viktorin writes: >> >>> [?] we feel that the only way to *enforce* that guidelines is to >>> provide environments where the `python` command does not work >>> (unless explicitly installed). >> >> Yes. The ?python? command is confusing, for the reasons you say. There >> should be ?python2? and ?python3? commands for Python 2 and Python 3 >> respectively, and no ?python? command should be installed by the >> operating system. >> >> The fact that ?/usr/bin/python? exists is an historical accident, and I >> agree with the proposal you state: the best way to correct the confusion >> is to bar the confusing command from being installed by packages. > > Because the above is ambiguous, I'll clarify: I am not calling for, and > PEP 394 does not call for, the banishment of the ?python? command. Well, Guido *is* calling for it :) It would break too many things, but after discussions on the PR, it's clear that we want a future where the "python" doesn't exist. But while it's available, it should point to Python 2. > What I'm saying is that muddying the rules further on what ?python? may > or may not mean is *worse than* banishing the ?python? command entirely. That's also consistent with the PR discussion. (But not that much with the original PEP, which said `python` is expected to eventually mean `python3`.) > So, short of banishing ?python? entirely, I think PEP 394 is already a > good clear way to address the issue. Existing, documented and supported > means to locally modify a ?python? command already exist and should be > sufficient. > >> I trust that PEP 394 will not be weakened in its effect, and I wish you >> well with using the already-supported, already-documented, PEP-394 >> compatible means to add local customisations for a ?python? command. Right. But some already-supported, already-documented mechanisms like Debian Alternatives or alternate package repos, are not compatible with PEP 394. And as a PEP 394 compliant distro, we also won't be promoting the /usr/local or $HOME/bin ways to change `python` (which makes me a bit sad, because that documentation might have included a link to the caveats listed in the PEP). From encukou at gmail.com Fri Apr 27 11:37:14 2018 From: encukou at gmail.com (Petr Viktorin) Date: Fri, 27 Apr 2018 11:37:14 -0400 Subject: [Python-Dev] PEP 394: Allow the `python` command to not be installed (and other minor edits) In-Reply-To: <85wowttuyt.fsf@benfinney.id.au> References: <4c4ec942-e17b-c4f4-4b2f-459fed2c32aa@gmail.com> <85wowttuyt.fsf@benfinney.id.au> Message-ID: Hello, After discussion on the [Pull Request], my update to PEP 394 changed scope somewhat. The new major changes are: - The `python` command may not exist at all in some cases (see the PEP for details) - The paragraph about the anticipated future where python points to Python 3 is removed. (We'd rather see a future where `python` doesn't exist and one always has to specify `python2` or `python3`.) - The PEP now explicitly says that in an active venv, python means that venv's interpreter. (Some view this as a design mistake, but one not worth reverting now.) There are also other edits and clarifications. Thanks for everyone involved, especially Guido for pitching in with the intended direction -- which was not clear from (or is genuinely different from) the 7-year-old PEP! I'll keep the PR open for a day or so, in case someone still wants to comment. [Pull Request]: https://github.com/python/peps/pull/630 On 04/26/18 19:21, Ben Finney wrote: > Petr Viktorin writes: > >> In Fedora, I found that PEP 394's strict recommendation that `python` >> points to `python2` is holding us back. > > I have read the message, but I don't see how you draw the link that PEP > 394 is holding you back. > >> The problems are: >> - For developers that are not following the language's development, >> the fact that `python` invokes `python2` sends a strong signal that 2 >> is somehow the preferred version, and it's OK to start new projects in >> it. > > I agree with the statement you make later in the message: > >> [?] we feel that the only way to *enforce* that guidelines is to >> provide environments where the `python` command does not work (unless >> explicitly installed). > > Yes. The ?python? command is confusing, for the reasons you say. There > should be ?python2? and ?python3? commands for Python 2 and Python 3 > respectively, and no ?python? command should be installed by the > operating system. > > The fact that ?/usr/bin/python? exists is an historical accident, and I > agree with the proposal you state: the best way to correct the confusion > is to bar the confusing command from being installed by packages. > >> - Users and sysadmins that *do* want to ?live in the future? are >> switching the symlink to `python3` themselves. We would like to give >> them a supported, documented way to do so -- and make surer they're >> aware of the caveats. > > The supported, documented way to add a command pointing to a different > command already exists, and there is no need to make a Python-specific > special case. > > Users who want to make a ?python? alias can do so in their shell; this > is supported and documented. > > Users who want to add a new command file can add a suitable directory > (e.g. ?$HOME/bin?) to their ?PATH? variable, and put a symlink in there > named ?python?. This is supported and documented. > > Sysadmins who want to create a system-wide command ?python? can put a > symlink at ?/usr/local/bin/python?. This is supported and documented. > > I disagree with making some special-case extra way; that would be both > cunfusing and superfluous. > >> - The `python` command is still not available out-of-the box on macOS, >> so it didn't completely live up to the expectation of being the >> cross-platform way to launch 2/3 source compatile scripts. > > That is one of the minor ways which macOS fails to conform to > community-agreed conventions. We should not let that intransigence > distort our discussion of best practices. > >> To help solve these, I would like to relax recommendations on the Unix >> ``python -> python2`` symlink in these cases: > > For the above reasons, I disagree that PEP 394 is limiting what you want > to do on free-software operating systems. > > For non-free operating systems, I don't think the already-discussed PEP > 394 should be weakened if the operating system vendor fails to conform. > >> - Users and administrators can, by a deliberate action, change >> ``python`` to invoke Python 3. > > Yes. That is well-known and long-standardised on Unix operating systems, > and is much more broadly understood than any Python-specific special > case would be. So I don't see how anyone is being held back. > > I trust that PEP 394 will not be weakened in its effect, and I wish you > well with using the already-supported, already-documented, PEP-394 > compatible means to add local customisations for a ?python? command. > From ethan at stoneleaf.us Fri Apr 27 11:57:55 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 27 Apr 2018 08:57:55 -0700 Subject: [Python-Dev] quoted text not being marking correctly [was PEP 572 really the most effective way to solve the problems it's targeting?] In-Reply-To: <16307199860.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> References: <162ffcf2f98.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> <39fe9f53-843b-3394-b7fc-c51a400a1409@mgmiller.net> <20180427005816.GL7400@ando.pearwood.info> <16307199860.2837.db5b03704c129196a4e9415e55413ce6@gmail.com> Message-ID: <5AE34883.60607@stoneleaf.us> On 04/27/2018 05:35 AM, Ryan Gonzalez wrote: [snip] Ryan, the quoted text in your emails is not being marked as such, meaning that the entire email appears to be from you. This makes it really difficult to pick out your responses. I would really appreciate it of you could figure out why that is happening and resolve it. (I have no clue so can't offer any ideas.) Thanks. -- ~Ethan~ From status at bugs.python.org Fri Apr 27 12:09:58 2018 From: status at bugs.python.org (Python tracker) Date: Fri, 27 Apr 2018 18:09:58 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20180427160958.10B8956CF8@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2018-04-20 - 2018-04-27) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 6615 (+28) closed 38510 (+26) total 45125 (+54) Open issues with patches: 2586 Issues opened (45) ================== #3692: improper scope in list comprehension, when used in class decla https://bugs.python.org/issue3692 reopened by Mariatta #32799: returned a resul https://bugs.python.org/issue32799 reopened by xiang.zhang #33144: random._randbelow optimization https://bugs.python.org/issue33144 reopened by serhiy.storchaka #33321: Add a Linux clang ubsan undefined behavior sanitizer buildbot https://bugs.python.org/issue33321 opened by gregory.p.smith #33325: Optimize sequences of constants in the compiler https://bugs.python.org/issue33325 opened by serhiy.storchaka #33326: Convert collections (cmp_op, hasconst, hasname and others) in https://bugs.python.org/issue33326 opened by godaygo #33327: Add a method to move messages to IMAPlib https://bugs.python.org/issue33327 opened by mcepl #33328: pdb error when stepping through generator https://bugs.python.org/issue33328 opened by Ricyteach #33330: Better error handling in PyImport_Cleanup() https://bugs.python.org/issue33330 opened by serhiy.storchaka #33331: Clean modules in the reversed order https://bugs.python.org/issue33331 opened by serhiy.storchaka #33332: Expose valid signal set (sigfillset()) https://bugs.python.org/issue33332 opened by pitrou #33333: ConfigParser.items returns items present in `DEFAULTSECT` when https://bugs.python.org/issue33333 opened by chrBrd #33335: turtle.onkey doesn't pass key information when key is None https://bugs.python.org/issue33335 opened by je1234 #33336: [imaplib] MOVE is a legal command https://bugs.python.org/issue33336 opened by mcepl #33337: Provide a supported Concrete Syntax Tree implementation in the https://bugs.python.org/issue33337 opened by lukasz.langa #33338: [lib2to3] Synchronize token.py and tokenize.py with the standa https://bugs.python.org/issue33338 opened by lukasz.langa #33339: Using default encoding with `subprocess.run()` is not obvious https://bugs.python.org/issue33339 opened by pekka.klarck #33340: Inaccurate docs on `import` behaviour https://bugs.python.org/issue33340 opened by sam_b #33341: python3 fails to build if directory or sysroot contains "*icc* https://bugs.python.org/issue33341 opened by locutusofborg #33342: urllib IPv6 parsing fails with special characters in passwords https://bugs.python.org/issue33342 opened by benaryorg #33343: [argparse] Add subcommand abbreviations https://bugs.python.org/issue33343 opened by porton #33345: Documentation for PowerShell instructions https://bugs.python.org/issue33345 opened by stevepiercy #33346: Syntax error with async generator inside dictionary comprehens https://bugs.python.org/issue33346 opened by pablogsal #33347: zlibmodule undefined reference https://bugs.python.org/issue33347 opened by Lucian Cristian #33348: lib2to3 doesn't parse f(*[] or []) https://bugs.python.org/issue33348 opened by zsol #33349: 2to3 fails to parse async generators in non-async functions https://bugs.python.org/issue33349 opened by zsol #33350: WinError 10038 is raised when loop.sock_connect is wrapped wit https://bugs.python.org/issue33350 opened by Alisue Lambda #33351: Support compiling with clang-cl on Windows https://bugs.python.org/issue33351 opened by Ethan Smith #33352: Windows: test_regrtest fails on installed Python https://bugs.python.org/issue33352 opened by vstinner #33353: test_asyncio: test_sock_sendfile_mix_with_regular_send() hangs https://bugs.python.org/issue33353 opened by vstinner #33354: Python2: test_ssl fails on non-ASCII path https://bugs.python.org/issue33354 opened by vstinner #33355: Windows 10 buildbot: 15 min timeout on test_mmap.test_large_fi https://bugs.python.org/issue33355 opened by vstinner #33356: Windows 10 buildbot: test__xxsubinterpreters.test_already_runn https://bugs.python.org/issue33356 opened by vstinner #33357: [EASY C] test_posix.test_posix_spawn_file_actions() leaks memo https://bugs.python.org/issue33357 opened by vstinner #33358: [EASY] x86 Ubuntu Shared 3.x: test_embed.test_pre_initializati https://bugs.python.org/issue33358 opened by vstinner #33360: ALternative recipe for password using secrets https://bugs.python.org/issue33360 opened by jpc4242 #33361: readline() + seek() on io.EncodedFile breaks next readline() https://bugs.python.org/issue33361 opened by da #33363: async for statement is not a syntax error in sync context https://bugs.python.org/issue33363 opened by zsol #33365: http/client.py does not print correct headers in debug https://bugs.python.org/issue33365 opened by mstrigl #33366: `contextvars` documentation incorrectly refers to "non-local s https://bugs.python.org/issue33366 opened by tomchristie #33367: Multiprocessing Pool workers initiated with maxtasksperchild d https://bugs.python.org/issue33367 opened by Soumyadipta Das #33369: Removing Popen log files in threads is racy on Windows https://bugs.python.org/issue33369 opened by pbos #33370: Addition of mypy cache to gitignore https://bugs.python.org/issue33370 opened by onlined #33373: ttk modules Label class does not respect background config opt https://bugs.python.org/issue33373 opened by cmaceachern #33374: generate-posix-vars failed when building Python 2.7.14 on Linu https://bugs.python.org/issue33374 opened by piotr.dobrogost Most recent 15 issues with no replies (15) ========================================== #33374: generate-posix-vars failed when building Python 2.7.14 on Linu https://bugs.python.org/issue33374 #33373: ttk modules Label class does not respect background config opt https://bugs.python.org/issue33373 #33370: Addition of mypy cache to gitignore https://bugs.python.org/issue33370 #33367: Multiprocessing Pool workers initiated with maxtasksperchild d https://bugs.python.org/issue33367 #33365: http/client.py does not print correct headers in debug https://bugs.python.org/issue33365 #33356: Windows 10 buildbot: test__xxsubinterpreters.test_already_runn https://bugs.python.org/issue33356 #33353: test_asyncio: test_sock_sendfile_mix_with_regular_send() hangs https://bugs.python.org/issue33353 #33352: Windows: test_regrtest fails on installed Python https://bugs.python.org/issue33352 #33349: 2to3 fails to parse async generators in non-async functions https://bugs.python.org/issue33349 #33348: lib2to3 doesn't parse f(*[] or []) https://bugs.python.org/issue33348 #33346: Syntax error with async generator inside dictionary comprehens https://bugs.python.org/issue33346 #33342: urllib IPv6 parsing fails with special characters in passwords https://bugs.python.org/issue33342 #33339: Using default encoding with `subprocess.run()` is not obvious https://bugs.python.org/issue33339 #33336: [imaplib] MOVE is a legal command https://bugs.python.org/issue33336 #33335: turtle.onkey doesn't pass key information when key is None https://bugs.python.org/issue33335 Most recent 15 issues waiting for review (15) ============================================= #33370: Addition of mypy cache to gitignore https://bugs.python.org/issue33370 #33366: `contextvars` documentation incorrectly refers to "non-local s https://bugs.python.org/issue33366 #33365: http/client.py does not print correct headers in debug https://bugs.python.org/issue33365 #33363: async for statement is not a syntax error in sync context https://bugs.python.org/issue33363 #33358: [EASY] x86 Ubuntu Shared 3.x: test_embed.test_pre_initializati https://bugs.python.org/issue33358 #33354: Python2: test_ssl fails on non-ASCII path https://bugs.python.org/issue33354 #33349: 2to3 fails to parse async generators in non-async functions https://bugs.python.org/issue33349 #33348: lib2to3 doesn't parse f(*[] or []) https://bugs.python.org/issue33348 #33338: [lib2to3] Synchronize token.py and tokenize.py with the standa https://bugs.python.org/issue33338 #33337: Provide a supported Concrete Syntax Tree implementation in the https://bugs.python.org/issue33337 #33336: [imaplib] MOVE is a legal command https://bugs.python.org/issue33336 #33333: ConfigParser.items returns items present in `DEFAULTSECT` when https://bugs.python.org/issue33333 #33332: Expose valid signal set (sigfillset()) https://bugs.python.org/issue33332 #33331: Clean modules in the reversed order https://bugs.python.org/issue33331 #33330: Better error handling in PyImport_Cleanup() https://bugs.python.org/issue33330 Top 10 most discussed issues (10) ================================= #33337: Provide a supported Concrete Syntax Tree implementation in the https://bugs.python.org/issue33337 14 msgs #33312: ubsan undefined behavior sanitizer flags struct _dictkeysobjec https://bugs.python.org/issue33312 9 msgs #33315: Allow queue.Queue to be used in type annotations https://bugs.python.org/issue33315 7 msgs #33277: Deprecate __loader__, __package__, __file__, and __cached__ on https://bugs.python.org/issue33277 6 msgs #33361: readline() + seek() on io.EncodedFile breaks next readline() https://bugs.python.org/issue33361 6 msgs #33275: glob.glob should explicitly note that results aren't sorted https://bugs.python.org/issue33275 5 msgs #33330: Better error handling in PyImport_Cleanup() https://bugs.python.org/issue33330 5 msgs #21822: KeyboardInterrupt during Thread.join hangs that Thread https://bugs.python.org/issue21822 4 msgs #31141: Start should be a keyword argument of the built-in sum https://bugs.python.org/issue31141 4 msgs #31463: test_multiprocessing_fork hangs test_subprocess https://bugs.python.org/issue31463 4 msgs Issues closed (26) ================== #25427: Remove the pyvenv script in Python 3.8 https://bugs.python.org/issue25427 closed by brett.cannon #27485: urllib.splitport -- is it official or not? https://bugs.python.org/issue27485 closed by lukasz.langa #31606: [Windows] Tests failing on installed Python 3.7a1 on Windows https://bugs.python.org/issue31606 closed by vstinner #32209: Crash in set_traverse Within the Garbage Collector's collect_g https://bugs.python.org/issue32209 closed by connorwfitzgerald #32232: building extensions as builtins is broken in 3.7 https://bugs.python.org/issue32232 closed by ncoghlan #32652: test_distutils: BuildRpmTestCase tests fail on RHEL buildbots https://bugs.python.org/issue32652 closed by vstinner #33128: PathFinder is twice on sys.meta_path https://bugs.python.org/issue33128 closed by ncoghlan #33131: Upgrade to pip 10 for Python 3.7 https://bugs.python.org/issue33131 closed by ncoghlan #33258: Unable to install 3.6.5 on Windows Server 2008 https://bugs.python.org/issue33258 closed by hpo0016 #33266: 2to3 doesn't parse all valid string literals https://bugs.python.org/issue33266 closed by zsol #33276: Clarify that __path__ can't be set to just anything https://bugs.python.org/issue33276 closed by brett.cannon #33280: Update link to Tcl/Tk 8.6 man pages in tkinter.rst https://bugs.python.org/issue33280 closed by serhiy.storchaka #33297: Mention Pillow package on tkinter.rst to work with more image https://bugs.python.org/issue33297 closed by serhiy.storchaka #33317: `repr()` of string in NFC and NFD forms does not differ https://bugs.python.org/issue33317 closed by benjamin.peterson #33322: Overridden __getitem__ not called on use of slice syntax when https://bugs.python.org/issue33322 closed by steven.daprano #33323: inconsistent stack trace for exceptions thrown in generators p https://bugs.python.org/issue33323 closed by serhiy.storchaka #33324: Bug in documentation 3.6: string-methods - str.center https://bugs.python.org/issue33324 closed by Alfonso Chavez #33329: sigaddset() can fail on some signal numbers https://bugs.python.org/issue33329 closed by pitrou #33334: Add support of NOP and EXTENDED_ARG in stack_effect() https://bugs.python.org/issue33334 closed by serhiy.storchaka #33344: Use logical negation of integers directly in arithmatic propos https://bugs.python.org/issue33344 closed by Kasra Vand #33359: curses.has_key failure https://bugs.python.org/issue33359 closed by benjamin.peterson #33362: string strip() strips extra characters that it shouldn't https://bugs.python.org/issue33362 closed by steven.daprano #33364: Conditionals not evaluating propertly https://bugs.python.org/issue33364 closed by serhiy.storchaka #33368: Inaccuracy in https://docs.python.org/3/library/re.html#re.mat https://bugs.python.org/issue33368 closed by serhiy.storchaka #33371: Clarify the predicate parameter of inspect.getmembers https://bugs.python.org/issue33371 closed by brian.curtin #33372: Wrong calculation https://bugs.python.org/issue33372 closed by tim.peters From chris.barker at noaa.gov Fri Apr 27 12:49:33 2018 From: chris.barker at noaa.gov (Chris Barker) Date: Fri, 27 Apr 2018 09:49:33 -0700 Subject: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow In-Reply-To: References: Message-ID: On Tue, Apr 24, 2018 at 2:21 AM, Victor Stinner wrote: > Even if the C language allows assignments in if, I avoid them, because > I regularly have to debug my own code in gdb ;-) > I personally haven't written a lot of C, so have no personal experience, but if this is at all a common approach among experienced C developers, it tells us a lot. We shouldn't add a feature that people would make a point of avoiding! OT note: > Now the question is which Python are allowed for babies. I recall that > a colleague was surprised and confused by context managers. Does it > mean that try/finally should be preferred? well, no, because try ... finally is even more confusing -- at least that's the impression I get from spending 20 minutes on it with newbies in my class last night :-) > Or metaclasses? metaclasses are well known to be advanced juju -- they should not (and probably don't) show up in everyday code. Even if they are used in everyday code, the use is usually hidden -- i.e. when a user subclasses from an ORM model class, they are using metaclasses, but they don't have to know that. That is -- they fall into the category of stuff that should only be used by library/framework developers -- and, in fact, the whole point is to make everyday code easier. In fact, the number of developers that need to write/debug metaclasses, context managers, decorators, is far fewer than the number of folks that USE those things. So the standards are very different. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Fri Apr 27 14:18:14 2018 From: chris.barker at noaa.gov (Chris Barker) Date: Fri, 27 Apr 2018 11:18:14 -0700 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> Message-ID: On Thu, Apr 26, 2018 at 1:33 PM, Tim Peters wrote: > And that *is* a thing that you will have to explain to newbies when they > encounter > > it for the first time. > > Sure. That doesn't frighten me, though. It's easy to explain what it > does - although it may be hard to explain when it's _desirable_ to use > it. > I'm with Raymond here -- though I'm not sure "newbies" is quite right -- I've found that newbies fall into two camps: folks to whom programming comes naturally, and those that it doesn't (OK, it's a distribution, but a bimodal one). And folks that are struggling with programming can struggle even with simple assignment (name binding), particularly when you add even function local scope. So having one more way to do assignment WILL make it harder to teach, not because it's that hard, but because it's one more thing to learn. But the fact is that as Python has evolved (particularly with the jump to py3) it has become less and less of a "scripting" language, and more of a "systems" language. And also harder to learn. Anyone remember CP4E? Python is not as good choice as a "newbie" language as it once was. Adding := will move it a little bit more along the complexity path -- not much, and that's where Python has gone anyway, so as Tim said, no one's going to suffer either way this decision goes. Hmm -- I wonder if a "pythonscript" will get forked off one day...... To judge from Stackoverflow volume, the single most misunderstood of > all Python operators - by far - is "is" - You now, I think instructors like me are partly responsible. "is" is rarely useful outside of comparing to singletons. Yet I use it early in instruction to do checks on name binding and show things with mutablilty, etc.... which has the unfortunate side effect of making it seem like a more common operator than it is. I've even had students write code like: if x is 3: and thanks to interning, it appears to work! -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.peters at gmail.com Fri Apr 27 15:19:43 2018 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 27 Apr 2018 14:19:43 -0500 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> Message-ID: [Lukasz] >> > And that *is* a thing that you will have to explain to newbies when >> > they encounter it for the first time. [Tim] >> Sure. That doesn't frighten me, though. It's easy to explain what it >> does - although it may be hard to explain when it's _desirable_ to use >> it. [Chris Barker ] > I'm with Raymond here -- though I'm not sure "newbies" is quite right -- > I've found that newbies fall into two camps: folks to whom programming comes > naturally, and those that it doesn't (OK, it's a distribution, but a bimodal > one). And folks that are struggling with programming can struggle even with > simple assignment (name binding), particularly when you add even function > local scope. Sure. What I wrote was shorthand for what's already been covered at length many times: what a binding expression does is "easy to explain" GIVEN THAT someone ALREADY UNDERSTANDS how binding a name works. The latter in fact seems difficult for a significant number of people to learn, but it's utterly unavoidable that they learn it if they're ever to write non-trivial Python programs. That's been true since Python's first release. Binding expressions would be introduced much later in any sane course. At THAT point, for students who haven't already dropped out, the semantics are darned-near trivial to explain: it binds the name to the object the expression evaluates to (all of which they _already_ understand by this point), and the value of the binding expression is that object (the only new bit). Unlike as for most other operators, you don't even have to weasel-word it to account for that a magical dunder method may change what ":=" does. As for the "is" operator, the meaning is baked into the language and can't be altered in the slightest. > So having one more way to do assignment WILL make it harder to > teach, not because it's that hard, but because it's one more thing to learn. On a scale of 1 to a million, try to quantify how much harder ;-) As above, I can't see it getting beyond a single digit, GIVEN THAT a student has already masteredf the far more complex assignment _statement_ (binding expressions are limited to the single simplest case of the many things an assignment statement can do). "And it returns the object" is a yawn. But, as I already granted, it may be truly hard to explain when it's a desirable thing to use. That takes experience and "good judgment", which - according to me - can be learned but can't really be taught. > But the fact is that as Python has evolved (particularly with the jump to > py3) it has become less and less of a "scripting" language, and more of a > "systems" language. And also harder to learn. Anyone remember CP4E? Python > is not as good choice as a "newbie" language as it once was. I agree - although I expect sticking to a subset of Python could make life easier for beginners. For example, would anyone in their right mind even mention async gimmicks when teaching beginners? Against that, though, one of the most unintentionally funny tech things I ever read was Bjarne Stroustrup writing about why C++ is an excellent choice for beginners. But he does have a point: if you throw away the bulk of everything C++ added, there's an easily usable little language exceedingly well hidden under it all ;-) > Adding := will move it a little bit more along the complexity path -- not > much, and that's where Python has gone anyway, so as Tim said, no one's > going to suffer either way this decision goes. Yet there will be much wailing and gnashing of teeth anyway ;-) ... >> To judge from Stackoverflow volume, the single most misunderstood of >> all Python operators - by far - is "is" - > You now, I think instructors like me are partly responsible. "is" is rarely > useful outside of comparing to singletons. Yet I use it early in instruction > to do checks on name binding and show things with mutablilty, etc.... which > has the unfortunate side effect of making it seem like a more common > operator than it is. > > I've even had students write code like: > > if x is 3: > > and thanks to interning, it appears to work! Yup, that's the real problem with "is": its semantics are dead simple, but "but under exactly what conditions are `x` and `y` bound to the same object?" is intractable. It seems to take a long time to get across the point, that the question itself is misguided. A full answer requires delving into transient implementation details, which is counterproductive because they _are_ accidents of the implementation du jour. What questioners need to be nudged into asking instead is for examples of when using "is" is thoroughly sane. From wes.turner at gmail.com Fri Apr 27 16:06:23 2018 From: wes.turner at gmail.com (Wes Turner) Date: Fri, 27 Apr 2018 16:06:23 -0400 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <20180426061000.GF7400@ando.pearwood.info> <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> <20180427053837.GN7400@ando.pearwood.info> Message-ID: On Friday, April 27, 2018, Chris Angelico wrote: > On Fri, Apr 27, 2018 at 8:18 PM, Wes Turner wrote: > > IDK, I could just be resistant to change, but this seems like something > that > > will decrease readability -- and slow down code review -- without any > real > > performance gain. So, while this would be useful for golfed-down (!) > > one-liners with pyline, > > I'm -1 on PEP 572. > > PEP 572 has never promised a performance gain, so "without any real > performance gain" is hardly a criticism. > > > How do I step through this simple example with a debugger? > > > > if re.search(pat, text) as match: > > print("Found:", match.group(0)) > > Step the 'if' statement. It will call re.search() and stash the result > in 'match'. Then the cursor would be put either on the 'print' (if the > RE matched) or on the next executable line (if it didn't). Right. Pdb doesn't step through the AST branches of a line, so ternary expressions and list comprehensions and defining variables at the end of the line are 'debugged' after they're done. Similarly, code coverage is line-based; so those expressions may appear to be covered but are not. > > > From https://en.wikipedia.org/wiki/Assignment_(computer_science) : > > > >> In some languages the symbol used is regarded as an operator (meaning > that > >> the assignment has a value) while others define the assignment as a > >> statement (meaning that it cannot be used in an expression). > > > > > > PEP 572 -- Assignment Expressions > > PEP 572 -- Assignment Operator (:=) and Assignment Expressions > > Huh? I don't get your point. Q: What is ':='? (How do I searchengine for it?) A: That's the assignment operator which only works in Python 3.8+. Q: When are variables defined -- or mutable names bound -- at the end of the expression accessible to the left of where they're defined? Q: What about tuple unpacking? Is there an ECMAscript-like destructuring PEP yet? A: Ternary expressions; list, dict, generator comprehensions; (@DOCS PLEASE HELP EXPLAIN THIS) Q: do these examples of the assignment expression operator all work? """ - debuggers have no idea what to do with all of this on one line - left-to-right doesn't apply to comprehensions results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0] - left-to-right doesn't apply to ternary expressions if (y := func(x)) if (x := 3) else 0: while (y := func(x)) if (x := 3) else 0: - left-to-right does apply to everything else? - *these* are discouraged: if (x := 3) or (y := func(x)): if (3) or (func(3)): if ((x := 3) if 1 else (y := func(x))): """ > ChrisA > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > wes.turner%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Fri Apr 27 16:08:21 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 28 Apr 2018 06:08:21 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <20180426061000.GF7400@ando.pearwood.info> <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> <20180427053837.GN7400@ando.pearwood.info> Message-ID: On Sat, Apr 28, 2018 at 6:06 AM, Wes Turner wrote: > > > On Friday, April 27, 2018, Chris Angelico wrote: >> >> On Fri, Apr 27, 2018 at 8:18 PM, Wes Turner wrote: >> > IDK, I could just be resistant to change, but this seems like something >> > that >> > will decrease readability -- and slow down code review -- without any >> > real >> > performance gain. So, while this would be useful for golfed-down (!) >> > one-liners with pyline, >> > I'm -1 on PEP 572. >> >> PEP 572 has never promised a performance gain, so "without any real >> performance gain" is hardly a criticism. >> >> > How do I step through this simple example with a debugger? >> > >> > if re.search(pat, text) as match: >> > print("Found:", match.group(0)) >> >> Step the 'if' statement. It will call re.search() and stash the result >> in 'match'. Then the cursor would be put either on the 'print' (if the >> RE matched) or on the next executable line (if it didn't). > > > Right. Pdb doesn't step through the AST branches of a line, so ternary > expressions and list comprehensions and defining variables at the end of the > line are 'debugged' after they're done. > > Similarly, code coverage is line-based; so those expressions may appear to > be covered but are not. I'm not sure I follow. In what situation would some code appear to be covered when it isn't, due to an assignment expression? ChrisA From wes.turner at gmail.com Fri Apr 27 16:24:35 2018 From: wes.turner at gmail.com (Wes Turner) Date: Fri, 27 Apr 2018 16:24:35 -0400 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <20180426061000.GF7400@ando.pearwood.info> <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> <20180427053837.GN7400@ando.pearwood.info> Message-ID: On Friday, April 27, 2018, Chris Angelico wrote: > On Sat, Apr 28, 2018 at 6:06 AM, Wes Turner wrote: > > > > > > On Friday, April 27, 2018, Chris Angelico wrote: > >> > >> On Fri, Apr 27, 2018 at 8:18 PM, Wes Turner > wrote: > >> > IDK, I could just be resistant to change, but this seems like > something > >> > that > >> > will decrease readability -- and slow down code review -- without any > >> > real > >> > performance gain. So, while this would be useful for golfed-down (!) > >> > one-liners with pyline, > >> > I'm -1 on PEP 572. > >> > >> PEP 572 has never promised a performance gain, so "without any real > >> performance gain" is hardly a criticism. > >> > >> > How do I step through this simple example with a debugger? > >> > > >> > if re.search(pat, text) as match: > >> > print("Found:", match.group(0)) > >> > >> Step the 'if' statement. It will call re.search() and stash the result > >> in 'match'. Then the cursor would be put either on the 'print' (if the > >> RE matched) or on the next executable line (if it didn't). > > > > > > Right. Pdb doesn't step through the AST branches of a line, so ternary > > expressions and list comprehensions and defining variables at the end of > the > > line are 'debugged' after they're done. > > > > Similarly, code coverage is line-based; so those expressions may appear > to > > be covered but are not. > > I'm not sure I follow. In what situation would some code appear to be > covered when it isn't, due to an assignment expression? When an assignment expression is in the else clause of a ternary expression, but the else clause does not execute because the condition is true, the assignment expression does not execute and so isn't covered. if ((1) or (x := 3)): if ((y := func(x)) if x else (x := 3)) Is this a new opportunity for misunderstanding? Assignment expressions, though they are noticeable :=, may not actually define the variable in cases where that part of the line doesn't run but reads as covered. > > ChrisA > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > wes.turner%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Fri Apr 27 16:29:54 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 28 Apr 2018 06:29:54 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <20180426061000.GF7400@ando.pearwood.info> <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> <20180427053837.GN7400@ando.pearwood.info> Message-ID: On Sat, Apr 28, 2018 at 6:24 AM, Wes Turner wrote: > > > On Friday, April 27, 2018, Chris Angelico wrote: >> >> On Sat, Apr 28, 2018 at 6:06 AM, Wes Turner wrote: >> > >> > >> > On Friday, April 27, 2018, Chris Angelico wrote: >> >> >> >> On Fri, Apr 27, 2018 at 8:18 PM, Wes Turner >> >> wrote: >> >> > IDK, I could just be resistant to change, but this seems like >> >> > something >> >> > that >> >> > will decrease readability -- and slow down code review -- without any >> >> > real >> >> > performance gain. So, while this would be useful for golfed-down (!) >> >> > one-liners with pyline, >> >> > I'm -1 on PEP 572. >> >> >> >> PEP 572 has never promised a performance gain, so "without any real >> >> performance gain" is hardly a criticism. >> >> >> >> > How do I step through this simple example with a debugger? >> >> > >> >> > if re.search(pat, text) as match: >> >> > print("Found:", match.group(0)) >> >> >> >> Step the 'if' statement. It will call re.search() and stash the result >> >> in 'match'. Then the cursor would be put either on the 'print' (if the >> >> RE matched) or on the next executable line (if it didn't). >> > >> > >> > Right. Pdb doesn't step through the AST branches of a line, so ternary >> > expressions and list comprehensions and defining variables at the end of >> > the >> > line are 'debugged' after they're done. >> > >> > Similarly, code coverage is line-based; so those expressions may appear >> > to >> > be covered but are not. >> >> I'm not sure I follow. In what situation would some code appear to be >> covered when it isn't, due to an assignment expression? > > > When an assignment expression is in the else clause of a ternary expression, > but the else clause does not execute because the condition is true, the > assignment expression does not execute and so isn't covered. > > if ((1) or (x := 3)): > if ((y := func(x)) if x else (x := 3)) > > Is this a new opportunity for misunderstanding? > > Assignment expressions, though they are noticeable :=, may not actually > define the variable in cases where that part of the line doesn't run but > reads as covered. Okay. How is this different from anything else involving if/else expressions? If your code coverage checker is unable to handle if/else, it's unable to handle it whether there's an assignment in there or not. I don't understand why people bring up all these arguments that have absolutely nothing to do with the proposal at hand. None of this has in any way changed. ChrisA From ethan at stoneleaf.us Fri Apr 27 16:37:47 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 27 Apr 2018 13:37:47 -0700 Subject: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 In-Reply-To: References: Message-ID: <5AE38A1B.3050104@stoneleaf.us> On 04/27/2018 04:34 AM, Serhiy Storchaka wrote: > 27.04.18 13:25, Nick Coghlan ????: >> and PEP 3099 (explicitly >> rejected ideas that also didn't get their own PEPs). > > "There will be no alternative binding operators such as :=." Which is why changing that requires a PEP now. -- ~Ethan~ From tim.peters at gmail.com Fri Apr 27 17:11:31 2018 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 27 Apr 2018 16:11:31 -0500 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <20180426061000.GF7400@ando.pearwood.info> <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> <20180427053837.GN7400@ando.pearwood.info> Message-ID: [Chris Angelico ] > ... > I don't understand why people bring up all these arguments that have > absolutely nothing to do with the proposal at hand. None of this has > in any way changed. That's easy: any time there's a long thread to which Guido has contributed at least twice, it will be seen as a Golden Opportunity to re-litigate every decision that's ever been made ;-) Some amount of that seems healthy to me (people are thinking about "language design" from a larger view than the proposal du jour). In this specific case, line-oriented coverage tools have missed accounting for all possible code paths since day #1; e.g., x = f() or g() You don't need to reply to messages so obviously irrelevant to the PEP unless you want to. It's not like Guido will read them and go "oh! a binding expression in a ternary conditional is a fundamentally new potential problem for a line-oriented coverage tool! that's fatal" ;-) From wes.turner at gmail.com Fri Apr 27 17:34:42 2018 From: wes.turner at gmail.com (Wes Turner) Date: Fri, 27 Apr 2018 17:34:42 -0400 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <20180426061000.GF7400@ando.pearwood.info> <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> <20180427053837.GN7400@ando.pearwood.info> Message-ID: On Friday, April 27, 2018, Tim Peters wrote: > [Chris Angelico ] > > ... > > I don't understand why people bring up all these arguments that have > > absolutely nothing to do with the proposal at hand. None of this has > > in any way changed. > > That's easy: any time there's a long thread to which Guido has > contributed at least twice, it will be seen as a Golden Opportunity to > re-litigate every decision that's ever been made ;-) > > Some amount of that seems healthy to me (people are thinking about > "language design" from a larger view than the proposal du jour). In > this specific case, line-oriented coverage tools have missed > accounting for all possible code paths since day #1; e.g., > > x = f() or g() > > You don't need to reply to messages so obviously irrelevant to the PEP > unless you want to. It's not like Guido will read them and go "oh! a > binding expression in a ternary conditional is a fundamentally new > potential problem for a line-oriented coverage tool! that's fatal" > ;-) I have shared with you the overlapping concerns about this feature proposal that I believe should be explained with DO and DON'T in the docs and/or the PEP and/or the style guide(s) for various organizations in the Pyrhon community. This feature does require additions to the style-guide(s); which is why so many have expressed concern about such a simple thing. If you want to write debuggable and coverage-testable code, do not use the assignment expression operator in ternary expressions or boolean-chained expressions. The assignment expression operator is the only way to define variables with only comprehension scope. Do not do this: x = 2 if (x == 3) or (x := 3): print(x) What do we call that mistake? > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > wes.turner%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Fri Apr 27 17:42:50 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 28 Apr 2018 07:42:50 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <20180426061000.GF7400@ando.pearwood.info> <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> <20180427053837.GN7400@ando.pearwood.info> Message-ID: On Sat, Apr 28, 2018 at 7:11 AM, Tim Peters wrote: > [Chris Angelico ] >> ... >> I don't understand why people bring up all these arguments that have >> absolutely nothing to do with the proposal at hand. None of this has >> in any way changed. > > That's easy: any time there's a long thread to which Guido has > contributed at least twice, it will be seen as a Golden Opportunity to > re-litigate every decision that's ever been made ;-) Well, now, that explains a lot! :-) > Some amount of that seems healthy to me (people are thinking about > "language design" from a larger view than the proposal du jour). In > this specific case, line-oriented coverage tools have missed > accounting for all possible code paths since day #1; e.g., > > x = f() or g() > > You don't need to reply to messages so obviously irrelevant to the PEP > unless you want to. It's not like Guido will read them and go "oh! a > binding expression in a ternary conditional is a fundamentally new > potential problem for a line-oriented coverage tool! that's fatal" > ;-) True, but sometimes it takes two or three emails before I actually understand the objection enough to know that it's actually irrelevant :| I'm going to start ignoring any message that I don't understand, in the hopes that it doesn't actually mean anything. :| ChrisA From tseaver at palladion.com Fri Apr 27 17:28:09 2018 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 27 Apr 2018 17:28:09 -0400 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <20180426061000.GF7400@ando.pearwood.info> <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> <20180427053837.GN7400@ando.pearwood.info> Message-ID: On 04/27/2018 05:11 PM, Tim Peters wrote: > In this specific case, line-oriented coverage tools have missed > accounting for all possible code paths since day #1; e.g., > > x = f() or g() > > You don't need to reply to messages so obviously irrelevant to the PEP > unless you want to. It's not like Guido will read them and go "oh! a > binding expression in a ternary conditional is a fundamentally new > potential problem for a line-oriented coverage tool! that's fatal" ;-) FWIW, Ned Batchelder's 'coverage.py' does a good job with branch coverage. I haven't seen anything in this discussion which indicates that binding expressions will change that at all. Tres. -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com From tim.peters at gmail.com Fri Apr 27 17:54:32 2018 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 27 Apr 2018 16:54:32 -0500 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <20180426061000.GF7400@ando.pearwood.info> <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> <20180427053837.GN7400@ando.pearwood.info> Message-ID: Wes, sorry, but I really don't follow what you're saying. For example, [Wes Turner ] > Do not do this: > > x = 2 > if (x == 3) or (x := 3): > print(x) > > What do we call that mistake? It displays 3 - while it appears to be silly code, there's nothing about it that's undefined. So I fail to see how showing that example anywhere would do anyone any good. You can do the same kind of thing today via, e.g., class Bindable: def __init__(self, value): self.bind(value) def bind(self, value): self.value = value return value def __bool__(self): return bool(self.value) def __eq__(self, other): return self.value == other def __str__(self): return str(self.value) Then: >>> x = Bindable(2) >>> if x == 3 or x.bind(3): ... print(x) 3 And I wouldn't put that example anywhere in any docs either ;-) From tim.peters at gmail.com Fri Apr 27 18:21:51 2018 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 27 Apr 2018 17:21:51 -0500 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <20180426061000.GF7400@ando.pearwood.info> <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> <20180427053837.GN7400@ando.pearwood.info> Message-ID: [Tres Seaver ] > FWIW, Ned Batchelder's 'coverage.py' does a good job with branch coverage. > I haven't seen anything in this discussion which indicates that binding > expressions will change that at all. I don't think you missed anything relevant either ;-) Binding operators are exactly as irrelevant to control-flow analyzers as, e.g., introducing a floor division operator (//) was. Data-flow analyzers (if there are any for Python) are a different story, since they need to be aware of all (re)binding operations - although at the byte code level, all such sites remain equally apparent (no new flavor of "store" operation is added by this PEP). From wes.turner at gmail.com Fri Apr 27 18:26:13 2018 From: wes.turner at gmail.com (Wes Turner) Date: Fri, 27 Apr 2018 18:26:13 -0400 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <20180426061000.GF7400@ando.pearwood.info> <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> <20180427053837.GN7400@ando.pearwood.info> Message-ID: It's certainly a contrived example. Actual code with such a mistake is generally far more subtle. The mistake is that it's assigning a value within a clause of a conditional that won't be evaluated. Oh well. I'll suffer the then worsened zig-zaggy eye movements in code reviews caused by defining values at the end of expressions that reference them which fit on a single line. There are a number of bad examples for style guides in these threads. := I wasn't aware of this switch, thanks! http://coverage.readthedocs.io/en/latest/branch.html coverage run --branch code.py On Friday, April 27, 2018, Tim Peters wrote: > Wes, sorry, but I really don't follow what you're saying. For example, > > [Wes Turner ] > > Do not do this: > > > > x = 2 > > if (x == 3) or (x := 3): > > print(x) > > > > What do we call that mistake? > > It displays 3 - while it appears to be silly code, there's nothing > about it that's undefined. So I fail to see how showing that example > anywhere would do anyone any good. > > You can do the same kind of thing today via, e.g., > > class Bindable: > def __init__(self, value): > self.bind(value) > > def bind(self, value): > self.value = value > return value > > def __bool__(self): > return bool(self.value) > > def __eq__(self, other): > return self.value == other > > def __str__(self): > return str(self.value) > > Then: > > >>> x = Bindable(2) > >>> if x == 3 or x.bind(3): > ... print(x) > 3 > > And I wouldn't put that example anywhere in any docs either ;-) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From v+python at g.nevcal.com Fri Apr 27 17:37:14 2018 From: v+python at g.nevcal.com (Glenn Linderman) Date: Fri, 27 Apr 2018 14:37:14 -0700 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <20180426061000.GF7400@ando.pearwood.info> <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> <20180427053837.GN7400@ando.pearwood.info> Message-ID: <73b8d454-1250-ca22-d5d9-f33ecbdd886d@g.nevcal.com> On 4/27/2018 2:11 PM, Tim Peters wrote: > That's easy: any time there's a long thread to which Guido has > contributed at least twice, it will be seen as a Golden Opportunity to > re-litigate every decision that's ever been made;-) You're getting pretty good at that QOTD thing, Tim :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Apr 27 21:23:13 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 28 Apr 2018 11:23:13 +1000 Subject: [Python-Dev] PEP 394: Allow the `python` command to not be installed (and other minor edits) In-Reply-To: References: <4c4ec942-e17b-c4f4-4b2f-459fed2c32aa@gmail.com> <85wowttuyt.fsf@benfinney.id.au> Message-ID: On 28 April 2018 at 01:37, Petr Viktorin wrote: > Hello, > After discussion on the [Pull Request], my update to PEP 394 changed scope > somewhat. The new major changes are: > > - The `python` command may not exist at all in some cases (see the PEP for > details) > - The paragraph about the anticipated future where python points to Python > 3 is removed. (We'd rather see a future where `python` doesn't exist and > one always has to specify `python2` or `python3`.) > That isn't currently *my* desired future, as I don't want to see a python3 to python4 naming transition at any point, I want a transition from python3 back to an unqualified name to accurately reflect the change in version management philosophy resulting from the extended Python 3 migration. (And then "python3" would linger solely as a backwards compatibility remnant, even if it referred to python 4+, or a switch to some kind of CalVer based versioning scheme) However, it may be possible to solve two problems at once here, as one of the issues we have in writing cross-platform instructions for both Windows and *nix systems is that "python" isn't guaranteed to be on the Windows path, but the "py" launcher *is*. This means the most robust tooling invocations on Windows are of the form "py -m pip ...", since they don't care about how your system PATH is configured. Getting not only "python", but also scripts installed with "py -m pip install --user..." to be reliably on the PATH is an arcane problem with assorted complications stemming from the differences between per-machine and per-user installations of Windows applications, and the "py -m module ..." approach avoids that mess entirely (as long as the tools of interest correctly support invocation via "-m"). The downside of those instructions is that it means we have a platform split where, depending on how your system is setup, which version of Python you're using, and whether or not you're running in a virtual environment, you need to invoke Python from the command line as one of: - python - python3 - py (more details at https://github.com/pypa/python-packaging-user-guide/issues/396 ) Thanks for everyone involved, especially Guido for pitching in with the > intended direction -- which was not clear from (or is genuinely different > from) the 7-year-old PEP! > It's a change, and not one that's been explicitly discussed with the current authors of the PEP yet (although I do see that Barry has chimed in on the PR). I think it's a reasonable way for us to go, *if* it's accompanied by an explicit decision that we want to drive cross-platform convergence around "py" as the convention for unqualified version independent access to a Python interpreter (regardless of whether or not a virtual environment is active or not) The key missing piece for doing that would be to define how we'd want a `py` launcher to work on *nix systems, and then provide that as part of CPython 3.8+ (and potentially backport it to a 3.7x maintenance release). Cheers, Nick. P.S. Note that on Windows, `py` always refers specifically to the launcher (it doesn't get shadowed in virtual environments), but the launcher itself is virtual environment aware (see https://www.python.org/dev/peps/pep-0486/ for details). -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Fri Apr 27 21:33:27 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 28 Apr 2018 13:33:27 +1200 Subject: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 In-Reply-To: References: Message-ID: <5AE3CF67.6020502@canterbury.ac.nz> Victor Stinner wrote: > In my opinion, the largest failure of Python 3 is that we failed to > provide a smooth and *slow* transition from Python 2 and Python 3. Although for some things, such as handling of non-ascii text, it's hard to see how a smooth transition *could* have been achieved. Is it a failure if we don't succeed in doing the impossible? -- Greg From guido at python.org Fri Apr 27 22:34:10 2018 From: guido at python.org (Guido van Rossum) Date: Fri, 27 Apr 2018 19:34:10 -0700 Subject: [Python-Dev] PEP 394: Allow the `python` command to not be installed (and other minor edits) In-Reply-To: References: <4c4ec942-e17b-c4f4-4b2f-459fed2c32aa@gmail.com> <85wowttuyt.fsf@benfinney.id.au> Message-ID: Um, the PEP has "Unix-Like Systems" in its heading, so discussing the Windows situation seems out of scope to me. You're one of its authors, so if you really want to keep the paragraph about the anticipated unified future we can keep it (though preferably this should be discussed in the issue, https://github.com/python/peps/pull/630). But I think this PEP is strongest in its guidelines for what distros and sysadmins should do *today*; I feel that that paragraph encourages hopes about a future that's farther away than most people care to plan, and not at all certain. On Fri, Apr 27, 2018 at 6:23 PM, Nick Coghlan wrote: > On 28 April 2018 at 01:37, Petr Viktorin wrote: > >> Hello, >> After discussion on the [Pull Request], my update to PEP 394 changed >> scope somewhat. The new major changes are: >> >> - The `python` command may not exist at all in some cases (see the PEP >> for details) >> - The paragraph about the anticipated future where python points to >> Python 3 is removed. (We'd rather see a future where `python` doesn't exist >> and one always has to specify `python2` or `python3`.) >> > > That isn't currently *my* desired future, as I don't want to see a python3 > to python4 naming transition at any point, I want a transition from python3 > back to an unqualified name to accurately reflect the change in version > management philosophy resulting from the extended Python 3 migration. (And > then "python3" would linger solely as a backwards compatibility remnant, > even if it referred to python 4+, or a switch to some kind of CalVer based > versioning scheme) > > However, it may be possible to solve two problems at once here, as one of > the issues we have in writing cross-platform instructions for both Windows > and *nix systems is that "python" isn't guaranteed to be on the Windows > path, but the "py" launcher *is*. This means the most robust tooling > invocations on Windows are of the form "py -m pip ...", since they don't > care about how your system PATH is configured. Getting not only "python", > but also scripts installed with "py -m pip install --user..." to be > reliably on the PATH is an arcane problem with assorted complications > stemming from the differences between per-machine and per-user > installations of Windows applications, and the "py -m module ..." approach > avoids that mess entirely (as long as the tools of interest correctly > support invocation via "-m"). > > The downside of those instructions is that it means we have a platform > split where, depending on how your system is setup, which version of Python > you're using, and whether or not you're running in a virtual environment, > you need to invoke Python from the command line as one of: > > - python > - python3 > - py > > (more details at https://github.com/pypa/python-packaging-user-guide/ > issues/396 ) > > Thanks for everyone involved, especially Guido for pitching in with the >> intended direction -- which was not clear from (or is genuinely different >> from) the 7-year-old PEP! >> > > It's a change, and not one that's been explicitly discussed with the > current authors of the PEP yet (although I do see that Barry has chimed in > on the PR). > > I think it's a reasonable way for us to go, *if* it's accompanied by an > explicit decision that we want to drive cross-platform convergence around > "py" as the convention for unqualified version independent access to a > Python interpreter (regardless of whether or not a virtual environment is > active or not) > > The key missing piece for doing that would be to define how we'd want a > `py` launcher to work on *nix systems, and then provide that as part of > CPython 3.8+ (and potentially backport it to a 3.7x maintenance release). > > Cheers, > Nick. > > P.S. Note that on Windows, `py` always refers specifically to the launcher > (it doesn't get shadowed in virtual environments), but the launcher itself > is virtual environment aware (see https://www.python.org/dev/ > peps/pep-0486/ for details). > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ericsnowcurrently at gmail.com Fri Apr 27 22:37:59 2018 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Fri, 27 Apr 2018 20:37:59 -0600 Subject: [Python-Dev] PEP 394: Allow the `python` command to not be installed (and other minor edits) In-Reply-To: References: <4c4ec942-e17b-c4f4-4b2f-459fed2c32aa@gmail.com> <85wowttuyt.fsf@benfinney.id.au> Message-ID: On Fri, Apr 27, 2018 at 7:23 PM, Nick Coghlan wrote: > The key missing piece for doing that would be to define how we'd want a `py` > launcher to work on *nix systems, and then provide that as part of CPython > 3.8+ (and potentially backport it to a 3.7x maintenance release). I was thinking along the same lines just today, with an email forthcoming. Now I don't have to write it. :) -eric From armin.rigo at gmail.com Fri Apr 27 23:08:13 2018 From: armin.rigo at gmail.com (Armin Rigo) Date: Sat, 28 Apr 2018 05:08:13 +0200 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: <2AF82160-56E7-4DFB-8978-AB963C46B53B@gmail.com> References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <2AF82160-56E7-4DFB-8978-AB963C46B53B@gmail.com> Message-ID: Hi, On 26 April 2018 at 07:50, Raymond Hettinger wrote: >> [Raymond Hettinger ] >>> After re-reading all the proposed code samples, I believe that >>> adopting the PEP will make the language harder to teach to people >>> who are not already software engineers. > > (...) > > Python is special, in part, because it is not one of those languages. > It has virtues that make it suitable even for elementary school children. > We can show well-written Python code to non-computer folks and walk > them through what it does without their brains melting (something I can't > do with many of the other languages I've used). There is a virtue > in encouraging simple statements that read like English sentences > organized into English-like paragraphs, presenting itself like > "executable pseudocode". I must admit that when I heard about this PEP I thought "this April 1st joke was already done long ago". I'm sorry to discover that, this time, it is not actually one. Thank you, Raymond, for an unlikely attempt at reminding people what made Python so special---in your opinion, and mine. A bient?t, Armin. From ncoghlan at gmail.com Sat Apr 28 00:08:37 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 28 Apr 2018 14:08:37 +1000 Subject: [Python-Dev] PEP 394: Allow the `python` command to not be installed (and other minor edits) In-Reply-To: References: <4c4ec942-e17b-c4f4-4b2f-459fed2c32aa@gmail.com> <85wowttuyt.fsf@benfinney.id.au> Message-ID: On 28 April 2018 at 12:34, Guido van Rossum wrote: > Um, the PEP has "Unix-Like Systems" in its heading, so discussing the > Windows situation seems out of scope to me. > Sorry, I conflated two issues there - while PEP 394 itself is specific to Unix-like systems, my thoughts on where I'd like to take it in the future are mainly informed by my experiences helping to maintain the Python Packaging User Guide these days, where the current platform dependence of "How does a user run Python, pip, and pip-installed Python tools from the command line?" is a frequent source of problems for folks just starting out. (We really don't want to be maintaining separate "Windows instructions" and "everywhere else instructions" indefinitely, but that's where we are currently) > You're one of its authors, so if you really want to keep the paragraph > about the anticipated unified future we can keep it (though preferably this > should be discussed in the issue, https://github.com/python/peps/pull/630). > But I think this PEP is strongest in its guidelines for what distros and > sysadmins should do *today*; I feel that that paragraph encourages hopes > about a future that's farther away than most people care to plan, and not > at all certain. > After reviewing the specifics of the proposed changes in the PR, I'm fine with dropping any reference to future plans for the unqualified "python" name for now (aside from clarifying how we expect it to work in virtual environments). That accurately reflects the status quo anyway - inside a virtual environment, 'python' has a clear expected meaning (the venv's Python), outside a virtual environment it's thoroughly ambiguous at best. The real deadline for figuring out the preferred post-Python-3 spelling will be once we decide to bump the major version number again, and that's at least a few years away, even if we eventually decide not to do a Python 3.10. Cheers, Nick. P.S. Historically, we hadn't been able to get much real traction with the "py-for-Unix-like-systems" idea, since it wasn't clear what problem, if any, it would solve at the *distro* level. However, we may have more luck if we're able to position it as a more forward-compatible replacement for explicit references to 'python3' that doesn't conflict with the assumption that `python` will refer specifically to `python2`. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From cesare.di.mauro at gmail.com Sat Apr 28 02:32:20 2018 From: cesare.di.mauro at gmail.com (Cesare Di Mauro) Date: Sat, 28 Apr 2018 08:32:20 +0200 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <2AF82160-56E7-4DFB-8978-AB963C46B53B@gmail.com> Message-ID: Hi, 2018-04-28 5:08 GMT+02:00 Armin Rigo : > Hi, > > On 26 April 2018 at 07:50, Raymond Hettinger > wrote: > >> [Raymond Hettinger ] > >>> After re-reading all the proposed code samples, I believe that > >>> adopting the PEP will make the language harder to teach to people > >>> who are not already software engineers. > > > > (...) > > > > Python is special, in part, because it is not one of those languages. > > It has virtues that make it suitable even for elementary school children. > > We can show well-written Python code to non-computer folks and walk > > them through what it does without their brains melting (something I can't > > do with many of the other languages I've used). There is a virtue > > in encouraging simple statements that read like English sentences > > organized into English-like paragraphs, presenting itself like > > "executable pseudocode". > > I must admit that when I heard about this PEP I thought "this April > 1st joke was already done long ago". I'm sorry to discover that, this > time, it is not actually one. Thank you, Raymond, for an unlikely > attempt at reminding people what made Python so special---in your > opinion, and mine. > > > A bient?t, > > Armin. > Same feeling here. What I really appreciate of Python from long time is its readability: the fact that usually I read the code as English-like sentences. It was nice to see the usage of the "as" keyword in the try/except construct as well as in the with one, instead of introducing another bunch of symbols which will make it more difficult to decode the meaning of the writing. Same for the "if/else" ternary operator, which I read like "[give] x if cond else y", instead of the cryptic "?:" of C-like languages. It was a nice and wise design decision. For similar reasons, I did/don't like the @ for matrix multiplication because it doesn't give me any immediately, useful information which makes it easier to decode the meaning. A "mul" binary operator would have worked better, for example. I hope that Python core developers refuse the temptation to introduce new operators using symbols for new features: it's a short way to keep backward-compatibility, for sure, but if the price to pay is the readability, then I don't think that it's worth to do it. Regarding the assignment operator, I also find it a (bad, since it's not so much readable inside expressions) duplicate of the assignment statement. To be more precise, why should we keep the latter once with the former we can do the same things (and more)? Then drop the assignment statement and just leave the operator! BTW, as a pythonist I've also felt the need to have some way to "bind" values to variables in some context, but it's pretty much related to comprehensions, for obvious reasons I think. I would have appreciated an "as" keyword, only inside such constructs, but I don't see any value extending it for any generic context, since we already have the assignment statement which works quite well and doesn't introduce nasty side-effects "ala C-like languages". So, IMO it's better to stay as we are instead of introducing another kludge to the language, if we cannot maintain a good readability. Cheers, Cesare -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Sat Apr 28 04:33:47 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 28 Apr 2018 18:33:47 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <20180426061000.GF7400@ando.pearwood.info> <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> <20180427053837.GN7400@ando.pearwood.info> Message-ID: <20180428083347.GA7288@ando.pearwood.info> On Fri, Apr 27, 2018 at 04:24:35PM -0400, Wes Turner wrote: > if ((1) or (x := 3)): > if ((y := func(x)) if x else (x := 3)) Wes, there is *absolutely nothing new* here. This sort of error is already possible in Python. Do you see a lot of code like: if (1 or sequence.append(3) or sequence[-1]): in real life? If you do, then I'm really, really sorry that you are forced to deal with such rubbish code, but honestly, the vast bulk of Python programmers do not write like that, and they won't write this either: if (1 or (x := 3)): [...] > Assignment expressions, though they are noticeable :=, may not actually > define the variable in cases where that part of the line doesn't run but > reads as covered. The same applies to any operation at all. /sarcasm I guess adding print() to the language was a mistake, because we can write rubbish code like this: if 1 or print(x): and then be confused by the fact that x doesn't get printed. /end sarcasm In another post, you insisted that we need to warn in the PEP and the docs not to do this sort of thing. Should we also go through and add these warnings to list.append, dict.update, set.add, etc? I trust that the answer to that is obviously no. And neither do we have to assume that people who use binding-expressions will lose their minds and start writing the sort of awful code that they don't write with anything else. -- Steve From mcepl at cepl.eu Sat Apr 28 04:23:38 2018 From: mcepl at cepl.eu (=?UTF-8?Q?Mat=C4=9Bj?= Cepl) Date: Sat, 28 Apr 2018 10:23:38 +0200 Subject: [Python-Dev] PEP 394: Allow the `python` command to not be installed (and other minor edits) References: <4c4ec942-e17b-c4f4-4b2f-459fed2c32aa@gmail.com> <85wowttuyt.fsf@benfinney.id.au> Message-ID: On 2018-04-28, 01:23 GMT, Nick Coghlan wrote: > That isn't currently *my* desired future, as I don't want to > see a python3 to python4 naming transition at any point, > I want a transition from python3 back to an unqualified name > to accurately reflect the change in version management > philosophy resulting from the extended Python 3 migration. > (And then "python3" would linger solely as a backwards > compatibility remnant, even if it referred to python 4+, or > a switch to some kind of CalVer based versioning scheme) +1 Mat?j -- https://matej.ceplovi.cz/blog/, Jabber: mcepl at ceplovi.cz GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 We can tell our level of faith in what God wants to do for us by our level of enthusiasm for what we want God to do for other. -- Dave Schmelzer From p.f.moore at gmail.com Sat Apr 28 05:34:42 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 28 Apr 2018 10:34:42 +0100 Subject: [Python-Dev] PEP 394: Allow the `python` command to not be installed (and other minor edits) In-Reply-To: References: <4c4ec942-e17b-c4f4-4b2f-459fed2c32aa@gmail.com> <85wowttuyt.fsf@benfinney.id.au> Message-ID: On 28 April 2018 at 05:08, Nick Coghlan wrote: > On 28 April 2018 at 12:34, Guido van Rossum wrote: >> >> Um, the PEP has "Unix-Like Systems" in its heading, so discussing the >> Windows situation seems out of scope to me. > > Sorry, I conflated two issues there - while PEP 394 itself is specific to > Unix-like systems, my thoughts on where I'd like to take it in the future > are mainly informed by my experiences helping to maintain the Python > Packaging User Guide these days, where the current platform dependence of > "How does a user run Python, pip, and pip-installed Python tools from the > command line?" is a frequent source of problems for folks just starting out. > (We really don't want to be maintaining separate "Windows instructions" and > "everywhere else instructions" indefinitely, but that's where we are > currently) Strong +1 on this. I am not a Unix user, so my opinions on PEP 394 are not relevant, but I agree with Nick that the current mess over "how to write general instructions for a newcomer on how to run Python or Python-related commands" is unsustainable. We have: 1. In a virtualenv - python 2. On windows - py (or "python" might work, if you're lucky, and it may or may not work the same as "py") 3. On Unix - python3 (or if you have a self-installed Python, or something like pyenv, "python" maybe, who knows). 4. If you're using something like Anaconda - quite likely "python", but I'm not honestly sure except on Windows And it's even worse once you get to something like pip: "pip", "pip3", "python -m pip", "py -m pip", "python3 -m pip", ... I don't want to push any particular solution here myself (my almost exclusively Windows experience is too biased for me to understand the trade-offs) but IMO, it's definitely something we need to solve. Paul From steve at holdenweb.com Sat Apr 28 05:57:04 2018 From: steve at holdenweb.com (Steve Holden) Date: Sat, 28 Apr 2018 10:57:04 +0100 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <9FA5BF19-DA14-4E29-82DE-9F1A64F16EAF@langa.pl> <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> Message-ID: On Fri, Apr 27, 2018 at 8:19 PM, Tim Peters wrote: > [Lukasz] > >> > And that *is* a thing that you will have to explain to newbies when > >> > they encounter it for the first time. > > ?Which they will presumably do either in class or by reading code. No sensible instructor or course author is going to bring name-binding expressions up until standard assignment has been thoroughly assimilated. In my own teaching experience I observed that those used to static languages took a little time to adapt to the indirection of Python's names, but not long. ? > ?[...] > > Sure. What I wrote was shorthand for what's already been covered at > length many times: what a binding expression does is "easy to > explain" GIVEN THAT someone ALREADY UNDERSTANDS how binding a name > works. The latter in fact seems difficult for a significant number of > people to learn, but it's utterly unavoidable that they learn it if > they're ever to write non-trivial Python programs. That's been true > since Python's first release. > > ?I was half-expecting someone to pop up and suggest only functional programming as a means to avoid having to teach assignment ... ? > Binding expressions would be introduced much later in any sane course. > At THAT point, for students who haven't already dropped out, the > semantics are darned-near trivial to explain: it binds the name to > the object the expression evaluates to (all of which they _already_ > understand by this point), and the value of the binding expression is > that object (the only new bit). > > Unlike as for most other operators, you don't even have to weasel-word > it to account for that a magical dunder method may change what ":=" > does. As for the "is" operator, the meaning is baked into the > language and can't be altered in the slightest. > > > > So having one more way to do assignment WILL make it harder to > > teach, not because it's that hard, but because it's one more thing to > learn. > > ?But surely that depends on HOW MUCH of the language you aim to teach. ?Over the years Python has become a much more complex language, but it has a fairly easily-identified subset that can act as a basis for building useful programs. Some instructors avoided teaching comprehensions, but a sensible course would try to ensure that students could understand the code they were most likely to encounter "in the wild." [ > ?...] > > You now, I think instructors like me are partly responsible. "is" is > rarely > > useful outside of comparing to singletons. Yet I use it early in > instruction > > to do checks on name binding and show things with mutablilty, etc.... > which > > has the unfortunate side effect of making it seem like a more common > > operator than it is. > > > ?I'd expand that to say that identity comparison is most useful for types whose instances are all unique. For other types there's the unfortunate impedance mismatch between identity and equality (which is user-definable anyway). ? > > I've even had students write code like: > > > > if x is 3: > > > > and thanks to interning, it appears to work! > > ?No, thanks to interning it DOES work. For interned values.? ?But instructors have to touch on implementation artefacts at times, and I used to find it instructive to write the same code with two different integer constants and ask why they gave different results. It certainly helped people to master the semantics of assignment (as did the phrase "Python never copies data on assignment").? Yup, that's the real problem with "is": its semantics are dead > simple, but "but under exactly what conditions are `x` and `y` bound > to the same object?" is intractable. It seems to take a long time to > get across the point, that the question itself is misguided. A full > answer requires delving into transient implementation details, which > is counterproductive because they _are_ accidents of the > implementation du jour. What questioners need to be nudged into > asking instead is for examples of when using "is" is thoroughly sane. > ?I'd argue that without some knowledge of the potential pitfalls students can't be expected to learn how to make that distinction. regards Steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Sat Apr 28 06:44:45 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 28 Apr 2018 20:44:45 +1000 Subject: [Python-Dev] Reserve ':=' for type-inferred variable initialization (was PEP 572) In-Reply-To: References: <20180427003605.GK7400@ando.pearwood.info> <20180427122846.GQ7400@ando.pearwood.info> Message-ID: <20180428104445.GW7400@ando.pearwood.info> On Fri, Apr 27, 2018 at 10:56:06PM +1000, Nick Coghlan wrote: > On 27 April 2018 at 22:28, Steven D'Aprano wrote: > > > On Fri, Apr 27, 2018 at 08:13:20AM +0200, Andrea Griffini wrote: > > > Now we got standard library features requiring type annotation > > > > We do? Did I miss them? Which std lib features are you referring to? > > > > (That's not a rhetorical question -- maybe I have missed something.) > > > > Data classes rely on the presence of annotations to spot field declarations > (it mostly doesn't care what those annotations actually say, but it does > need them to be present in order to create the list of field names). Thanks Nick. But they're not exactly used as *type* annotations. It was inevitable that something in the std lib would eventually make use of annotations, but unless they're used for type-checking, that's hardly eating away at the promise "no mandatory static typing". -- Steve From ned at nedbatchelder.com Sat Apr 28 07:18:21 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sat, 28 Apr 2018 07:18:21 -0400 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <20180426061000.GF7400@ando.pearwood.info> <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> <20180427053837.GN7400@ando.pearwood.info> Message-ID: On 4/27/18 5:28 PM, Tres Seaver wrote: > On 04/27/2018 05:11 PM, Tim Peters wrote: > >> In this specific case, line-oriented coverage tools have missed >> accounting for all possible code paths since day #1; e.g., >> >> x = f() or g() >> >> You don't need to reply to messages so obviously irrelevant to the PEP >> unless you want to. It's not like Guido will read them and go "oh! a >> binding expression in a ternary conditional is a fundamentally new >> potential problem for a line-oriented coverage tool! that's fatal" ;-) > FWIW, Ned Batchelder's 'coverage.py' does a good job with branch coverage. > I haven't seen anything in this discussion which indicates that binding > expressions will change that at all. > > Coverage.py can measure branch coverage, but it is still line-oriented.? There's no support for conditionals and branches within a line (though I've done some wicked hacks to experiment with it: https://nedbatchelder.com/blog/200804/wicked_hack_python_bytecode_tracing.html). It's entirely true that binding expressions don't change this situation at all, EXCEPT that the entire point of binding expressions is to be able to express in one statement what used to take more than one.? With binding expressions, actions may be coverage-hidden within one statement that without them would have been coverage-visible in more than one statement. I'm not sure that's an argument against binding expressions, but we should at least acknowledge that the motivation for them is to provide the option to write fewer, longer statements.? That option is not always a good idea, for a variety of reasons. --Ned. From rosuav at gmail.com Sat Apr 28 07:37:22 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 28 Apr 2018 21:37:22 +1000 Subject: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20 In-Reply-To: References: <4DA349EE-7C0C-4BD2-B2C7-A7D984869248@langa.pl> <20180426061000.GF7400@ando.pearwood.info> <04057199-F38D-47C8-A5D6-39F618824988@langa.pl> <20180427053837.GN7400@ando.pearwood.info> Message-ID: On Sat, Apr 28, 2018 at 9:18 PM, Ned Batchelder wrote: > It's entirely true that binding expressions don't change this situation at > all, EXCEPT that the entire point of binding expressions is to be able to > express in one statement what used to take more than one. With binding > expressions, actions may be coverage-hidden within one statement that > without them would have been coverage-visible in more than one statement. So far, all the examples in the PEP have the exact same coverage with and without assignment expressions, with a few exceptions where coverage is *improved* by them (where the alternative is to duplicate a function call). By combining multiple lines into one, we also ensure that all of it is executed exactly once, instead of having conditional execution. ChrisA From mcepl at cepl.eu Sat Apr 28 07:33:45 2018 From: mcepl at cepl.eu (=?UTF-8?Q?Mat=C4=9Bj?= Cepl) Date: Sat, 28 Apr 2018 13:33:45 +0200 Subject: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 References: <5AE3CF67.6020502@canterbury.ac.nz> Message-ID: On 2018-04-28, 01:33 GMT, Greg Ewing wrote: >> In my opinion, the largest failure of Python 3 is that we >> failed to provide a smooth and *slow* transition from Python >> 2 and Python 3. > > Although for some things, such as handling of non-ascii text, it's > hard to see how a smooth transition *could* have been achieved. > Is it a failure if we don't succeed in doing the impossible? +1 My experience told me that by far the biggest problem in porting M2Crypto was dealiing with complete mess which was whole string/bytes confusioin in py2k. I don't see how it could be overcame gradually. Mat?j -- https://matej.ceplovi.cz/blog/, Jabber: mcepl at ceplovi.cz GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 In those days spirits were brave, the stakes were high, men were real men, women were real women and small furry creatures from Alpha Centauri were real small furry creatures from Alpha Centauri. -- Douglas Adams From songofacandy at gmail.com Sat Apr 28 08:06:10 2018 From: songofacandy at gmail.com (INADA Naoki) Date: Sat, 28 Apr 2018 12:06:10 +0000 Subject: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 In-Reply-To: <5AE3CF67.6020502@canterbury.ac.nz> References: <5AE3CF67.6020502@canterbury.ac.nz> Message-ID: On Sat, Apr 28, 2018 at 10:36 AM Greg Ewing wrote: > Victor Stinner wrote: > > In my opinion, the largest failure of Python 3 is that we failed to > > provide a smooth and *slow* transition from Python 2 and Python 3. > Although for some things, such as handling of non-ascii text, it's > hard to see how a smooth transition *could* have been achieved. > Is it a failure if we don't succeed in doing the impossible? I don't think it's your (I'm not core developer at the time) failure. On the other hand, we should avoid many changes (e.g. bytes[index]) when doing such big change next time. -- INADA Naoki From tritium-list at sdamon.com Sat Apr 28 08:26:24 2018 From: tritium-list at sdamon.com (Alex Walters) Date: Sat, 28 Apr 2018 08:26:24 -0400 Subject: [Python-Dev] PEP 572 contradicts PEP 3099 Message-ID: <14282401d3deec$1fae2170$5f0a6450$@sdamon.com> PEP 3099 is the big list of things that will not happen in Python 3. Everything on that list is still true, 12 years after it was posted. However... "There will be no alternative binding operators such as :=." While earlier versions of PEP 572 avoided breaking this declaration, the current version does not. Now, this isn't a major issue or anything - if 572 is accepted, that section of 3099 would just need a note added (or just be removed). I don't want this message to impact the fate of 572. From solipsis at pitrou.net Sat Apr 28 11:45:24 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 28 Apr 2018 17:45:24 +0200 Subject: [Python-Dev] PEP 572: Usage of assignment expressions in C References: Message-ID: <20180428174524.7e122239@fsol> On Fri, 27 Apr 2018 09:49:33 -0700 Chris Barker wrote: > On Tue, Apr 24, 2018 at 2:21 AM, Victor Stinner wrote: > > > Even if the C language allows assignments in if, I avoid them, because > > I regularly have to debug my own code in gdb ;-) > > > > I personally haven't written a lot of C, so have no personal experience, > but if this is at all a common approach among experienced C developers, it > tells us a lot. I think it's a matter of taste and personal habit. Some people will often do it, some less. Note that C also has a tendency to make it more useful, because doesn't have exceptions, so functions need to (ab)use return values when they want to indicate an error. When you're calling such functions (for example I/O functions), you routinely have to check for special values indicating an error, so it's common to see code such as: // Read up to n bytes from file descriptor if ((bytes_read = read(fd, buf, n)) == -1) { // Error occurred while reading, do something } Regards Antoine. From solipsis at pitrou.net Sat Apr 28 11:50:02 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 28 Apr 2018 17:50:02 +0200 Subject: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 References: Message-ID: <20180428175002.62163b10@fsol> On Fri, 27 Apr 2018 14:13:32 +0200 Victor Stinner wrote: > > I don't think that having Python X.Y which introduces backward > incompatible changes is an issue by itself. We did it multiple times > during the Python 3 cycle: my PEP 446 (non-inheritable file > descriptors) and PEP 475 (retry syscalls interrupted by signals, PEP > co-written with Charles-Fran?ois Natali) introduced backward > incompatible changes in Python 3.4 and 3.5. You're comparing apples and oranges. PEP 446 and PEP 475 are only noticeable by programmers of low-level system and I/O routines (*I* haven't noticed them, for example, though I did quite a bit of network programming with Python), while the changes introduced by Python 3 affect pretty much everyone, even people who only write small simple scripts. Regards Antoine. From greg.ewing at canterbury.ac.nz Sat Apr 28 22:52:50 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 29 Apr 2018 14:52:50 +1200 Subject: [Python-Dev] PEP 572 contradicts PEP 3099 In-Reply-To: <14282401d3deec$1fae2170$5f0a6450$@sdamon.com> References: <14282401d3deec$1fae2170$5f0a6450$@sdamon.com> Message-ID: <5AE53382.2030603@canterbury.ac.nz> Alex Walters wrote: > PEP 3099 is the big list of things that will not happen in Python 3. > > "There will be no alternative binding operators such as :=." The thread referenced by that is taling about a different issue, i.e. using a different symbol to rebind names in an outer scope. -- Greg From tritium-list at sdamon.com Sat Apr 28 23:11:41 2018 From: tritium-list at sdamon.com (Alex Walters) Date: Sat, 28 Apr 2018 23:11:41 -0400 Subject: [Python-Dev] PEP 572 contradicts PEP 3099 In-Reply-To: <5AE53382.2030603@canterbury.ac.nz> References: <14282401d3deec$1fae2170$5f0a6450$@sdamon.com> <5AE53382.2030603@canterbury.ac.nz> Message-ID: <166ec801d3df67$cc4536f0$64cfa4d0$@sdamon.com> > -----Original Message----- > From: Python-Dev list=sdamon.com at python.org> On Behalf Of Greg Ewing > Sent: Saturday, April 28, 2018 10:53 PM > To: 'Python-Dev' > Subject: Re: [Python-Dev] PEP 572 contradicts PEP 3099 > > Alex Walters wrote: > > PEP 3099 is the big list of things that will not happen in Python 3. > > > > "There will be no alternative binding operators such as :=." > > The thread referenced by that is taling about a different issue, > i.e. using a different symbol to rebind names in an outer scope. > Yeah, that's not really the issue, if there is an issue - just the wording of direct assertion. I'm just adding to the list of things that need to be touched if 572 is accepted, and that includes clarifying what was meant by that. > -- > Greg > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium- > list%40sdamon.com From rosuav at gmail.com Sat Apr 28 23:42:30 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 29 Apr 2018 13:42:30 +1000 Subject: [Python-Dev] PEP 572 contradicts PEP 3099 In-Reply-To: <166ec801d3df67$cc4536f0$64cfa4d0$@sdamon.com> References: <14282401d3deec$1fae2170$5f0a6450$@sdamon.com> <5AE53382.2030603@canterbury.ac.nz> <166ec801d3df67$cc4536f0$64cfa4d0$@sdamon.com> Message-ID: On Sun, Apr 29, 2018 at 1:11 PM, Alex Walters wrote: > > >> -----Original Message----- >> From: Python-Dev > list=sdamon.com at python.org> On Behalf Of Greg Ewing >> Sent: Saturday, April 28, 2018 10:53 PM >> To: 'Python-Dev' >> Subject: Re: [Python-Dev] PEP 572 contradicts PEP 3099 >> >> Alex Walters wrote: >> > PEP 3099 is the big list of things that will not happen in Python 3. >> > >> > "There will be no alternative binding operators such as :=." >> >> The thread referenced by that is taling about a different issue, >> i.e. using a different symbol to rebind names in an outer scope. >> > > Yeah, that's not really the issue, if there is an issue - just the wording > of direct assertion. I'm just adding to the list of things that need to be > touched if 572 is accepted, and that includes clarifying what was meant by > that. I'm not sure that it matters. PEP 3099 states a number of changes that were not going to happen in Python 3000, which we now know as Python 3.0. That's still true even if Python 3.8 gains a feature. (Though most of the rejected proposals are sufficiently backward-incompatible that they're almost certainly going to remain rejected.) ChrisA From ja.py at farowl.co.uk Sun Apr 29 04:34:03 2018 From: ja.py at farowl.co.uk (Jeff Allen) Date: Sun, 29 Apr 2018 09:34:03 +0100 Subject: [Python-Dev] Every Release Can Be a Mini "Python 4000", Within Reason (was (name := expression) doesn't fit the narrative of PEP 20) In-Reply-To: <5AE2D36E.7060801@canterbury.ac.nz> References: <5AE2D36E.7060801@canterbury.ac.nz> Message-ID: <3193344c-24c1-5224-c40e-91d26687b51c@farowl.co.uk> On 27/04/2018 08:38, Greg Ewing wrote: > How would you complete the following sentence? "The ':=' > symbol is a much better symbol for assignment than '=', > because..." > ... users new to programming but with a scientific background expect '=' to be a statement of an algebraic relationship between mathematical quantities, not an instruction to the machine to do something. That's easy to answer.? (I can remember this particular light bulb moment in a fellow student, who had been using a different name in every assignment statement, and had found loops impossible to understand.) Also it frees up '=' to be used with something like its expected meaning in conditional statements, without making parsing hard/impossible. There are arguments the other way, like brevity and familiarity to other constituencies. But I feel we all know this. Having chosen to go the '=', '==' route, the cost is large to change, especially to get the other half of the benefit ('=' as a predicate). So I think the question might be who is it better for and how much do we care. And whether the days are gone when anyone learns algebra before programming. I speculate this all goes back to some pre-iteration version of FORmula TRANslation, where to its inventors '=' was definition and these really were "statements" in the normal sense of stating a truth. Jeff Allen -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sun Apr 29 06:18:17 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 29 Apr 2018 20:18:17 +1000 Subject: [Python-Dev] PEP 572 contradicts PEP 3099 In-Reply-To: <5AE53382.2030603@canterbury.ac.nz> References: <14282401d3deec$1fae2170$5f0a6450$@sdamon.com> <5AE53382.2030603@canterbury.ac.nz> Message-ID: On 29 April 2018 at 12:52, Greg Ewing wrote: > Alex Walters wrote: > >> PEP 3099 is the big list of things that will not happen in Python 3. >> >> "There will be no alternative binding operators such as :=." >> > > The thread referenced by that is taling about a different issue, > i.e. using a different symbol to rebind names in an outer scope. > Right, and that's also noted again in the accepted PEP which introduced "nonlocal" declarations: https://www.python.org/dev/peps/pep-3104/#rebinding-operator Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthony.flury at btinternet.com Sun Apr 29 07:39:18 2018 From: anthony.flury at btinternet.com (Anthony Flury) Date: Sun, 29 Apr 2018 12:39:18 +0100 Subject: [Python-Dev] Review of Pull Request 5974 please Message-ID: All, Can someone please review Pull Request 5974 on Python3.8 - the Pull request was submitted on 4th March - this pull request is associated with bpo-32933 To summarize the point of this pull request: It fixes a bug of omission within mock_open (part of unittest.mock) The functionality of mock_open enables the test code to mock a file being opened with some data which can be read. Importantly, mock_open has a read_data attrribute which can be used to specify the data to read from the file. The mocked file which is opened correctly supports file.read(), file.readlines(), file.readline(). These all make use of the read_data as expected, and the mocked file also supports being opened as a context manager. But the mock_open file does not support iteration? - so pythonic code which uses a for loop to iterate around the file content will only ever appear to iterate around an empty file, regardless of the read_data attribute when the mock_open is created So non-pythonic methods to iterate around the file contents - such as this : ??? data = opened_file.readlines() ??? for line in data: ??? ??? process_line(line) and this : ?? ??? line = opened_file.readline() ??? ?? while line: ?????????????? process_line(line) ??? ??? ??? ?? line = opened_file.readline() Can both be tested with the mocked file containing simulated data (using the read_data attribute) as expected. But this code (which by any standard is the 'correct' way to iterate around the file content of a text file): ??? for line in opened_file: ??? ??? process_line(line) Will only ever appear to iterate around an empty file when tested using mock_open. I would like this to be reviewed so it can be back-ported into Python3.7 and 3.6 if at all possible. I know that the bug has existed since the original version of mock_open, but it does seem strange that code under test which uses a pythonic code structure can't be fully tested fully using the standard library. -- Anthony Flury email : *Anthony.flury at btinternet.com* Twitter : *@TonyFlury * From yselivanov.ml at gmail.com Sun Apr 29 10:26:30 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Sun, 29 Apr 2018 14:26:30 +0000 Subject: [Python-Dev] Review of Pull Request 5974 please In-Reply-To: References: Message-ID: Reviewed. This seems to be an omission that needs to fixed, thanks for the PR! Almost good to go in 3.8. As for 3.7, this isn't a bug fix it's up to Ned if he wants to accept it. Yury On Sun, Apr 29, 2018 at 8:02 AM Anthony Flury via Python-Dev < python-dev at python.org> wrote: > All, > Can someone please review Pull Request 5974 > on Python3.8 - the Pull > request was submitted on 4th March - this pull request is associated > with bpo-32933 > To summarize the point of this pull request: > It fixes a bug of omission within mock_open > < https://docs.python.org/3/library/unittest.mock.html?highlight=mock_open#unittest.mock.mock_open > (part of unittest.mock) > The functionality of mock_open enables the test code to mock a file > being opened with some data which can be read. Importantly, mock_open > has a read_data attrribute which can be used to specify the data to read > from the file. > The mocked file which is opened correctly supports file.read(), > file.readlines(), file.readline(). These all make use of the read_data > as expected, and the mocked file also supports being opened as a context > manager. > But the mock_open file does not support iteration - so pythonic code > which uses a for loop to iterate around the file content will only ever > appear to iterate around an empty file, regardless of the read_data > attribute when the mock_open is created > So non-pythonic methods to iterate around the file contents - such as > this : > data = opened_file.readlines() > for line in data: > process_line(line) > and this : > line = opened_file.readline() > while line: > process_line(line) > line = opened_file.readline() > Can both be tested with the mocked file containing simulated data (using > the read_data attribute) as expected. > But this code (which by any standard is the 'correct' way to iterate > around the file content of a text file): > for line in opened_file: > process_line(line) > Will only ever appear to iterate around an empty file when tested using > mock_open. > I would like this to be reviewed so it can be back-ported into Python3.7 > and 3.6 if at all possible. I know that the bug has existed since the > original version of mock_open, but it does seem strange that code under > test which uses a pythonic code structure can't be fully tested fully > using the standard library. > -- > Anthony Flury > email : *Anthony.flury at btinternet.com* > Twitter : *@TonyFlury * > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com -- Yury From mmangoba at python.org Sun Apr 29 13:07:57 2018 From: mmangoba at python.org (Mark Mangoba) Date: Sun, 29 Apr 2018 10:07:57 -0700 Subject: [Python-Dev] Bugs Migration to OpenShift Message-ID: Hi All, We?re planning to finish up the bugs.python.org migration to Red Hat OpenShift by May 14th (US Pycon Sprints). For the most part everything will stay same, with the exception of cleaning up some old URL?s and redirects from the previous hosting provider: Upfront Software. We will post a more concrete timeline here by May 1st, but wanted to share this exciting news to move bugs.python.org into a more stable and optimal state. Thank you all for your patience and feedback. A special thanks to Maciej Szulik and Red Hat for helping the PSF with this project. Best regards, Mark -- Mark Mangoba | PSF IT Manager | Python Software Foundation | mmangoba at python.org | python.org | Infrastructure Staff: infrastructure-staff at python.org | GPG: 2DE4 D92B 739C 649B EBB8 CCF6 DC05 E024 5F4C A0D1 From lists at eitanadler.com Sun Apr 29 13:45:15 2018 From: lists at eitanadler.com (Eitan Adler) Date: Sun, 29 Apr 2018 10:45:15 -0700 Subject: [Python-Dev] Every Release Can Be a Mini "Python 4000", Within Reason (was (name := expression) doesn't fit the narrative of PEP 20) In-Reply-To: <3193344c-24c1-5224-c40e-91d26687b51c@farowl.co.uk> References: <5AE2D36E.7060801@canterbury.ac.nz> <3193344c-24c1-5224-c40e-91d26687b51c@farowl.co.uk> Message-ID: On 29 April 2018 at 01:34, Jeff Allen wrote: > On 27/04/2018 08:38, Greg Ewing wrote: > I speculate this all goes back to some pre-iteration version of FORmula > TRANslation, where to its inventors '=' was definition and these really were > "statements" in the normal sense of stating a truth. https://www.hillelwayne.com/post/equals-as-assignment/ -- Eitan Adler From wes.turner at gmail.com Sun Apr 29 15:36:06 2018 From: wes.turner at gmail.com (Wes Turner) Date: Sun, 29 Apr 2018 15:36:06 -0400 Subject: [Python-Dev] Every Release Can Be a Mini "Python 4000", Within Reason (was (name := expression) doesn't fit the narrative of PEP 20) In-Reply-To: References: <5AE2D36E.7060801@canterbury.ac.nz> <3193344c-24c1-5224-c40e-91d26687b51c@farowl.co.uk> Message-ID: On Sunday, April 29, 2018, Eitan Adler wrote: > On 29 April 2018 at 01:34, Jeff Allen wrote: > > On 27/04/2018 08:38, Greg Ewing wrote: > > > I speculate this all goes back to some pre-iteration version of FORmula > > TRANslation, where to its inventors '=' was definition and these really > were > > "statements" in the normal sense of stating a truth. > > https://www.hillelwayne.com/post/equals-as-assignment/ https://en.wikipedia.org/wiki/Assignment_(computer_science) C and C++ are '=' and '=='. The Sympy solver, for example, solves Eq(lhs, rhs) equations and expressions that are assumed to be equal to zero. http://docs.sympy.org/latest/tutorial/solvers.html The sage solver defines __eq__ (==) so expressions with variables produce symbolic equations and inequalities (relations). http://doc.sagemath.org/html/en/reference/calculus/sage/symbolic/relation.html PyDatalog defines __eq__ so that expressions with variables produce logic queries. https://sites.google.com/site/pydatalog/Online-datalog-tutorial The assignment Wikipedia article lists languages other than C and C++ which chose = and == for assignment and definable equality testing. https://en.wikipedia.org/wiki/Equality_(mathematics) https://en.wikipedia.org/wiki/Extensionality https://en.wikipedia.org/wiki/Logical_equality > > > -- > Eitan Adler > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > wes.turner%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From larry at hastings.org Sun Apr 29 16:45:07 2018 From: larry at hastings.org (Larry Hastings) Date: Sun, 29 Apr 2018 13:45:07 -0700 Subject: [Python-Dev] PEP 572 at the Language Summit next week Message-ID: <684f4e2a-7a0d-297c-f515-ac10567c231f@hastings.org> In case it helps, we're planning on presentations on / a discussion of PEP 572 at the 2018 Python Language Summit next Wednesday.? (I'm assuming it won't be pronounced upon before then--after all, what's the rush?)? Naturally the discussion isn't going to escape the room until it gets reported on by Jake Edge, but delegates at the Summit will hopefully emerge well-informed and comfortable with the result of the discussion. See (some of) you at the Summit! //arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Sun Apr 29 23:51:34 2018 From: guido at python.org (Guido van Rossum) Date: Sun, 29 Apr 2018 20:51:34 -0700 Subject: [Python-Dev] Every Release Can Be a Mini "Python 4000", Within Reason (was (name := expression) doesn't fit the narrative of PEP 20) In-Reply-To: References: <5AE2D36E.7060801@canterbury.ac.nz> <3193344c-24c1-5224-c40e-91d26687b51c@farowl.co.uk> Message-ID: On Sun, Apr 29, 2018 at 10:45 AM, Eitan Adler wrote: > On 29 April 2018 at 01:34, Jeff Allen wrote: > > On 27/04/2018 08:38, Greg Ewing wrote: > > > I speculate this all goes back to some pre-iteration version of FORmula > > TRANslation, where to its inventors '=' was definition and these really > were > > "statements" in the normal sense of stating a truth. > > https://www.hillelwayne.com/post/equals-as-assignment/ > That blog post was brought up before in this discussion (probably on python-ideas). I have my doubts about whether it accurately represents the historic truth though. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Mon Apr 30 02:22:39 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 30 Apr 2018 18:22:39 +1200 Subject: [Python-Dev] Every Release Can Be a Mini "Python 4000", Within Reason (was (name := expression) doesn't fit the narrative of PEP 20) In-Reply-To: <3193344c-24c1-5224-c40e-91d26687b51c@farowl.co.uk> References: <5AE2D36E.7060801@canterbury.ac.nz> <3193344c-24c1-5224-c40e-91d26687b51c@farowl.co.uk> Message-ID: <5AE6B62F.9050902@canterbury.ac.nz> Jeff Allen wrote: > I speculate this all goes back to some pre-iteration version of FORmula > TRANslation, where to its inventors '=' was definition and these really > were "statements" in the normal sense of stating a truth. Yeah, also the earliest FORTRAN didn't even *have* comparison operators. A conditional branch was something like IF (X-Y) 10, 20, 30 going three different ways depending on whether X-Y was negative, zero or positive. Then when comparison operators were added, a lot of people didn't have "<" and ">" characters available to them, so FORTRAN used ".EQ.", ".LT.", ".GT." etc. instead. We're actually quite spoiled with our "==" operator! -- Greg From greg.ewing at canterbury.ac.nz Mon Apr 30 02:30:32 2018 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 30 Apr 2018 18:30:32 +1200 Subject: [Python-Dev] PEP 572 contradicts PEP 3099 In-Reply-To: References: <14282401d3deec$1fae2170$5f0a6450$@sdamon.com> <5AE53382.2030603@canterbury.ac.nz> Message-ID: <5AE6B808.7070502@canterbury.ac.nz> Nick Coghlan wrote: > On 29 April 2018 at 12:52, Greg Ewing > wrote: > > Alex Walters wrote: > > PEP 3099 is the big list of things that will not happen in Python 3. > > "There will be no alternative binding operators such as :=." > > The thread referenced by that is taling about a different issue, > i.e. using a different symbol to rebind names in an outer scope. > > Right, and that's also noted again in the accepted PEP which introduced > "nonlocal" declarations: > https://www.python.org/dev/peps/pep-3104/#rebinding-operator Perhaps PEP 3099 could be amended to say "no alternative binding operators for the purpose of distinguishing local and nonlocal bindings." -- Greg From vstinner at redhat.com Mon Apr 30 03:38:45 2018 From: vstinner at redhat.com (Victor Stinner) Date: Mon, 30 Apr 2018 09:38:45 +0200 Subject: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3 In-Reply-To: <5AE3CF67.6020502@canterbury.ac.nz> References: <5AE3CF67.6020502@canterbury.ac.nz> Message-ID: 2018-04-28 3:33 GMT+02:00 Greg Ewing : > Victor Stinner wrote: >> >> In my opinion, the largest failure of Python 3 is that we failed to >> provide a smooth and *slow* transition from Python 2 and Python 3. > > Although for some things, such as handling of non-ascii text, it's > hard to see how a smooth transition *could* have been achieved. > Is it a failure if we don't succeed in doing the impossible? Technically, it is easy to add an option to Python 2 to raise an exception on str+unicode and str < unicode. You can imagine the same option or a different one to really get the bytes type of Python 3 (b'abc'[0] returns 97). Such option would ease a lot to port code to Python 3, since you only have to care of bytes vs Unicode issue. You don't have to worry about the looooong list of other Python 3 changes (like the new stdlib names...). Note: Python 3 Unicode is stricter in other ways, like UTF-8 reject lone surrogates. But I don't think that it matters for most users, and technically it would also be possible to add an option to Python 2 to get the new behaviour. Moreover, Python 2 *already* has an option to switch to the new division mode: $ python2 -Q new -c 'print(1/2)' 0.5 I'm proposing that a backward incompatible change must always be prepared in the previous release. Victor From tjreedy at udel.edu Mon Apr 30 03:40:52 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 30 Apr 2018 03:40:52 -0400 Subject: [Python-Dev] Every Release Can Be a Mini "Python 4000", Within Reason (was (name := expression) doesn't fit the narrative of PEP 20) In-Reply-To: References: <5AE2D36E.7060801@canterbury.ac.nz> <3193344c-24c1-5224-c40e-91d26687b51c@farowl.co.uk> Message-ID: On 4/29/2018 11:51 PM, Guido van Rossum wrote: > On Sun, Apr 29, 2018 at 10:45 AM, Eitan Adler > wrote: > > On 29 April 2018 at 01:34, Jeff Allen > wrote: > > On 27/04/2018 08:38, Greg Ewing wrote: > > > I speculate this all goes back to some pre-iteration version of FORmula > > TRANslation, where to its inventors '=' was definition and these really were > > "statements" in the normal sense of stating a truth. > > https://www.hillelwayne.com/post/equals-as-assignment/ > > > > That blog post was brought up before in this discussion (probably on > python-ideas). I have my doubts about whether it accurately represents > the historic truth though. It is woefully incomplete in omitting the common usage of = to mean 'equals' both as statement (comparison) and command (assignment) in both English and math. I don't have any math books that I know of that predate computers, but I suspect the usage is not new. The pre-C computer language history has a gaping hole: BASIC, which uses = for both assignment and comparison, was released May 1, 1964. I don't believe the syntax allowed any ambiguity as to the meaning of each occurrence. To me, it is the use of anything else that needs explaining. -- Terry Jan Reedy From vstinner at redhat.com Mon Apr 30 03:54:43 2018 From: vstinner at redhat.com (Victor Stinner) Date: Mon, 30 Apr 2018 09:54:43 +0200 Subject: [Python-Dev] Bugs Migration to OpenShift In-Reply-To: References: Message-ID: Does it mean that the final switch will happen during the sprints? Would it be possible to do it before or after? If bugs.python.org doesn't work during the sprint, it will be much harder to contribute to CPython during the sprints. (If I misunderstood, ignore my message :-)) Victor 2018-04-29 19:07 GMT+02:00 Mark Mangoba : > Hi All, > > We?re planning to finish up the bugs.python.org migration to Red Hat > OpenShift by May 14th (US Pycon Sprints). For the most part > everything will stay same, with the exception of cleaning up some old > URL?s and redirects from the previous hosting provider: Upfront > Software. > > We will post a more concrete timeline here by May 1st, but wanted to > share this exciting news to move bugs.python.org into a more stable > and optimal state. > > Thank you all for your patience and feedback. A special thanks to > Maciej Szulik and Red Hat for helping the PSF with this project. > > Best regards, > Mark > > -- > Mark Mangoba | PSF IT Manager | Python Software Foundation | > mmangoba at python.org | python.org | Infrastructure Staff: > infrastructure-staff at python.org | GPG: 2DE4 D92B 739C 649B EBB8 CCF6 > DC05 E024 5F4C A0D1 > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com From vstinner at redhat.com Mon Apr 30 04:02:53 2018 From: vstinner at redhat.com (Victor Stinner) Date: Mon, 30 Apr 2018 10:02:53 +0200 Subject: [Python-Dev] PEP 572: Usage of assignment expressions in C In-Reply-To: <20180428174524.7e122239@fsol> References: <20180428174524.7e122239@fsol> Message-ID: 2018-04-28 17:45 GMT+02:00 Antoine Pitrou : > // Read up to n bytes from file descriptor > if ((bytes_read = read(fd, buf, n)) == -1) { > // Error occurred while reading, do something > } About C, there is a popular coding style (not used by our PEP 7) for comparison: if (-1 == n) ... The advantage is to prevent typo mistakes because "-1 = n" is a syntax error. Victor From vstinner at redhat.com Mon Apr 30 04:11:15 2018 From: vstinner at redhat.com (Victor Stinner) Date: Mon, 30 Apr 2018 10:11:15 +0200 Subject: [Python-Dev] PEP 394: Allow the `python` command to not be installed (and other minor edits) In-Reply-To: References: <4c4ec942-e17b-c4f4-4b2f-459fed2c32aa@gmail.com> <85wowttuyt.fsf@benfinney.id.au> Message-ID: 2018-04-27 17:37 GMT+02:00 Petr Viktorin : > (...) > - The paragraph about the anticipated future where python points to Python 3 > is removed. Instead of editing old PEPs, would it make sense to write a new one which replaces the old one? The PEP 394 has been written in 2011 and accepted in 2012. Editing an accepted PEP makes it harder to dig into the history of PEPs. I know that the PEP 8 is updated regularly, so it's not a requirement, just a proposal for the special PEP 394. Victor From ncoghlan at gmail.com Mon Apr 30 09:36:34 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 30 Apr 2018 23:36:34 +1000 Subject: [Python-Dev] PEP 394: Allow the `python` command to not be installed (and other minor edits) In-Reply-To: References: <4c4ec942-e17b-c4f4-4b2f-459fed2c32aa@gmail.com> <85wowttuyt.fsf@benfinney.id.au> Message-ID: On 30 April 2018 at 18:11, Victor Stinner wrote: > 2018-04-27 17:37 GMT+02:00 Petr Viktorin : > > (...) > > - The paragraph about the anticipated future where python points to > Python 3 > > is removed. > > Instead of editing old PEPs, would it make sense to write a new one > which replaces the old one? > > The PEP 394 has been written in 2011 and accepted in 2012. Editing an > accepted PEP makes it harder to dig into the history of PEPs. > > I know that the PEP 8 is updated regularly, so it's not a requirement, > just a proposal for the special PEP 394. > We edit PEP 394 in place for the same reason we edit PEP 8 in place: so people have a consistent place to get our current recommendations. It isn't like a packaging interoperability spec or the Python language & library specs, where folks regularly need to know what changed between particular revisions. If folks really want to know how the recommendations have changed over time, then browsing https://github.com/python/peps/commits/master/pep-0394.txt isn't overly difficult. (That said, I also wouldn't be opposed to adding an inline change log to the PEP, since it helps highlights the cases where the recommendations *have* changed, which can be helpful for folks that were already familiar with the older versions) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at hotpy.org Mon Apr 30 09:38:58 2018 From: mark at hotpy.org (Mark Shannon) Date: Mon, 30 Apr 2018 14:38:58 +0100 Subject: [Python-Dev] PEP 575: Unifying function/method classes In-Reply-To: <5ACF8552.6020607@UGent.be> References: <5ACF8552.6020607@UGent.be> Message-ID: On 12/04/18 17:12, Jeroen Demeyer wrote: > Dear Python developers, > > I would like to request a review of PEP 575, which is about changing the > classes used for built-in functions and Python functions and methods. > The text of the PEP can be found at The motivation of PEP 575 is to allow introspection of built-in functions and to allow functions implemented in Python to be re-implemented in C. These are excellent goals. The PEP then elaborates a complex class hierarchy, and various extensions to the C API. This adds a considerable maintainance burden and restricts future changes and optimisations to CPython. While a unified *interface* makes sense, a unified class hierarchy and implementation, IMO, do not. The hierarchy also seems to force classes that are dissimilar to share a common base-class. Bound-methods may be callables, but they are not functions, they are a pair of a function and a "self" object. As the PEP points out, Cython functions are able to mimic Python functions, why not do the same for CPython builtin-functions? As an aside, rather than unifying the classes of all non-class callables, CPython's builtin-function class could be split in two. Currently it is both a bound-method and a function. The name 'builtin_function_or_method' is a give away :) Consider the most common "function" and "method" classes: >>> class C: ... def f(self): pass # "functions" >>> type(C.f) >>> type(len) >>> type(list.append) >>> type(int.__add__) # "bound-methods" >>> type(C().f) >>> type([].append) >>> type(1 .__add__) IMO, there are so many versions of "function" and "bound-method", that a unified class hierarchy and the resulting restriction to the implementation will make implementing a unified interface harder, not easier. For "functions", all that is needed is to specify an interface, say a single property "__signature__". Then all that a class that wants to be a "function" need do is have a "__signature__" property and be callable. For "bound-methods", we should reuse the interface of 'method'; two properties, "__func__" and "__self__". Cheers, Mark. From ncoghlan at gmail.com Mon Apr 30 09:49:43 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 30 Apr 2018 23:49:43 +1000 Subject: [Python-Dev] PEP 575: Unifying function/method classes In-Reply-To: References: <5ACF8552.6020607@UGent.be> Message-ID: On 30 April 2018 at 23:38, Mark Shannon wrote: > > On 12/04/18 17:12, Jeroen Demeyer wrote: > >> Dear Python developers, >> >> I would like to request a review of PEP 575, which is about changing the >> classes used for built-in functions and Python functions and methods. The >> text of the PEP can be found at >> > > > The motivation of PEP 575 is to allow introspection of built-in functions > and to allow functions implemented in Python to be re-implemented in C. > > These are excellent goals. > That summary misses the 3rd goal, which is the one that answers your other questions: to allow 3rd party extension modules access to the hot paths in CPython that are currently restricted to true built-in and Python native functions (without making those hot paths measurably slower). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at hotpy.org Mon Apr 30 10:30:54 2018 From: mark at hotpy.org (Mark Shannon) Date: Mon, 30 Apr 2018 15:30:54 +0100 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: Hi, On 17/04/18 08:46, Chris Angelico wrote: > Having survived four rounds in the boxing ring at python-ideas, PEP > 572 is now ready to enter the arena of python-dev. I'm very strongly opposed to this PEP. Would Python be better with two subtly different assignment operators? The answer of "no" seems self evident to me. Do we need an assignment expression at all (regardless of the chosen operator)? I think we do not. Assignment is clear at the moment largely because of the context; it can only occur at the statement level. Consequently, assignment and keyword arguments are never confused despite have the same form `name = expr` List comprehensions ------------------- The PEP uses the term "simplifying" when it really means "shortening". One example is stuff = [[y := f(x), x/y] for x in range(5)] as a simplification of stuff = [(lambda y: [y,x/y])(f(x)) for x in range(5)] IMO, the "simplest" form of the above is the named helper function. def meaningful_name(x): t = f(x) return t, x/t [meaningful_name(i) for i in range(5)] Is longer, but much simpler to understand. I am also concerned that the ability to put assignments anywhere allows weirdnesses like these: try: ... except (x := Exception) as x: ... with (x: = open(...)) as x: ... def do_things(fire_missiles=False, plant_flowers=False): ... do_things(plant_flowers:=True) # whoops! It is easy to say "don't do that", but why allow it in the first place? Cheers, Mark. From rosuav at gmail.com Mon Apr 30 11:30:37 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 1 May 2018 01:30:37 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On Tue, May 1, 2018 at 12:30 AM, Mark Shannon wrote: > List comprehensions > ------------------- > The PEP uses the term "simplifying" when it really means "shortening". > One example is > stuff = [[y := f(x), x/y] for x in range(5)] > as a simplification of > stuff = [(lambda y: [y,x/y])(f(x)) for x in range(5)] Now try to craft the equivalent that captures the condition in an if: results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0] Do that one with a lambda function. > IMO, the "simplest" form of the above is the named helper function. > > def meaningful_name(x): > t = f(x) > return t, x/t > > [meaningful_name(i) for i in range(5)] > > Is longer, but much simpler to understand. Okay, but what if there is no meaningful name? It's easy to say "pick a meaningful name". It's much harder to come up with an actual name that is sufficiently meaningful that a reader need not go look at the definition of the function. > I am also concerned that the ability to put assignments anywhere > allows weirdnesses like these: > > try: > ... > except (x := Exception) as x: > ... > > with (x: = open(...)) as x: > ... We've been over this argument plenty, and I'm not going to rehash it. > def do_things(fire_missiles=False, plant_flowers=False): ... > do_things(plant_flowers:=True) # whoops! If you want your API to be keyword-only, make it keyword-only. If you want a linter that recognizes unused variables, get a linter that recognizes unused variables. Neither of these is the fault of the proposed syntax; you could just as easily write this: do_things(plant_flowers==True) but we don't see myriad reports of people typing too many characters and blaming the language. ChrisA From paddy3118 at gmail.com Mon Apr 30 03:09:35 2018 From: paddy3118 at gmail.com (Paddy McCarthy) Date: Mon, 30 Apr 2018 08:09:35 +0100 Subject: [Python-Dev] PEP 572: A backward step in readability Message-ID: The PEP s section on Frequently raised objections includes: (https://www.python.org/dev/peps/pep-0572/#this-could-be-used-to-create-ugly-code) > This could be used to create ugly code! > > So can anything else. This is a tool, and it is up to the programmer to use it where > it makes sense, and not use it where superior constructs can be used. The above is so dismissive of a very real problem. * Python makes syntax decisions to make it easier to read. * One of the first, and enduring ones was "No assignment within expressions". * I had both seen, and debugged problems in the C-language caused by assignment expressions; and was glad to be rid of them in Python. Assignment within expressions was, I think, difficult to *teach* in C. And will become something error-prone to *learn* in Python. The Python community has grown over the decades, and we have attracted community members who want to help and are motivated, but Python is not a tick-list of features. A PEP that can detract from readability; *readability*, a central tenet of Python, should be rejected, (on principle!), when such objections are treated so dismissively. From antoine at python.org Mon Apr 30 11:45:46 2018 From: antoine at python.org (Antoine Pitrou) Date: Mon, 30 Apr 2018 17:45:46 +0200 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: Le 30/04/2018 ? 17:30, Chris Angelico a ?crit?: > >> def do_things(fire_missiles=False, plant_flowers=False): ... >> do_things(plant_flowers:=True) # whoops! > > If you want your API to be keyword-only, make it keyword-only. If you > want a linter that recognizes unused variables, get a linter that > recognizes unused variables. Neither of these is the fault of the > proposed syntax; you could just as easily write this: > > do_things(plant_flowers==True) Unless you have a `plant_flowers` variable already defined, this will raise a NameError, not plant a silent bug. Regards Antoine. From J.Demeyer at UGent.be Mon Apr 30 11:55:37 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Mon, 30 Apr 2018 17:55:37 +0200 Subject: [Python-Dev] PEP 575: Unifying function/method classes In-Reply-To: <43fe216c540847aa838bba67987b9657@xmail101.UGent.be> References: <5ACF8552.6020607@UGent.be> <43fe216c540847aa838bba67987b9657@xmail101.UGent.be> Message-ID: <5AE73C79.6090409@UGent.be> On 2018-04-30 15:38, Mark Shannon wrote: > While a unified *interface* makes sense, a unified class hierarchy and > implementation, IMO, do not. The main reason for the common base class is performance: in the bytecode interpreter, when we call an object, CPython currently has a special case for calling Python functions, a special case for calling methods, a special case for calling method descriptors, a special case for calling built-in functions. By introducing a common base class, we reduce the number of special cases. Second, we allow using this fast path for custom classes. With PEP 575, it is possible to create new classes with the same __call__ performance as the current built-in function class. > Bound-methods may be callables, but they are not functions, they are a > pair of a function and a "self" object. From the Python language point of view, that may be true but that's not how you want to implement methods. When I write a method in C, I want that it can be called either as unbound method or as bound method: the C code shouldn't see the difference between the calls X.foo(obj) or obj.foo(). And you want both calls to be equally fast, so you don't want that the bound method just wraps the unbound method. For this reason, it makes sense to unify functions and methods. > IMO, there are so many versions of "function" and "bound-method", that a > unified class hierarchy and the resulting restriction to the > implementation will make implementing a unified interface harder, not > easier. PEP 575 does not add any restrictions: I never claimed that all callables should inherit from base_function. Regardless, why would the common base class add restrictions? You can still add attributes and customize whatever you want in subclasses. Jeroen. From steve at pearwood.info Mon Apr 30 12:37:58 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 1 May 2018 02:37:58 +1000 Subject: [Python-Dev] PEP 572: A backward step in readability In-Reply-To: References: Message-ID: <20180430163758.GF7400@ando.pearwood.info> On Mon, Apr 30, 2018 at 08:09:35AM +0100, Paddy McCarthy wrote: [...] > A PEP that can detract from readability; *readability*, a central > tenet of Python, should > be rejected, (on principle!), when such objections are treated so dismissively. Unless you have an objective measurement of readability, that objection is mere subjective personal preference, and not one that everyone agrees with. I for one think that used wisely, binding expressions will be *more* readable than the alternatives. (Even though := is not my preferred syntax.) The "not readable" objection has been made, extremely vehemently, against nearly all major syntax changes to Python: - comprehensions? not readable, easy to abuse, hard for beginners to comprehend; - decorators? not readable, looks like Perl with the arbitrary use of @ symbol, hard to understand; - ternary if operator? not readable, doesn't look enough like C, weird order; - augmented assignments += etc -- not readable, too terse, requires the reader to be familiar with C. I was guilty of making that last one. And if I had been around for the debate over decorators, I probably would have hated them too. But all four additions to the syntax have been *great* for the language, despite the nay-sayers. I still know people with many years experience in Python who say they don't get comprehensions, and of course it is true that they are hard for beginners to get. But the advantages from comprehensions is immeasurably greater than the disadvantages. Will this PEP be like comprehensions or decorators, and completely change the way we write Python code (for the better)? I doubt it. But I expect it will be like augmented assignment and the ternary if: it will make certain kinds of code more pleasurable to write *and read*, and when it doesn't, people won't use it. -- Steve From p.f.moore at gmail.com Mon Apr 30 12:53:59 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 30 Apr 2018 17:53:59 +0100 Subject: [Python-Dev] PEP 572: A backward step in readability In-Reply-To: <20180430163758.GF7400@ando.pearwood.info> References: <20180430163758.GF7400@ando.pearwood.info> Message-ID: On 30 April 2018 at 17:37, Steven D'Aprano wrote: > On Mon, Apr 30, 2018 at 08:09:35AM +0100, Paddy McCarthy wrote: > [...] >> A PEP that can detract from readability; *readability*, a central >> tenet of Python, should >> be rejected, (on principle!), when such objections are treated so dismissively. > > Unless you have an objective measurement of readability, that objection > is mere subjective personal preference, and not one that everyone agrees > with. I for one think that used wisely, binding expressions will be > *more* readable than the alternatives. (Even though := is not my > preferred syntax.) On the other hand, the PEP doesn't do much to address the various claims of readability issues. Whether they are subjective or not, well-founded or not, the PEP does (in my view) come across as unnecessarily dismissive of the question of readability. I suspect that's largely due to Chris being extremely tired of having the same arguments over and over again, and I can understand that. However, I think it could be covered more completely - the remainder of your post is exactly the sort of response that would be useful to have recorded. But one way or another, I think people will vote on the PEP based on their (subjective) views, and so readability will get factored into the overall response, one way or another. To what extent that response affects Guido's final decision (I can't see him delegating on this one!) remains to be seen, and honestly, I'm willing to trust Guido's intuition. Paul From yselivanov.ml at gmail.com Mon Apr 30 12:53:32 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Mon, 30 Apr 2018 16:53:32 +0000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On Mon, Apr 30, 2018 at 11:32 AM Chris Angelico wrote: > On Tue, May 1, 2018 at 12:30 AM, Mark Shannon wrote: > > List comprehensions > > ------------------- > > The PEP uses the term "simplifying" when it really means "shortening". > > One example is > > stuff = [[y := f(x), x/y] for x in range(5)] > > as a simplification of > > stuff = [(lambda y: [y,x/y])(f(x)) for x in range(5)] > Now try to craft the equivalent that captures the condition in an if: > results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0] Easy: results = [] for x in input_data: y = f(x) if y > 0: results.append((x, y, x/y)) Longer, but way more readable and debuggable if you're into that. This has worked for us many years and only a handful of people complained about this. OTOH, I see plenty of people complaining that nested list comprehensions are hard to read. In my own code reviews I ask people to avoid using complex comprehensions all the time. > Do that one with a lambda function. Why would I? Is using lambda functions mandatory? > > IMO, the "simplest" form of the above is the named helper function. > > > > def meaningful_name(x): > > t = f(x) > > return t, x/t > > > > [meaningful_name(i) for i in range(5)] > > > > Is longer, but much simpler to understand. > Okay, but what if there is no meaningful name? It's easy to say "pick > a meaningful name". It's much harder to come up with an actual name > that is sufficiently meaningful that a reader need not go look at the > definition of the function. That's a weird argument, Chris :-) If `f(x)` has no meaningful name, then *what* is the result of the comprehension? Perhaps some meaningless data? ;) > > I am also concerned that the ability to put assignments anywhere > > allows weirdnesses like these: > > > > try: > > ... > > except (x := Exception) as x: > > ... > > > > with (x: = open(...)) as x: > > ... > We've been over this argument plenty, and I'm not going to rehash it. Hand-waving the question the way you do simply alienates more core devs to the PEP. And PEP 572 hand-waves a lot of questions and concerns. Asking people to dig for answers in 700+ emails about the PEP is a bit too much, don't you agree? I think it's PEP's author responsibility to address questions right in their PEP. > > def do_things(fire_missiles=False, plant_flowers=False): ... > > do_things(plant_flowers:=True) # whoops! > If you want your API to be keyword-only, make it keyword-only. If you Another hand-waving. Should we deprecate passing arguments by name if their corresponding parameters are not keyword-only? Mark shows another potential confusion between '=' and ':=' that people will have, and it's an interesting one. > want a linter that recognizes unused variables, get a linter that > recognizes unused variables. Many want Python to be readable and writeable without linters. > Neither of these is the fault of the > proposed syntax; you could just as easily write this: > do_things(plant_flowers==True) > but we don't see myriad reports of people typing too many characters > and blaming the language. Strange. I see people who struggle to format their code properly or use the language properly *every day* ;) Yury From rosuav at gmail.com Mon Apr 30 13:01:19 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 1 May 2018 03:01:19 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On Tue, May 1, 2018 at 2:53 AM, Yury Selivanov wrote: > On Mon, Apr 30, 2018 at 11:32 AM Chris Angelico wrote: > >> On Tue, May 1, 2018 at 12:30 AM, Mark Shannon wrote: >> > List comprehensions >> > ------------------- >> > The PEP uses the term "simplifying" when it really means "shortening". >> > One example is >> > stuff = [[y := f(x), x/y] for x in range(5)] >> > as a simplification of >> > stuff = [(lambda y: [y,x/y])(f(x)) for x in range(5)] > >> Now try to craft the equivalent that captures the condition in an if: > >> results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0] > > Easy: > > results = [] > for x in input_data: > y = f(x) > if y > 0: > results.append((x, y, x/y)) > > Longer, but way more readable and debuggable if you're into that. This has > worked for us many years and only a handful of people complained about > this. > > OTOH, I see plenty of people complaining that nested list comprehensions > are hard to read. In my own code reviews I ask people to avoid using > complex comprehensions all the time. > >> Do that one with a lambda function. > > Why would I? Is using lambda functions mandatory? The claim was that assignment expressions were nothing more than a shorthand for lambda functions. You can't rewrite my example with a lambda function, ergo assignment expressions are not a shorthand for lambda functions. Do you agree? >> > IMO, the "simplest" form of the above is the named helper function. >> > >> > def meaningful_name(x): >> > t = f(x) >> > return t, x/t >> > >> > [meaningful_name(i) for i in range(5)] >> > >> > Is longer, but much simpler to understand. > >> Okay, but what if there is no meaningful name? It's easy to say "pick >> a meaningful name". It's much harder to come up with an actual name >> that is sufficiently meaningful that a reader need not go look at the >> definition of the function. > > That's a weird argument, Chris :-) > > If `f(x)` has no meaningful name, then *what* is the result of the > comprehension? Perhaps some meaningless data? ;) f(x) might have side effects. Can you give a meaningful name to the trivial helper function? Not every trivial helper can actually have a name that saves people from having to read the body of the function. >> > I am also concerned that the ability to put assignments anywhere >> > allows weirdnesses like these: >> > >> > try: >> > ... >> > except (x := Exception) as x: >> > ... >> > >> > with (x: = open(...)) as x: >> > ... > >> We've been over this argument plenty, and I'm not going to rehash it. > > Hand-waving the question the way you do simply alienates more core devs to > the PEP. And PEP 572 hand-waves a lot of questions and concerns. Asking > people to dig for answers in 700+ emails about the PEP is a bit too much, > don't you agree? > > I think it's PEP's author responsibility to address questions right in > their PEP. If I answer every question, I make that number into 800+, then 900+, then 1000+. If I don't, I'm alienating everyone by being dismissive. If every question is answered in the PEP, the document itself becomes so long that nobody reads it. Damned if I do, damned if I don't. Got any alternative suggestions? >> > def do_things(fire_missiles=False, plant_flowers=False): ... >> > do_things(plant_flowers:=True) # whoops! > >> If you want your API to be keyword-only, make it keyword-only. If you > > Another hand-waving. Should we deprecate passing arguments by name if > their corresponding parameters are not keyword-only? > > Mark shows another potential confusion between '=' and ':=' that people > will have, and it's an interesting one. A very rare one compared to the confusions that we already have with '=' and '=='. And this is another argument that we've been over, multiple times. >> want a linter that recognizes unused variables, get a linter that >> recognizes unused variables. > > Many want Python to be readable and writeable without linters. And it will be. But there are going to be certain types of bug that you won't catch as quickly. You can't use language syntax to catch every bug. That's provably impossible. >> Neither of these is the fault of the >> proposed syntax; you could just as easily write this: > >> do_things(plant_flowers==True) > >> but we don't see myriad reports of people typing too many characters >> and blaming the language. > > Strange. I see people who struggle to format their code properly or use the > language properly *every day* ;) And do they blame the language for having a comparison operator that is so easy to type? Or do they fix their bugs and move on? Again, language syntax is not the solution to bugs. ChrisA From yselivanov.ml at gmail.com Mon Apr 30 13:34:49 2018 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Mon, 30 Apr 2018 17:34:49 +0000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: On Mon, Apr 30, 2018 at 1:03 PM Chris Angelico wrote: > > That's a weird argument, Chris :-) > > > > If `f(x)` has no meaningful name, then *what* is the result of the > > comprehension? Perhaps some meaningless data? ;) > f(x) might have side effects. Can you give a meaningful name to the > trivial helper function? I don't understand your question. How is `f(x)` having side effects or not having them is relevant to the discussion? Does ':=' work only with pure functions? > Not every trivial helper can actually have a > name that saves people from having to read the body of the function. I don't understand this argument either, sorry. > >> We've been over this argument plenty, and I'm not going to rehash it. > > > > Hand-waving the question the way you do simply alienates more core devs to > > the PEP. And PEP 572 hand-waves a lot of questions and concerns. Asking > > people to dig for answers in 700+ emails about the PEP is a bit too much, > > don't you agree? > > > > I think it's PEP's author responsibility to address questions right in > > their PEP. > If I answer every question, I make that number into 800+, then 900+, > then 1000+. If I don't, I'm alienating everyone by being dismissive. > If every question is answered in the PEP, the document itself becomes > so long that nobody reads it. Damned if I do, damned if I don't. Got > any alternative suggestions? IMO, big part of why that we have 100s of emails is because people are very concerned with readability. The PEP just hand-waives the question entirely, instead of listing good and realistic examples of code, as well as listing bad examples. So that, you know, people could compare them and understand *both* pros and cons. Instead we have a few very questionable examples in the PEP that most people don't like at all. Moreover, half of the PEP is devoted to fixing comprehensions scoping, which is almost an orthogonal problem to adding a new syntax. So my suggestion remains to continue working on the PEP, improving it and making it more comprehensive. You're free to ignore this advice, but don't be surprised that you see new emails about what ':=' does to code readability (with the same arguments). PEP 572 proponents answering to every email with the same dismissive template doesn't help either. > >> > def do_things(fire_missiles=False, plant_flowers=False): ... > >> > do_things(plant_flowers:=True) # whoops! > > > >> If you want your API to be keyword-only, make it keyword-only. If you > > > > Another hand-waving. Should we deprecate passing arguments by name if > > their corresponding parameters are not keyword-only? > > > > Mark shows another potential confusion between '=' and ':=' that people > > will have, and it's an interesting one. > A very rare one compared to the confusions that we already have with > '=' and '=='. And this is another argument that we've been over, > multiple times. How do you know if it's rare or not? '=' is used to assign, ':=' is used to assign, '==' is used to compare. I can easily imagine people being confused why '=' works for setting an argument, and why ':=' doesn't. Let's agree to disagree on this one :) > > Strange. I see people who struggle to format their code properly or use the > > language properly *every day* ;) > And do they blame the language for having a comparison operator that > is so easy to type? Or do they fix their bugs and move on? Again, > language syntax is not the solution to bugs. I'm not sure how to correlate what I was saying with your reply, sorry. Anyways, Chris, I think that the PEP hand-waves a lot of questions and doesn't have a comprehensive analysis of how the PEP will affect syntax and readability. It's up to you to consider taking my advice or not. I'll try to (again) restrain myself posting about this topic. Y From barry at python.org Mon Apr 30 14:21:45 2018 From: barry at python.org (Barry Warsaw) Date: Mon, 30 Apr 2018 11:21:45 -0700 Subject: [Python-Dev] Superseding PEPs (was Re: PEP 394: Allow the `python` command to not be installed (and other minor edits)) In-Reply-To: References: <4c4ec942-e17b-c4f4-4b2f-459fed2c32aa@gmail.com> <85wowttuyt.fsf@benfinney.id.au> Message-ID: <7A67B505-CB04-4662-954F-A4FF0140FC47@python.org> On Apr 30, 2018, at 06:36, Nick Coghlan wrote: > Instead of editing old PEPs, would it make sense to write a new one > which replaces the old one? As a general rule, at least the way I think about it, Informational PEPs can mostly be updated inline, evolving with new insights as we go along. As Nick points out, this gives folks a consistent place to read our current recommendations and guidelines. They?re informational so almost by definition contemporaneous. Standards Track PEPs on the other hand shouldn?t be changed once finalized, except around the margins, e.g. typos, updated URLs, that kind of thing. These PEPs should capture the discussion and design at the time that the feature is solidified and lands; they do *not* serve as latest up-to-date documentation on the feature. If the feature changes or becomes obsolete in future versions of Python, a new PEP should be written to supersede the old PEP. We even have an official `Superseded` Status value, and a `Superseded-By` header. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From guido at python.org Mon Apr 30 14:33:49 2018 From: guido at python.org (Guido van Rossum) Date: Mon, 30 Apr 2018 11:33:49 -0700 Subject: [Python-Dev] PEP 572: A backward step in readability In-Reply-To: References: <20180430163758.GF7400@ando.pearwood.info> Message-ID: TBH I think the text of the PEP could be much improved -- for example it should use motivating examples from real code, not artificial examples to show edge cases of the semantics. At this point I don't think that more people expressing an opinion one way or another are going to make a difference. Nobody whose vote matters is going to be convinced either way. On Mon, Apr 30, 2018 at 9:53 AM, Paul Moore wrote: > On 30 April 2018 at 17:37, Steven D'Aprano wrote: > > On Mon, Apr 30, 2018 at 08:09:35AM +0100, Paddy McCarthy wrote: > > [...] > >> A PEP that can detract from readability; *readability*, a central > >> tenet of Python, should > >> be rejected, (on principle!), when such objections are treated so > dismissively. > > > > Unless you have an objective measurement of readability, that objection > > is mere subjective personal preference, and not one that everyone agrees > > with. I for one think that used wisely, binding expressions will be > > *more* readable than the alternatives. (Even though := is not my > > preferred syntax.) > > On the other hand, the PEP doesn't do much to address the various > claims of readability issues. Whether they are subjective or not, > well-founded or not, the PEP does (in my view) come across as > unnecessarily dismissive of the question of readability. I suspect > that's largely due to Chris being extremely tired of having the same > arguments over and over again, and I can understand that. However, I > think it could be covered more completely - the remainder of your post > is exactly the sort of response that would be useful to have recorded. > > But one way or another, I think people will vote on the PEP based on > their (subjective) views, and so readability will get factored into > the overall response, one way or another. To what extent that response > affects Guido's final decision (I can't see him delegating on this > one!) remains to be seen, and honestly, I'm willing to trust Guido's > intuition. > > Paul > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From nad at python.org Mon Apr 30 15:37:21 2018 From: nad at python.org (Ned Deily) Date: Mon, 30 Apr 2018 15:37:21 -0400 Subject: [Python-Dev] IMPORTANT - final 3.7.0 beta cutoff! Message-ID: <01EC16A4-A520-4660-8592-B55241712F92@python.org> Just a reminder that 3.7.0b4 is almost upon us. Please get your feature fixes, bug fixes, and documentation updates in before 2018-04-30 ~23:59 Anywhere on Earth (UTC-12:00). That's about 16 hours from now. IMPORTANT: We are now in the final phase of 3.7.0. Tomorrow's 3.7.0b4 is the final beta planned for 3.7.0. After tomorrow, the next planned release is the 3.7.0 release candidate, on 2018-05-21, for final testing. Our goal is to have no changes between the release candidate and final; after rc1, changes applied to the 3.7 branch will be released in 3.7.1. Between now and the rc1 cutoff, please double-check that there are no critical problems outstanding and that documentation for new features in 3.7 is complete (including NEWS and What's New items), and that 3.7 is getting exposure and tested with our various platorms and third-party distributions and applications. Also, during the time leading up to the release candidate, we will be completing the What's New in 3.7 document. As noted before, the ABI for 3.7.0 was frozen as of 3.7.0b3. You should now be treating the 3.7 branch as if it were already released and in maintenance mode. That means you should only push the kinds of changes that are appropriate for a maintenance release: non-ABI-changing bug and feature fixes and documentation updates. If you find a problem that requires an ABI-altering or other significant user-facing change (for example, something likely to introduce an incompatibility with existing users' code or require rebuilding of user extension modules), please make sure to set the b.p.o issue to "release blocker" priority and describe there why you feel the change is necessary. If you are reviewing PRs for 3.7 (and please do!), be on the lookout for and flag potential incompatibilities (we've all made them). Thanks again for all of your hard work towards making 3.7.0 yet another great release - coming to a website near you on 06-15! --Ned -- Ned Deily nad at python.org -- [] From tseaver at palladion.com Mon Apr 30 15:53:04 2018 From: tseaver at palladion.com (Tres Seaver) Date: Mon, 30 Apr 2018 15:53:04 -0400 Subject: [Python-Dev] PEP 572: A backward step in readability In-Reply-To: <20180430163758.GF7400@ando.pearwood.info> References: <20180430163758.GF7400@ando.pearwood.info> Message-ID: On 04/30/2018 12:37 PM, Steven D'Aprano wrote: > - comprehensions? not readable, easy to abuse, hard for beginners > to comprehend; I still refer to them as "list incomprehensions" in my head, particularly for those whic expand across line breaks. Tres. -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com From vstinner at redhat.com Mon Apr 30 16:29:16 2018 From: vstinner at redhat.com (Victor Stinner) Date: Mon, 30 Apr 2018 22:29:16 +0200 Subject: [Python-Dev] [python-committers] IMPORTANT - final 3.7.0 beta cutoff! In-Reply-To: <01EC16A4-A520-4660-8592-B55241712F92@python.org> References: <01EC16A4-A520-4660-8592-B55241712F92@python.org> Message-ID: Hi, I modified the behaviour of the socketserver module: ThreadingMixIn now waits until all threads complete and ForkingMixIn now waits until all child processes complete https://bugs.python.org/issue31233 https://bugs.python.org/issue31151 I was supposed to do something, like add an option to at least opt-in for the old behaviour, but I didn't have the bandwidth to work on these issues. Python 3.7 may be shipped with the new behavior and no option to get the old behaviour (stop immediately, don't wait for threads/processes). Any volunteer to work on these issues? Victor 2018-04-30 21:37 GMT+02:00 Ned Deily : > Just a reminder that 3.7.0b4 is almost upon us. Please get your > feature fixes, bug fixes, and documentation updates in before > 2018-04-30 ~23:59 Anywhere on Earth (UTC-12:00). That's about 16 > hours from now. > > IMPORTANT: We are now in the final phase of 3.7.0. Tomorrow's 3.7.0b4 > is the final beta planned for 3.7.0. After tomorrow, the next planned > release is the 3.7.0 release candidate, on 2018-05-21, for final > testing. Our goal is to have no changes between the release candidate > and final; after rc1, changes applied to the 3.7 branch will be > released in 3.7.1. Between now and the rc1 cutoff, please > double-check that there are no critical problems outstanding and that > documentation for new features in 3.7 is complete (including NEWS and > What's New items), and that 3.7 is getting exposure and tested with > our various platorms and third-party distributions and applications. > Also, during the time leading up to the release candidate, we will be > completing the What's New in 3.7 document. > > As noted before, the ABI for 3.7.0 was frozen as of 3.7.0b3. You > should now be treating the 3.7 branch as if it were already released > and in maintenance mode. That means you should only push the kinds of > changes that are appropriate for a maintenance release: > non-ABI-changing bug and feature fixes and documentation updates. If > you find a problem that requires an ABI-altering or other significant > user-facing change (for example, something likely to introduce an > incompatibility with existing users' code or require rebuilding of > user extension modules), please make sure to set the b.p.o issue to > "release blocker" priority and describe there why you feel the change > is necessary. If you are reviewing PRs for 3.7 (and please do!), be > on the lookout for and flag potential incompatibilities (we've all > made them). > > Thanks again for all of your hard work towards making 3.7.0 yet > another great release - coming to a website near you on 06-15! > > --Ned > > -- > Ned Deily > nad at python.org -- [] > > _______________________________________________ > python-committers mailing list > python-committers at python.org > https://mail.python.org/mailman/listinfo/python-committers > Code of Conduct: https://www.python.org/psf/codeofconduct/ From paddy3118 at gmail.com Mon Apr 30 16:30:20 2018 From: paddy3118 at gmail.com (Paddy McCarthy) Date: Mon, 30 Apr 2018 21:30:20 +0100 Subject: [Python-Dev] PEP 572: A backward step in readability In-Reply-To: <20180430163758.GF7400@ando.pearwood.info> References: <20180430163758.GF7400@ando.pearwood.info> Message-ID: On 30 April 2018 at 17:37, Steven D'Aprano wrote: > > On Mon, Apr 30, 2018 at 08:09:35AM +0100, Paddy McCarthy wrote: > [...] > > A PEP that can detract from readability; *readability*, a central > > tenet of Python, should > > be rejected, (on principle!), when such objections are treated so dismissively. > Unless you have an objective measurement of readability, that objection > is mere subjective personal preference, and not one that everyone agrees > with. True, as is the dismissal from the PEP. It is the PEP, looking to force change to the language, to prove its point rather than dismiss statements of its detractors. > > The "not readable" objection has been made, extremely vehemently, > against nearly all major syntax changes to Python: I don't count myself as usually against change. I applaud the move to Python 3, I use all of the language features you mention at times; many in my Rosetta Code task examples; but this change opens the door to a class of bug that will take care to avoid and which I remember cutting my C coding to get away from. The PEP fails to adequately address the concerns of "us naysayers" For example; if someone were to find out that "assignment expressions were faster", then you would be hard pressed to stop their over-use. As soon as someone assigns to a name and uses that same name in the one expression, you need a better grasp of the order of expression evaluation to read it. hat is best avoided, in my subjective, personal, view. > -- > Steve With respect, but in disagreement - Paddy. From ja.py at farowl.co.uk Mon Apr 30 16:00:14 2018 From: ja.py at farowl.co.uk (Jeff Allen) Date: Mon, 30 Apr 2018 21:00:14 +0100 Subject: [Python-Dev] Every Release Can Be a Mini "Python 4000", Within Reason (was (name := expression) doesn't fit the narrative of PEP 20) In-Reply-To: <5AE6B62F.9050902@canterbury.ac.nz> References: <5AE2D36E.7060801@canterbury.ac.nz> <3193344c-24c1-5224-c40e-91d26687b51c@farowl.co.uk> <5AE6B62F.9050902@canterbury.ac.nz> Message-ID: <4fcfb653-4d8b-f209-588f-fe1b88689ee7@farowl.co.uk> On 30/04/2018 07:22, Greg Ewing wrote: > Jeff Allen wrote: >> I speculate this all goes back to some pre-iteration version of >> FORmula TRANslation, where to its inventors '=' was definition and >> these really were "statements" in the normal sense of stating a truth. > > Yeah, also the earliest FORTRAN didn't even *have* comparison > operators. A conditional branch was something like > I should have known that would turn out to be the most interesting part in my message. Not to take us further off topic, I'll just say thanks to Eitan's reply, I found this: http://www.softwarepreservation.org/projects/FORTRAN/BackusEtAl-Preliminary%20Report-1954.pdf They were not "statements", but "formulas" while '=' was assignment (sec 8) *and* comparison (sec 10B). So conversely to our worry, they actually wanted users to think of assignment initially as a mathematical formula (page 2) in order to exploit the similarity to a familiar concept, albeit a=a+i makes no sense from this perspective. Jeff Allen -------------- next part -------------- An HTML attachment was scrubbed... URL: From larry at hastings.org Mon Apr 30 20:27:08 2018 From: larry at hastings.org (Larry Hastings) Date: Mon, 30 Apr 2018 17:27:08 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: Message-ID: <9abfb93b-423b-2d1a-b11c-b1d926611d2f@hastings.org> On 04/30/2018 07:30 AM, Mark Shannon wrote: > Would Python be better with two subtly different assignment operators? > The answer of "no" seems self evident to me. Maybe this has been covered in the thread earlier--if so, I missed it, sorry.? But ISTM that Python already has multiple ways to perform an assignment. All these statements assign to x: x = y for x in y: with y as x: except Exception as x: And, if you want to get *super* pedantic: import x def x(): ... class x: ... I remain -1 on 572, but I'm not sure it can genuinely be said that Python only has one way to assign a value to a variable. //arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Mon Apr 30 20:59:03 2018 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 1 May 2018 10:59:03 +1000 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: <9abfb93b-423b-2d1a-b11c-b1d926611d2f@hastings.org> References: <9abfb93b-423b-2d1a-b11c-b1d926611d2f@hastings.org> Message-ID: <20180501005903.GH7400@ando.pearwood.info> On Mon, Apr 30, 2018 at 05:27:08PM -0700, Larry Hastings wrote: > > On 04/30/2018 07:30 AM, Mark Shannon wrote: > >Would Python be better with two subtly different assignment operators? > >The answer of "no" seems self evident to me. > > Maybe this has been covered in the thread earlier--if so, I missed it, > sorry.? But ISTM that Python already has multiple ways to perform an > assignment. > > All these statements assign to x: [snip SEVEN distinct binding operations] > I remain -1 on 572, but I'm not sure it can genuinely be said that > Python only has one way to assign a value to a variable. "Not sure"? Given that you listed seven ways, how much more evidence do you need to conclude that it is simply wrong to say that Python has a single way to assign a value to a name? :-) -- Steve From tjreedy at udel.edu Mon Apr 30 21:25:15 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 30 Apr 2018 21:25:15 -0400 Subject: [Python-Dev] Every Release Can Be a Mini "Python 4000", Within Reason (was (name := expression) doesn't fit the narrative of PEP 20) In-Reply-To: <4fcfb653-4d8b-f209-588f-fe1b88689ee7@farowl.co.uk> References: <5AE2D36E.7060801@canterbury.ac.nz> <3193344c-24c1-5224-c40e-91d26687b51c@farowl.co.uk> <5AE6B62F.9050902@canterbury.ac.nz> <4fcfb653-4d8b-f209-588f-fe1b88689ee7@farowl.co.uk> Message-ID: On 4/30/2018 4:00 PM, Jeff Allen wrote: > They were not "statements", but "formulas" while '=' was assignment (sec > 8) *and* comparison (sec 10B). So conversely to our worry, they actually > wanted users to think of assignment initially as a mathematical formula > (page 2) in order to exploit the similarity to a familiar concept, > albeit a=a+i makes no sense from this perspective. When explaining iterative algorithms, such as Newton's method, mathematicians write things like a' = a+1 or a(sub i+1 or sub new) = f(a(sub i or sub old)) . For computer, we drop the super/subscript. Or one can write more circuitously, anew = update(aold) aold = anew The abbreviations should be explained when teaching loops. For proving that the body of a loop maintains a loop constant, one may reinstate the super- or sub-script. -- Terry Jan Reedy From chris.jerdonek at gmail.com Mon Apr 30 22:36:43 2018 From: chris.jerdonek at gmail.com (Chris Jerdonek) Date: Mon, 30 Apr 2018 19:36:43 -0700 Subject: [Python-Dev] PEP 572: Assignment Expressions In-Reply-To: References: <5add0911.81b7500a.d7e15.acb1SMTPIN_ADDED_MISSING@mx.google.com> <20180423092806.530b5996@fsol> <83e528a7-9758-fcbf-eb91-b6f1763b5e2e@mail.de> <20180424231911.GN11616@ando.pearwood.info> Message-ID: On Thu, Apr 26, 2018 at 10:33 AM, Sven R. Kunze wrote: > On 25.04.2018 01:19, Steven D'Aprano wrote: >> >> Sorry, gcd(diff, n) is not the "perfect name", and I will tell you that >> sometimes g is better. [...] > > We were talking about the real-world code snippet of Tim (as a justification > of := ) and alternative rewritings of it without resorting to new syntax. Apologies if this idea has already been discussed (I might have missed the relevant email), but thinking back to Tim's earlier example-- if (diff := x - x_base) and (g := gcd(diff, n)) > 1: return g it occurs to me this could be implemented with current syntax using a pattern like the following: stashed = [None] def stash(x): stashed[0] = x return x if stash(x - x_base) and stash(gcd(stashed[0], n)) > 1: return stashed[0] There are many variations to this idea, obviously. For example, one could allow passing a "name" to stash(), or combine stash / stashed into a single, callable object that allows setting and reading from its store. I wonder if one of them could be made into a worthwhile pattern or API.. --Chris From raymond.hettinger at gmail.com Mon Apr 30 23:08:43 2018 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Mon, 30 Apr 2018 20:08:43 -0700 Subject: [Python-Dev] PEP 572: A backward step in readability In-Reply-To: <20180430163758.GF7400@ando.pearwood.info> References: <20180430163758.GF7400@ando.pearwood.info> Message-ID: <79D88AF0-C718-46AB-AB01-C4E7CA3E93D5@gmail.com> > On Apr 30, 2018, at 9:37 AM, Steven D'Aprano wrote: > > On Mon, Apr 30, 2018 at 08:09:35AM +0100, Paddy McCarthy wrote: > [...] >> A PEP that can detract from readability; *readability*, a central >> tenet of Python, should >> be rejected, (on principle!), when such objections are treated so dismissively. > > Unless you have an objective measurement of readability, that objection > is mere subjective personal preference, and not one that everyone agrees > with. Sorry Steven, but that doesn't seem like it is being fair to Paddy. Of course, readability can't be measured objectively with ruler (that is a false standard). However, readability is still a real issue that affects us daily even though objective measurement aren't possible. All of us who do code reviews make assessments of readability on a daily basis even though we have no objective measures. We know hard to read when we see it. In this thread, several prominent and highly experienced devs reported finding it difficult to parse some of the examples and some mis-parsed the semantics of the examples. It is an objective fact that they reported readability issues. That is of great concern and shouldn't be blown off with a comment that readability, "is a mere subjective personal preference". At its heart, readability is the number one concern in language design. Also, there another area where it looks like valid concerns are being dismissed out of hand. Several respondents worried that the proposed feature will lead to writing bad code. Their comments seem to have been swept under the table with responses along the lines of "well any feature can be used badly, so we don't care about that, some people will write bad code no matter what we do". While that is true to some extent, there remains a valid issue concerning the propensity for misuse. ISTM the proposed feature relies on users showing a good deal of self-restriaint and having a clear knowledge of boundary between the "clear-win" cases (like the regex match object example) and the puzzling cases (assignments being used in and-operator and or-operator chains). It also relies on people not making hard to find mistakes (like mistyping := when == was intended). There is a real difference between a feature that could be abused versus a feature that has a propensity for being misused, being mistyped, or being misread (all of which have occurred multiple times in these threads). > The "not readable" objection has been made, extremely vehemently, > against nearly all major syntax changes to Python: I think that is a false recollection of history. Comprehensions were welcomed and highly desired. Decorators were also highly sought after -- there was only a question of the best possible syntax. The ternary operator was clamored for by an enormous number of users (though there was little agreement on the best spelling). Likewise, the case for augmented assignments was somewhat strong (eliminating having to spell the assignment target twice). Each of those proposals had their debates, but none of them had a bunch of core devs flat-out opposed like we do now. It really isn't the same at all. However, even if the history had been recalled correctly, it would still be a logical fallacy to posit "in the past, people opposed syntax changes that later proved to be popular, therefore we should ignore all concerns being expressed today". To me, that seems like a rhetorical trick for dismissing a bunch of thoughtful posts. Adding this new syntax is a one-way trip -- we don't get to express regrets later. Accordingly, it would be nice if the various concerns being presented were addressed directly rather than being dismissed with a turn of phrase. Nor should it matter whether concerns were articulately expressed (being articulate isn't always correlated with being right). Raymond From raymond.hettinger at gmail.com Mon Apr 30 23:30:34 2018 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Mon, 30 Apr 2018 20:30:34 -0700 Subject: [Python-Dev] PEP 572: Usage of assignment expressions in C In-Reply-To: <20180428174524.7e122239@fsol> References: <20180428174524.7e122239@fsol> Message-ID: > On Apr 28, 2018, at 8:45 AM, Antoine Pitrou wrote: > >> I personally haven't written a lot of C, so have no personal experience, >> but if this is at all a common approach among experienced C developers, it >> tells us a lot. > > I think it's a matter of taste and personal habit. Some people will > often do it, some less. Note that C also has a tendency to make it > more useful, because doesn't have exceptions, so functions need to > (ab)use return values when they want to indicate an error. When you're > calling such functions (for example I/O functions), you routinely have > to check for special values indicating an error, so it's common to see > code such as: > > // Read up to n bytes from file descriptor > if ((bytes_read = read(fd, buf, n)) == -1) { > // Error occurred while reading, do something > } Thanks Antoine, this is an important point that I hope doesn't get lost. In a language with exceptions, assignment expressions are less needful. Also, the pattern of having of having mutating methods return None further limits the utility. Raymond