[Python-checkins] peps: PEP 501: switch to a more normal binary operator
nick.coghlan
python-checkins at python.org
Sat Aug 22 23:04:41 CEST 2015
https://hg.python.org/peps/rev/a7d8ee21a9dd
changeset: 5974:a7d8ee21a9dd
user: Nick Coghlan <ncoghlan at gmail.com>
date: Sun Aug 23 07:04:33 2015 +1000
summary:
PEP 501: switch to a more normal binary operator
files:
pep-0501.txt | 112 +++++++++++++++++++++++---------------
1 files changed, 67 insertions(+), 45 deletions(-)
diff --git a/pep-0501.txt b/pep-0501.txt
--- a/pep-0501.txt
+++ b/pep-0501.txt
@@ -29,19 +29,30 @@
has not been properly escaped before being passed to the ``os.system`` call.
To address that problem (and a number of other concerns), this PEP proposes an
-alternative approach to compiler supported interpolation, based on a new
-``__interpolate__`` magic method, and using a substitution syntax inspired by
+alternative approach to compiler supported interpolation, based on a new ``$``
+binary operator with a syntactically constrained right hand side, a new
+``__interpolate__`` magic method, and a substitution syntax inspired by
that used in ``string.Template`` and ES6 JavaScript, rather than adding a 4th
substitution variable syntax to Python.
+Some examples of the proposed syntax::
+
+ msg = str$'My age next year is ${age+1}, my anniversary is ${anniversary:%A, %B %d, %Y}.'
+ print(_$"This is a $translated $message")
+ translated = l20n$"{{ $user }} is running {{ appname }}"
+ myquery = sql$"SELECT $column FROM $table;"
+ mycommand = sh$"cat $filename"
+ mypage = html$"<html><body>${response.body}</body></html>"
+
Proposal
========
-This PEP proposes that the new syntax::
+This PEP proposes the introduction of a new binary operator specifically for
+interpolation of arbitrary expressions::
- value = !interpolator "Substitute $names and ${expressions} at runtime"
+ value = interpolator $ "Substitute $names and ${expressions} at runtime"
-be interpreted as::
+This would be effectively interpreted as::
_raw_template = "Substitute $names and ${expressions} at runtime"
_parsed_fields = (
@@ -54,25 +65,24 @@
_parsed_fields,
_field_values)
-Whitespace would be permitted between the interpolator name and the opening
-quote, but not required in most cases.
+The right hand side of the new operator would be syntactically constrained to
+be a string literal.
The ``str`` builtin type would gain an ``__interpolate__`` implementation that
-supported the following ``str.format`` based semantics::
+supported the following ``str.format`` inspired semantics::
>>> import datetime
>>> name = 'Jane'
>>> age = 50
>>> anniversary = datetime.date(1991, 10, 12)
- >>> !str'My name is $name, my age next year is ${age+1}, my anniversary is ${anniversary:%A, %B %d, %Y}.'
+ >>> str$'My name is $name, my age next year is ${age+1}, my anniversary is ${anniversary:%A, %B %d, %Y}.'
'My name is Jane, my age next year is 51, my anniversary is Saturday, October 12, 1991.'
- >>> !str'She said her name is ${name!r}.'
+ >>> str$'She said her name is ${name!r}.'
"She said her name is 'Jane'."
-The interpolation prefix could be used with single-quoted, double-quoted and
-triple quoted strings. It may also be used with raw strings, but in that case
-whitespace would be required between the interpolator name and the trailing
-string.
+The interpolation operator could be used with single-quoted, double-quoted and
+triple quoted strings, including raw strings. It would not support bytes
+literals as the right hand side of the expression.
This PEP does not propose to remove or deprecate any of the existing
string formatting mechanisms, as those will remain valuable when formatting
@@ -122,18 +132,16 @@
Specification
=============
-In source code, interpolation expressions are introduced by the new character
-``!``. This is a new kind of expression, consisting of::
+This PEP proposes the introduction of ``$`` as a new binary operator designed
+specifically to support interpolation of template strings::
- !DOTTED_NAME TEMPLATE_STRING
+ INTERPOLATOR $ TEMPLATE_STRING
-Similar to ``yield`` expressions, this construct can be used without
-parentheses as a standalone expression statement, as the sole expression on the
-right hand side of an assignment or return statement, and as the sole argument
-to a function. In other situations, it requires containing parentheses to avoid
-ambiguity.
+This would work as a normal binary operator (precedence TBD), with the
+exception that the template string would be syntactically constrained to be a
+string literal, rather than permitting arbitrary expressions.
-The template string must be a Unicode string (byte strings are not permitted),
+The template string must be a Unicode string (bytes literals are not permitted),
and string literal concatenation operates as normal within the template string
component of the expression.
@@ -180,7 +188,7 @@
For the following example interpolation expression::
- !str 'abc${expr1:spec1}${expr2!r:spec2}def${expr3:!s}ghi $ident $$jkl'
+ str$'abc${expr1:spec1}${expr2!r:spec2}def${expr3:!s}ghi $ident $$jkl'
the parsed fields tuple would be::
@@ -253,7 +261,7 @@
>>> def foo(data):
... return data + 20
...
- >>> !str 'input=$bar, output=${foo(bar)}'
+ >>> str$'input=$bar, output=${foo(bar)}'
'input=10, output=30'
Is essentially equivalent to::
@@ -267,9 +275,9 @@
The proposed interpolation expressions make it potentially attractive to write
code like the following::
- myquery = !str "SELECT $column FROM $table;"
- mycommand = !str "cat $filename"
- mypage = !str "<html><body>$content</body></html>"
+ myquery = str$"SELECT $column FROM $table;"
+ mycommand = str$"cat $filename"
+ mypage = str$"<html><body>${response.body}</body></html>"
These all represent potential vectors for code injection attacks, if any of the
variables being interpolated happen to come from an untrusted source. The
@@ -277,9 +285,9 @@
use case specific interpolators that take care of quoting interpolated values
appropriately for the relevant security context::
- myquery = !sql "SELECT $column FROM $table;"
- mycommand = !sh "cat $filename"
- mypage = !html "<html><body>$content</body></html>"
+ myquery = sql$"SELECT $column FROM $table;"
+ mycommand = sh$"cat $filename"
+ mypage = html$"<html><body>${response.body}</body></html>"
This PEP does not cover adding such interpolators to the standard library,
but instead ensures they can be readily provided by third party libraries.
@@ -305,13 +313,13 @@
Unmatched braces::
- >>> !str 'x=${x'
+ >>> str$'x=${x'
File "<stdin>", line 1
SyntaxError: missing '}' in interpolation expression
Invalid expressions::
- >>> !str 'x=${!x}'
+ >>> str$'x=${!x}'
File "<fstring>", line 1
!x
^
@@ -350,9 +358,9 @@
field_values[expr] = values[field_num]
return field_values
-And would then be invoked as::
+And would could then be invoked as::
- print(!i18n "This is a $translated $message")
+ print(_$"This is a $translated $message")
Any actual implementation would need to address other issues (most notably
message catalog extraction), but this gives the general idea of what might be
@@ -365,11 +373,11 @@
With the syntax in this PEP, an l20n interpolator could be written as::
- translated = !l20n "{{ $user }} is running {{ appname }}"
+ translated = l20n$"{{ $user }} is running {{ appname }}"
With the syntax proposed in PEP 498 (and neglecting the difficulty of doing
catalog lookups using PEP 498's semantics), the necessary brace escaping would
-make the string look like this in order to interpolating the user variable
+make the string look like this in order to interpolate the user variable
while preserving all of the expected braces::
interpolated = "{{{{ ${user} }}}} is running {{{{ appname }}}}"
@@ -388,8 +396,8 @@
potentially be applied to the logging module's event reporting APIs, permitting
relevant details to be captured using forms like::
- !logging.debug "Event: $event; Details: $data"
- !logging.critical "Error: $error; Details: $data"
+ logging.debug$"Event: $event; Details: $data"
+ logging.critical$"Error: $error; Details: $data"
Discussion
@@ -398,14 +406,28 @@
Refer to PEP 498 for additional discussion, as several of the points there
also apply to this PEP.
-Compatibility with IPython magic strings
-----------------------------------------
+Determining relative precedence
+-------------------------------
-IPython uses "!" to introduce custom interactive constructs. These are only
-used at statement level, and could continue to be special cased in the
-IPython runtime.
+The PEP doesn't currently specify the relative precedence of the new operator,
+as the only examples considered so far concern standalone expressions or simple
+variable assignments.
-This existing usage *did* help inspire the syntax proposed in this PEP.
+Development of a reference implementation based on the PEP 498 reference
+implementation may help answer that question.
+
+Deferring support for binary interpolation
+------------------------------------------
+
+Supporting binary interpolation with this syntax would be relatively
+straightforward (just a matter of relaxing the syntactic restrictions on the
+right hand side of the operator), but poses a signficant likelihood of
+producing confusing type errors when a text interpolator was presented with
+binary input.
+
+Since the proposed operator is useful without binary interpolation support, and
+such support can be readily added later, further consideration of binary
+interpolation is considered out of scope for the current PEP.
Preserving the raw template string
----------------------------------
--
Repository URL: https://hg.python.org/peps
More information about the Python-checkins
mailing list