[Python-ideas] Draft PEP on string interpolation

Mon Aug 24 23:29:07 CEST 2015

On Mon, Aug 24, 2015 at 3:57 PM, Mike Miller <python-ideas at mgmiller.net>
wrote:

> Hi, here's my latest idea, riffing on other's latest this weekend.
>
> Let's call these e-strings (for expression), as it's easier to refer to
> the letter of the proposals than three digit numbers.
>
> So, an e-string looks like an f-string, though at compile-time, it is
> converted to an object instead (like i-string):
>
>     print(e'Hello {friend}, filename: {filename}.')   # converts to ==>
>
>     print(estr('Hello {friend}, filename: {filename}.', friend=friend,
>                                                         filename=filename))
>
> An estr is a subclass of str, therefore able to do the nice things a
> string can do.  Rendering is deferred, and it also has a raw member,
> escape(), and translate() methods:
>
>     class estr(str):
>         # init: saves self.raw, args, kwargs for later
>         # methods, ops render it
>         # def escape(self, escape_func):  # handles escaping
>         # def translate(self, template, safe=True): # optional i18n support
>

* How do I overload/subclass [class estr()]?
* Does it always just read LC_ALL='utf8' (or where do I specify that
global/thread/frame-local?)
* How do I escape_func?

Jinja2 uses MarkupSafe, with a class named Markup:

class Markup():
    def __html__()
    def __html_format__()

IPython can display objects with _repr_fmt_() callables,
which TBH I prefer because it's not name mangled
and so more easily testable. [3,4]

Existing IPython rich display methods [5,6,7,8]

_mime_map = dict(
    _repr_png_="image/png",
    _repr_jpeg_="image/jpeg",
    _repr_svg_="image/svg+xml",
    _repr_html_="text/html",
    _repr_json_="application/json",
    _repr_javascript_="application/javascript",
)

# _repr_latex_ = "text/latex"
# _repr_retina_ = "image/png"

Suggestd IPython methods

- [ ] _repr_shell_
  - [ ] single_quote_shell_escape
  - [ ] double_quote_shell_escape
- [ ] _repr_sql_ (*NOTE: SQL variants, otherworldly-escaping dependency /
newb errors)

[1] https://pypi.python.org/pypi/MarkupSafe
[2] https://github.com/mitsuhiko/markupsafe
[3] https://ipython.org/ipython-doc/dev/config/integrating.html
[4] https://ipython.org/ipython-doc/dev/config/integrating.html#rich-display
[5] https://github.com/ipython/ipython/blob/master/IPython/utils/capture.py
[6]
https://github.com/ipython/ipython/blob/master/IPython/utils/tests/test_capture.py
[7] https://github.com/ipython/ipython/blob/master/IPython/core/display.py
[8]
https://github.com/ipython/ipython/blob/master/IPython/core/tests/test_display.py

* IPython: _repr_fmt_()
* MarkupSafe: __html__()

> To make it as simple as possible to use by end-developers, it 1) doesn't
> require str() to be run explicitly, it renders itself when needed via its
> various methods and operators.  Look for .raw, if you need the original.
>
> Also, 2) a bit of responsibility is pushed to stdlib/pypi.  In a handful
> of sensitive places, the object is checked beforehand and escaped when
> needed:
>
>     def os_system(command):   # imagine os.system, subprocess, dbapi, etc.
>         if isinstance(command, estr):
>             command = command.escape(shlex.quote)  # each chooses its own
> rules
>         do_something(command)
>
> This means a billion lines of code using e-strings won't have to care
> about them, only a handful of places.  What is easiest to type is now safe
> as well:
>
>     os.system(e'cat {filename}')  # sleep easy
>
> A translate method might available also (though we may have given up on
> i18n already), to provide a new raw string from a message catalog:
>
>     rendered = message.translate(translated_message)  # fmt syntax TBD
>
> This should enable the safety and features we'd like, without burdening
> the everyday user.  I've created a sample script, here is the output:
>
>     # consider:   estr('Hello {friend}, filename: {filename}.')
>     friend:       'John'
>     filename:     "somefile; rm -rf ~ 'foo' <html>"
>
>     original:     Hello {friend}, filename: {filename}.
>     print():      Hello John, filename: somefile; rm -rf ~ 'foo' <html>.
>
>     shell escape:
>         Hello John, filename: 'somefile; rm -rf ~ '"'"'foo'"'"' <html>'.
>     html escape:
>         Hello John, filename: somefile; rm -rf ~ &#x27;foo&#x27;
> <html>.
>     sql escape:   Hello "John", filename: "somefile; rm -rf ~ 'foo'
> <html>".
>     logger DEBUG  Hello John, filename: somefile; rm -rf ~ 'foo' <html>.
>
>     upper+utf8:   b"HELLO JOHN, FILENAME: SOMEFILE; RM -RF ~ 'FOO' <HTML>."
>     translated:   Hola John, archivo: somefile; rm -rf ~ 'foo' <html>.
>
>
> Anything I've missed?
>
> -Mike
>
>
> On 08/20/2015 04:10 PM, Mike Miller wrote:
>
>> The ground seems to be settling on the issue, so I have tried my hand at
>> a grand
>> unified pep for string interpolation.
>>
>> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150824/c23eafeb/attachment.html>