[Python-Dev] Parsing f-strings from PEP 498 -- Literal String Interpolation

[Eric V. Smith]

On 11/4/2016 2:03 PM, Fabio Zadrozny wrote:

>     Using PyParser_ASTFromString is the easiest possible way to do this.
>     Given a string, it returns an AST node. What could be simpler?
>
>
> ​I think that for implementation purposes, given the python
> infrastructure, it's fine, but for specification purposes, probably
> incorrect... As I don't think f-strings should accept:
>
>  f"start {import sys; sys.version_info[0];} end" (i.e.:
> ​
> PyParser_ASTFromString doesn't just return an expression, it accepts any
> valid Python code, even code which can't be used in an f-string).

Not so. It should only accept expressions, not statements:

 >>> f"start {import sys; sys.version_info[0];} end"
   File "<fstring>", line 1
     (import sys; sys.version_info[0];)
           ^
SyntaxError: invalid syntax

>         Also, you could still have a separate grammar saying how to look
>         inside
>         f-strings (this would make the lives of other implementors
>         easier) even
>         if it was a post-processing step as you're doing now.
>
>
>     Yes. I've contemplated exposing the f-string scanner. That's the
>     part that returns expressions (as strings) and literal strings. I
>     realize that won't help 3.6.
>
>
> Nice...
>
> As a note, just for the record, my own interest on f-strings is knowing
> how exactly they are parsed for providing a preview of PyDev with syntax
> highlighting and preliminary support for f-strings (which at the very
> minimum besides syntax highlighting for the parts of f-strings should
> also show syntax errors inside them).

I understand there's a need to make the specification more rigorous. 
Hopefully we'll get there.

Eric.