[Python-checkins] bpo-40246: Revert reporting of invalid string prefixes (GH-19888)

[Lysandros Nikolaou]

https://github.com/python/cpython/commit/846d8b28ab9bb6197ee81372820311c0abe509c0
commit: 846d8b28ab9bb6197ee81372820311c0abe509c0
branch: master
author: Lysandros Nikolaou <lisandrosnik at gmail.com>
committer: GitHub <noreply at github.com>
date: 2020-05-04T12:32:18+01:00
summary:

bpo-40246: Revert reporting of invalid string prefixes (GH-19888)

Due to backwards compatibility concerns regarding keywords immediately followed by a string without whitespace between them (like in `bg="#d00" if clear else"#fca"`) will fail to parse,
commit 41d5b94af44e34ac05d4cd57460ed104ccf96628 has to be reverted.

files:
A Misc/NEWS.d/next/Core and Builtins/2020-05-03-23-28-11.bpo-40246.c1D7x8.rst
M Include/errcode.h
M Lib/test/test_fstring.py
M Parser/pegen/pegen.c
M Parser/tokenizer.c
M Python/pythonrun.c

diff --git a/Include/errcode.h b/Include/errcode.h
index 9af8d5c03d59b..b37cd261d5ec4 100644
--- a/Include/errcode.h
+++ b/Include/errcode.h
@@ -31,7 +31,6 @@ extern "C" {
 #define E_LINECONT      25      /* Unexpected characters after a line continuation */
 #define E_IDENTIFIER    26      /* Invalid characters in identifier */
 #define E_BADSINGLE     27      /* Ill-formed single statement input */
-#define E_BADPREFIX     28      /* Bad string prefixes */
 
 #ifdef __cplusplus
 }
diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py
index fe465b7e1d43d..ac5aa9a76efe7 100644
--- a/Lib/test/test_fstring.py
+++ b/Lib/test/test_fstring.py
@@ -864,7 +864,7 @@ def test_invalid_string_prefixes(self):
                              "Bf''",
                              "BF''",]
         double_quote_cases = [case.replace("'", '"') for case in single_quote_cases]
-        self.assertAllRaise(SyntaxError, 'invalid string prefix',
+        self.assertAllRaise(SyntaxError, 'unexpected EOF while parsing',
                             single_quote_cases + double_quote_cases)
 
     def test_leading_trailing_spaces(self):
diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-05-03-23-28-11.bpo-40246.c1D7x8.rst b/Misc/NEWS.d/next/Core and Builtins/2020-05-03-23-28-11.bpo-40246.c1D7x8.rst
new file mode 100644
index 0000000000000..62cd632ffd070
--- /dev/null
+++ b/Misc/NEWS.d/next/Core and Builtins/2020-05-03-23-28-11.bpo-40246.c1D7x8.rst	
@@ -0,0 +1 @@
+Reporting a specialised error message for invalid string prefixes, which was introduced in :issue:`40246`, is being reverted due to backwards compatibility concerns for strings that immediately follow a reserved keyword without whitespace between them. Constructs like `bg="#d00" if clear else"#fca"` were failing to parse, which is not an acceptable breakage on such short notice.
diff --git a/Parser/pegen/pegen.c b/Parser/pegen/pegen.c
index 391f9b91eab90..c311593af70f5 100644
--- a/Parser/pegen/pegen.c
+++ b/Parser/pegen/pegen.c
@@ -334,9 +334,6 @@ tokenizer_error(Parser *p)
         case E_IDENTIFIER:
             msg = "invalid character in identifier";
             break;
-        case E_BADPREFIX:
-            RAISE_SYNTAX_ERROR("invalid string prefix");
-            return -1;
         case E_EOFS:
             RAISE_SYNTAX_ERROR("EOF while scanning triple-quoted string literal");
             return -1;
diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c
index 95dfc5388037d..0f2b6af5e50ad 100644
--- a/Parser/tokenizer.c
+++ b/Parser/tokenizer.c
@@ -1396,10 +1396,6 @@ tok_get(struct tok_state *tok, const char **p_start, const char **p_end)
         *p_start = tok->start;
         *p_end = tok->cur;
 
-        if (c == '"' || c == '\'') {
-            tok->done = E_BADPREFIX;
-            return ERRORTOKEN;
-        }
         /* async/await parsing block. */
         if (tok->cur - tok->start == 5 && tok->start[0] == 'a') {
             /* May be an 'async' or 'await' token.  For Python 3.7 or
diff --git a/Python/pythonrun.c b/Python/pythonrun.c
index 79147e430a1ad..1b79a33c814da 100644
--- a/Python/pythonrun.c
+++ b/Python/pythonrun.c
@@ -1609,9 +1609,6 @@ err_input(perrdetail *err)
     case E_BADSINGLE:
         msg = "multiple statements found while compiling a single statement";
         break;
-    case E_BADPREFIX:
-        msg = "invalid string prefix";
-        break;
     default:
         fprintf(stderr, "error=%d\n", err->error);
         msg = "unknown parsing error";