[Python-checkins] bpo-36256: Fix bug in parsermodule when parsing if statements (GH-12488)

[Pablo Galindo]

https://github.com/python/cpython/commit/00eb97b4a7d9a73b88ed7c76faee4e49204d5a00
commit: 00eb97b4a7d9a73b88ed7c76faee4e49204d5a00
branch: 3.7
author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com>
committer: Pablo Galindo <Pablogsal at gmail.com>
date: 2019-03-21T23:56:20Z
summary:

bpo-36256: Fix bug in parsermodule when parsing if statements (GH-12488)

bpo-36256: Fix bug in parsermodule when parsing if statements

In the parser module, when validating nodes before starting the parsing with to create a ST in "parser_newstobject" there is a problem that appears when two arcs in the same DFA state has transitions with labels with the same type. For example, the DFA for if_stmt has a state with
two labels with the same type: "elif" and "else" (type NAME). The algorithm tries one by one the arcs until the label that starts the arc transition has a label with the same type of the current child label we are trying to accept. In this case, the arc for "elif" comes before the arc for "else"and passes this test (because the current child label is "else" and has the same type as "elif"). This lead to expecting a namedexpr_test (305) instead of a colon (11). The solution is to compare also the string representation (in case there is one) of the labels to see if the transition that we have is the correct one.
(cherry picked from commit 9a0000d15d27361eaa47b77600c7c00a9787a894)

Co-authored-by: Pablo Galindo <Pablogsal at gmail.com>

files:
A Misc/NEWS.d/next/Core and Builtins/2019-03-21-00-24-18.bpo-12477.OZHa0t.rst
M Lib/test/test_parser.py
M Modules/parsermodule.c

diff --git a/Lib/test/test_parser.py b/Lib/test/test_parser.py
index 274e26061a19..94e454663573 100644
--- a/Lib/test/test_parser.py
+++ b/Lib/test/test_parser.py
@@ -318,6 +318,10 @@ def test_try_stmt(self):
         self.check_suite("try: pass\nexcept: pass\nelse: pass\n"
                          "finally: pass\n")
 
+    def test_if_stmt(self):
+        self.check_suite("if True:\n  pass\nelse:\n  pass\n")
+        self.check_suite("if True:\n  pass\nelif True:\n  pass\nelse:\n  pass\n")
+
     def test_position(self):
         # An absolutely minimal test of position information.  Better
         # tests would be a big project.
diff --git a/Misc/NEWS.d/next/Core and Builtins/2019-03-21-00-24-18.bpo-12477.OZHa0t.rst b/Misc/NEWS.d/next/Core and Builtins/2019-03-21-00-24-18.bpo-12477.OZHa0t.rst
new file mode 100644
index 000000000000..aada7f912a6c
--- /dev/null
+++ b/Misc/NEWS.d/next/Core and Builtins/2019-03-21-00-24-18.bpo-12477.OZHa0t.rst	
@@ -0,0 +1,2 @@
+Fix bug in parsermodule when parsing a state in a DFA that has two or more
+arcs with labels of the same type. Patch by Pablo Galindo.
diff --git a/Modules/parsermodule.c b/Modules/parsermodule.c
index 38e5f750d572..67c874267f24 100644
--- a/Modules/parsermodule.c
+++ b/Modules/parsermodule.c
@@ -666,7 +666,12 @@ validate_node(node *tree)
         for (arc = 0; arc < dfa_state->s_narcs; ++arc) {
             short a_label = dfa_state->s_arc[arc].a_lbl;
             assert(a_label < _PyParser_Grammar.g_ll.ll_nlabels);
-            if (_PyParser_Grammar.g_ll.ll_label[a_label].lb_type == ch_type) {
+
+            const char *label_str = _PyParser_Grammar.g_ll.ll_label[a_label].lb_str;
+            if ((_PyParser_Grammar.g_ll.ll_label[a_label].lb_type == ch_type)
+                && ((ch->n_str == NULL) || (label_str == NULL)
+                     || (strcmp(ch->n_str, label_str) == 0))
+               ) {
                 /* The child is acceptable; if non-terminal, validate it recursively. */
                 if (ISNONTERMINAL(ch_type) && !validate_node(ch))
                     return 0;
@@ -679,17 +684,24 @@ validate_node(node *tree)
         /* What would this state have accepted? */
         {
             short a_label = dfa_state->s_arc->a_lbl;
-            int next_type;
             if (!a_label) /* Wouldn't accept any more children */
                 goto illegal_num_children;
 
-            next_type = _PyParser_Grammar.g_ll.ll_label[a_label].lb_type;
-            if (ISNONTERMINAL(next_type))
+            int next_type = _PyParser_Grammar.g_ll.ll_label[a_label].lb_type;
+            const char *expected_str = _PyParser_Grammar.g_ll.ll_label[a_label].lb_str;
+
+            if (ISNONTERMINAL(next_type)) {
                 PyErr_Format(parser_error, "Expected node type %d, got %d.",
                              next_type, ch_type);
-            else
+            }
+            else if (expected_str != NULL) {
+                PyErr_Format(parser_error, "Illegal terminal: expected '%s'.",
+                             expected_str);
+            }
+            else {
                 PyErr_Format(parser_error, "Illegal terminal: expected %s.",
                              _PyParser_TokenNames[next_type]);
+            }
             return 0;
         }