[issue27364] Deprecate invalid unicode escape sequences

[Emanuel Barry]

Emanuel Barry added the comment:

I originally considered making two different patches, so there you go. deprecate_invalid_escapes_only_1.patch has the deprecation plus a test, and invalid_stdlib_escapes_1.patch fixes all invalid escapes in the stdlib.

My code was the cause, although no directly; it was 'assert(!PyErr_Occurred())' at the beginning of PyObject_Call in Objects/abstract.c which failed.

This happened when I ran the whole test suite (although just running test_ast was fine to reproduce it) with the '-W error' command line switch. One stdlib module (I don't remember which one) had one single invalid escape sequence in it, and then test_ast.ASTValidatorTests.test_stdlib_validates triggered the failed assertion. Fixing the invalid escape removes the failure and all tests pass.

One can reliably reproduce the crash with the patch by adding a string with an invalid escape in any of the stdlib files (and running with '-W error'):

No invalid sequence:

>>> import unittest, test.test_ast
>>> unittest.main(test.test_ast)
..............................................................................
----------------------------------------------------------------------
Ran 78 tests in 5.538s

OK

With an invalid sequence in a file:

>>> import unittest, test.test_ast
>>> unittest.main(test.test_ast)
............................................Fatal Python error: a function returned a result with an error set
DeprecationWarning: invalid escape sequence 'w'

During handling of the above exception, another exception occurred:

SystemError: <built-in function compile> returned a result with an error set

Current thread 0x00001ba0 (most recent call first):
  File "E:\GitHub\cpython\lib\ast.py", line 35 in parse
  File "E:\GitHub\cpython\lib\test\test_ast.py", line 944 in test_stdlib_validates
  File "E:\GitHub\cpython\lib\unittest\case.py", line 600 in run
  File "E:\GitHub\cpython\lib\unittest\case.py", line 648 in __call__
  File "E:\GitHub\cpython\lib\unittest\suite.py", line 122 in run
  File "E:\GitHub\cpython\lib\unittest\suite.py", line 84 in __call__
  File "E:\GitHub\cpython\lib\unittest\suite.py", line 122 in run
  File "E:\GitHub\cpython\lib\unittest\suite.py", line 84 in __call__
  File "E:\GitHub\cpython\lib\unittest\runner.py", line 176 in run
  File "E:\GitHub\cpython\lib\unittest\main.py", line 255 in runTests
  File "E:\GitHub\cpython\lib\unittest\main.py", line 94 in __init__
  File "<stdin>", line 1 in <module>

Then I get the usual "Python has stopped working" Windows prompt (strangely enough, before I'd get a prompt saying "Assertion failed" with the line, but not this time).

I'm not sure where the error lies exactly. Should I open another issue for that?

----------
Added file: http://bugs.python.org/file43549/deprecate_invalid_escapes_only_1.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27364>
_______________________________________