[Python-checkins] gh-104169: Ensure the tokenizer doesn't overwrite previous errors (#104170)
pablogsal
webhook-mailer at python.org
Thu May 4 10:15:33 EDT 2023
https://github.com/python/cpython/commit/eba64d2afb4c429e80d863dc0dd7808bdbef30d3
commit: eba64d2afb4c429e80d863dc0dd7808bdbef30d3
branch: main
author: Pablo Galindo Salgado <Pablogsal at gmail.com>
committer: pablogsal <Pablogsal at gmail.com>
date: 2023-05-04T15:15:26+01:00
summary:
gh-104169: Ensure the tokenizer doesn't overwrite previous errors (#104170)
files:
M Parser/tokenizer.c
diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c
index 7c07d2011fda..52d0d9a534cb 100644
--- a/Parser/tokenizer.c
+++ b/Parser/tokenizer.c
@@ -1277,6 +1277,12 @@ _syntaxerror_range(struct tok_state *tok, const char *format,
int col_offset, int end_col_offset,
va_list vargs)
{
+ // In release builds, we don't want to overwrite a previous error, but in debug builds we
+ // want to fail if we are not doing it so we can fix it.
+ assert(tok->done != E_ERROR);
+ if (tok->done == E_ERROR) {
+ return ERRORTOKEN;
+ }
PyObject *errmsg, *errtext, *args;
errmsg = PyUnicode_FromFormatV(format, vargs);
if (!errmsg) {
More information about the Python-checkins
mailing list