[Python-checkins] gh-96678: Fix UB of null pointer arithmetic (GH-96782)

[miss-islington]

https://github.com/python/cpython/commit/81e36f350b75d2ed2668825f7df6e059b57f859c
commit: 81e36f350b75d2ed2668825f7df6e059b57f859c
branch: main
author: Matthias Görgens <matthias.goergens at gmail.com>
committer: miss-islington <31488909+miss-islington at users.noreply.github.com>
date: 2022-09-13T06:14:35-07:00
summary:

gh-96678: Fix UB of null pointer arithmetic (GH-96782)



Automerge-Triggered-By: GH:pablogsal

files:
A Misc/NEWS.d/next/Core and Builtins/2022-09-13-12-06-46.gh-issue-96678.NqGFyb.rst
M Parser/tokenizer.c

diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-09-13-12-06-46.gh-issue-96678.NqGFyb.rst b/Misc/NEWS.d/next/Core and Builtins/2022-09-13-12-06-46.gh-issue-96678.NqGFyb.rst
new file mode 100644
index 000000000000..bdd33c8d2ca9
--- /dev/null
+++ b/Misc/NEWS.d/next/Core and Builtins/2022-09-13-12-06-46.gh-issue-96678.NqGFyb.rst	
@@ -0,0 +1 @@
+Fix undefined behaviour in C code of null pointer arithmetic.
diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c
index d16af89df555..b3b11855f4ef 100644
--- a/Parser/tokenizer.c
+++ b/Parser/tokenizer.c
@@ -1533,7 +1533,7 @@ tok_get(struct tok_state *tok, const char **p_start, const char **p_end)
     } while (c == ' ' || c == '\t' || c == '\014');
 
     /* Set start of current token */
-    tok->start = tok->cur - 1;
+    tok->start = tok->cur == NULL ? NULL : tok->cur - 1;
 
     /* Skip comment, unless it's a type comment */
     if (c == '#') {