[issue26076] redundant checks in tok_get in Parser\tokenizer.c

[Oren Milman]

New submission from Oren Milman:

In Parser\tokenizer.c, in tok_get, in the identification of a potential NUMBER token, in case the token starts with a '0', the next char of the token is retrieved, followed by two redundant checks:
if (c == '.')
    goto fraction;
if (c == 'j' || c == 'J')
    goto imaginary;

These two are redundant, because they check for the case of a token starting with '0.' or '0j', but even without them, the flow in either of these would reach the code after the /* maybe old-style octal; c is first char of it */ comment.
This code (after consuming all zeros and all decimal digits) would again perform those exact two checks, and handle them exactly the same.

My proposal is simply to remove the first two checks.

I have attached the patched tokenizer.c (the redundant checks are just commented out).

----------
components: Interpreter Core
files: tokenizer.c.withoutRedundantChecks.c
messages: 257927
nosy: Oren Milman
priority: normal
severity: normal
status: open
title: redundant checks in tok_get in Parser\tokenizer.c
type: enhancement
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file41572/tokenizer.c.withoutRedundantChecks.c

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26076>
_______________________________________