[issue43014] tokenize spends a lot of time in `re.compile(...)`

[Steven D'Aprano]

Steven D'Aprano <steve+python at pearwood.info> added the comment:

Just for the record:

> The optimization takes the execution from ~6300ms to ~4500ms on my machine (representing a 28% - 39% improvement depending on how you calculate it)

The correct answer is 28%, which uses the initial value as the base: (6300-4500)/6300 ≈ 28%. You are starting at 6300ms and speeding it up by 28%:

>>> 6300 - 28/100*6300
4536.0

Using 4500 as the base would only make sense if you were calculating a slowdown from 4500ms to 6300ms: we started at 4500 and *increase* the time by 39%:

>>> 4500 + 39/100*4500
6255.0


Hope this helps.

----------
nosy: +steven.daprano

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43014>
_______________________________________