[issue3353] make built-in tokenizer available via Python C API
Pablo Galindo Salgado
report at bugs.python.org
Wed Jan 27 05:49:44 EST 2021
Pablo Galindo Salgado <pablogsal at gmail.com> added the comment:
I am -1 exposing the C-API of the tokenizer. For the new parser several modifications of the C tokenizer had to be done and some of them modify existing behaviour slightly. I don't want to corner ourselves in a place where we cannot make improvements because is a backwards incompatible change because the API is exposed.
----------
nosy: +pablogsal
_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue3353>
_______________________________________
More information about the Python-bugs-list
mailing list