[Python-checkins] [python/cpython] fc354f: bpo-25324: copy tok_name before changing it (#1608...

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/python/cpython
  Commit: fc354f07855a9197e71f851ad930cbf5652f9160
      https://github.com/python/cpython/commit/fc354f07855a9197e71f851ad930cbf5652f9160
  Author: Albert-Jan Nijburg <albertjan at trinket.io>
  Date:   2017-05-31 (Wed, 31 May 2017)

  Changed paths:
    M Doc/library/token.rst
    M Include/token.h
    M Lib/test/test_tokenize.py
    M Lib/token.py
    M Lib/tokenize.py
    M Misc/NEWS
    M Parser/tokenizer.c

  Log Message:
  -----------
  bpo-25324: copy tok_name before changing it (#1608)

* add test to check if were modifying token

* copy list so import tokenize doesnt have side effects on token

* shorten line

* add tokenize tokens to token.h to get them to show up in token

* move ERRORTOKEN back to its previous location, and fix nitpick

* copy comments from token.h automatically

* fix whitespace and make more pythonic

* change to fix comments from @haypo

* update token.rst and Misc/NEWS

* change wording

* some more wording changes