[New-bugs-announce] [issue1037] Ill-coded identifier crashes python when coding spec is utf-8

[Hye-Shik Chang]

New submission from Hye-Shik Chang:

Illegal identifier makes python crash on UTF-8 source codes/interpreters.

Python 3.0x (py3k:57555M, Aug 27 2007, 21:23:47) 
[GCC 3.4.6 [FreeBSD] 20060305] on freebsd6
>>> compile(b'#coding:utf-8\n\xfc', '', 'exec')
zsh: segmentation fault (core dumped)  ./python

The problem is that tokenizer.c:verify_identifer doesn't check
return value from PyUnicode_DecodeUTF8 but some invalid utf8
sequences could be there.

----------
components: Unicode
keywords: py3k
messages: 55335
nosy: hyeshik.chang
priority: high
severity: normal
status: open
title: Ill-coded identifier crashes python when coding spec is utf-8
type: crash
versions: Python 3.0

__________________________________
Tracker <report at bugs.python.org>
<http://bugs.python.org/issue1037>
__________________________________