[pypy-svn] r65775 - pypy/branch/parser-compiler/pypy/interpreter/pyparser

[benjamin at codespeak.net]

Author: benjamin
Date: Sun Jun 14 21:35:10 2009
New Revision: 65775

Modified:
   pypy/branch/parser-compiler/pypy/interpreter/pyparser/pytokenizer.py
Log:
pass the operator as the token's value

Modified: pypy/branch/parser-compiler/pypy/interpreter/pyparser/pytokenizer.py
==============================================================================
--- pypy/branch/parser-compiler/pypy/interpreter/pyparser/pytokenizer.py	(original)
+++ pypy/branch/parser-compiler/pypy/interpreter/pyparser/pytokenizer.py	Sun Jun 14 21:35:10 2009
@@ -221,7 +221,6 @@
                                              lnum-1, 0, token_list)
                     if token in python_opmap:
                         punct = python_opmap[token]
-                        token = ""
                     else:
                         punct = tokens.OP
                     token_list.append((punct, token, lnum, pos, line))