[Python-checkins] CVS: python/dist/src/Lib/test tokenize_tests.py,1.1,1.2

[Ka-Ping Yee]

Update of /cvsroot/python/python/dist/src/Lib/test
In directory usw-pr-cvs1:/tmp/cvs-serv29727/test

Modified Files:
	tokenize_tests.py 
Log Message:
Add tokenizer support and tests for u'', U"", uR'', Ur"", etc.


Index: tokenize_tests.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/tokenize_tests.py,v
retrieving revision 1.1
retrieving revision 1.2
diff -C2 -r1.1 -r1.2
*** tokenize_tests.py	1997/10/27 22:15:03	1.1
--- tokenize_tests.py	2001/01/15 22:04:30	1.2
***************
*** 111,114 ****
--- 111,128 ----
  """ + R'''spam
  '''
+ x = u'abc' + U'ABC'
+ y = u"abc" + U"ABC"
+ x = ur'abc' + Ur'ABC' + uR'ABC' + UR'ABC'
+ y = ur"abc" + Ur"ABC" + uR"ABC" + UR"ABC"
+ x = ur'\\' + UR'\\'
+ x = ur'\'' + ''
+ y = ur'''
+ foo bar \\
+ baz''' + UR'''
+ foo'''
+ y = Ur"""foo
+ bar \\ baz
+ """ + uR'''spam
+ '''
  
  # Indentation