Removing comments... tokenize error

[qwweeeit]

Thanks! If you answer to my posts one more time I could  consider you as 
my tutor...

It was strange to have found a bug...! In any case I will not go deeper 
into the matter, because for me it's enough your explanatiom.
I corrected the problem by hand removing the tokens spanning multiple lines
(there were only 8 cases...).

Instead I haven't understood your hint about comments...
I succeded  in realizing a python script which removes comments.

Here it is (in all its cumbersome and criptic appearence!...):
    
# removeCommentsTok.py   
import tokenize
Input = "pippo1"    
Output = "pippo2"    
f = open(Input)
fOut=open(Output,"w")

nLastLine=0
for i in tokenize.generate_tokens(f.readline): 
.   if i[0]==52 and nLastLine != (i[2])[0]:
.   .   fOut.write((i[4].replace(i[1],'')).rstrip()+'\n')
.   .   nLastLine=(i[2])[0]
.   elif i[0]==4 and nLastLine != (i[2])[0]:
.   .   fOut.write((i[4]))
.   .   nLastLine=(i[2])[0]
f.close()
fOut.close()

Some explanations for the guys like me...:
- 52 and 4 are the arbitrary codes for comments and NEWLINE respectively
- the comment removing is obtained by clearing the comment (i[1]) in the 
  input line (i[4])
- I also right trimmed the line to get rid off the remaining blanks.