[Tutor] tokenize row numbering

Chris Smith smichr at bigfoot.com
Sat Apr 30 01:01:00 CEST 2005


I got over my intimidation with tokenize and learned to use it to strip 
comments from code files.  In the process, I learned that the tuples 
that it returns for the position in the code of various tokens is 
1-based in the row rather than 0-based, so the tuple pair (3,1),(3,2) 
would be the start and stop position of the "=" OP in the following 
code:

###
# line 1 according to tokenize tuple
# line 2
a=b #line 3
###

Does anyone have an idea of *why* the rows/physical lines of code 
beginning their count at 1 instead of 0? In order to process the code I 
either have to subtract 1 from all the first elements of tuple 
positions or else insert a blank line at the beginning of the list that 
I make out of my code when I want to process it.

Is this a candidate for a feature change request?
/c



More information about the Tutor mailing list