Tokens?

[BBands]

In the invaluable 'Dr. Dobb's Python-URL! - weekly Python news and
links' of April 17 Peter Otten writes: "Michele Simionato's little
script lets you search for a name in Python scripts, avoiding false
positives that a standard tool like grep would yield." Can someone
explain why this is so? I have attached the script below.

import sys, tokenize, token

def count_name(name, script):
    "Count the occurrences of a Python name in a script"
    counter = 0
    for tok_code, tok_value, (srow, scol), (erow, ecol), line in \
            tokenize.generate_tokens(file(script).readline):
        if tok_code == token.NAME and tok_value == name:
            counter += 1
            print 'line %s: %s' %(srow, line),
    if counter: print '*** %s ***\n' % script
    return counter

if __name__ == '__main__':
    name = sys.argv[1]
    scripts = sys.argv[2:]
    total = sum(count_name(name, script) for script in scripts)
    print 'Found %d occurrences of %r' % (total, name)

    jab--who laments the day that Doctor Dobbs' Journal of Computer
Callisthenics and Orthodontics changed its name.