Using PLY

[Maurice LING]

> 
> Here's a kludgy but quick solution- modify the LexToken class in
> lex.py to keep track of number of type occurences.
> 
> class LexToken(object):  # change to new style class
>     type_count = {}   # store the count here
>     def __setattr__(self, key, value):
>         if key == 'type':
>             # when type attribute is assigned, increment counter
>             if value not in self.type_count:
>                 self.type_count[value] = 1
>             else:
>                 self.type_count[value] += 1
>         object.__setattr__(self, key, value)
> 
>     # ... and proceed with the original definition of LexToken
> 
>     def __str__(self):
>         return "LexToken(%s,%r,%d)" %
> (self.type,self.value,self.lineno)
>     def __repr__(self):
>         return str(self)
>     def skip(self,n):
>         try:
>             self._skipn += n
>         except AttributeError:
>             self._skipn = n
> -----------------------------------------
> 
> After you've run the lexer, lex.LexToken.type_count will the contain
> number of occurences of each token type.
> 
> -----------------------------------------
> 
> (Caveats-  1. I haven't tested this code.   2. I've got PLY 1.3;
> syntax may have changed in newer versions.  In fact, I hope it's
> changed; while PLY works very well, its usage could be way more
> pythonic)

Thank you, it works well. I think this should be included in the next 
release.

I am able to do a "print lex.LexToken.type_count" after each token and 
it did show the incremental numbers of each tokens, except t_ignore.

Thanks again
maurice