[Python-bugs-list] [ python-Bugs-456175 ] add dictionary "tokens" to token.py
noreply@sourceforge.net
noreply@sourceforge.net
Tue, 28 Aug 2001 10:56:43 -0700
Bugs item #456175, was opened at 2001-08-28 08:48
You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=456175&group_id=5470
Category: Python Library
Group: Feature Request
Status: Open
Resolution: None
Priority: 5
Submitted By: Grant Griffin (dspguru)
Assigned to: Nobody/Anonymous (nobody)
>Summary: add dictionary "tokens" to token.py
Initial Comment:
I would like for token.py to provide a dictionary,
called "tokens" that maps the name of each token to
its value, e.g.:
token.tokens[token.INDENT] is 'INDENT'
This would be useful for debugging programs that use
tokenize.
A user currently could create a dictionary like this
using introspection on token.py (assuming tokens are
in all caps) but that's a lot of trouble. Instead, I
believe it would be better to solve the problem once-
and-for-all by building the extra logic required to
create the dictionary into token.main.
thanks much,
=g2
----------------------------------------------------------------------
>Comment By: Grant Griffin (dspguru)
Date: 2001-08-28 10:56
Message:
Logged In: YES
user_id=70844
Thanks much, Skip. That's not a bad idea, but here's what
I ended up doing:
def token_names():
import tokenize
res = {}
names = dir(tokenize)
for name in names:
attr = getattr(tokenize, name)
if type(attr) is type(1):
res[attr] = name
return res
Instead of modifying token.py as I had suggested
previously, perhaps something like the above could be added
to tokenize.
- There isn't much code here, but it took me awhile to
figure out how to include my own batteries <wink>.
- I learned the hard way that it's better to get these
symbols from tokenize than token because tokenize adds
the "COMMENT" token.)
thanks,
=g2
----------------------------------------------------------------------
Comment By: Skip Montanaro (montanaro)
Date: 2001-08-28 09:42
Message:
Logged In: YES
user_id=44345
I encountered the same problem when programming with Gtk.
Gtk defines an enormous number of enumeration types. While
I could define one or more dictionaries to map enumerations
back to names as you suggested, I chose a somewhat different
approach. I defined a ConstantMap class whose instances
create a new dictionary containing all non-instance/non-
class objects created since the last ConstantMap class was
instantiated.
class ConstantMap:
markdict = {}
def __init__(self):
"""add new global symbols to instance's name map"""
self.names = {}
import types
for key,val in globals().items():
if (not self.markdict.has_key(key) and
key[:2] != "__" and
type(val) not in [types.InstanceType,
types.ClassType]):
self.names[val] = key
self.markdict.update(globals())
def __call__(self, key):
"""return a name mapping"""
return self.names[key]
def mark(self):
"""checkpoint current globals"""
I then instantiate ConstantMap after each block of
enumeration constants are defined like so:
# GdkPropertyState's
PROPERTY_NEW_VALUE = 0
PROPERTY_DELETE = 1
GdkPropertyState = ConstantMap()
# GdkWindowState's
WINDOW_STATE_WITHDRAWN = 1 << 0
WINDOW_STATE_ICONIFIED = 1 << 1
WINDOW_STATE_MAXIMIZED = 1 << 2
WINDOW_STATE_STICKY = 1 << 3
GdkWindowState = ConstantMap()
# GdkSettingAction's
SETTING_ACTION_NEW = 0
SETTING_ACTION_CHANGED = 1
SETTING_ACTION_DELETED = 2
GdkSettingAction = ConstantMap()
# GdkScrollDirection's
SCROLL_UP = 0
SCROLL_DOWN = 1
SCROLL_LEFT = 2
SCROLL_RIGHT = 3
GdkScrollDirection = ConstantMap()
This is more than you're asking for in this case, but may
be a somewhat more general foundation for a problem that
bugs many people debugging code that uses imported
constants.
Skip
----------------------------------------------------------------------
You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=456175&group_id=5470