How to in Python

[Chris Mellon]

On Dec 21, 2007 1:02 PM,  <MartinRinehart at gmail.com> wrote:
>
>
> Chris Mellon wrote:
> > Is there some reason that you think Python is incapable of
> > implementing lexers that do this, just because Python lexer accepts
> > it?
>
> Absolutely not. My opinion is that it's a bug. A very, very minor bug,
> but still six-legged.
>

I understand the argument but I'm wondering why you're asking
questions on this list about it. You don't seem to be implementing the
lexer in Python (because otherwise the question wouldn't have come
up), and you don't seem to be parsing Python code. Where's the python
angle? Not that this list is totally intolerant of offtopic
discussions, but you don't seem to even have a hint of Python interest
here - the questions seem to be better suited for comp.parsers.general
or something.

> > Note that if you're using your lexer to mark up or pretty print or
> > whatever Python source, it's wrong - 0x is (rightly or not) a valid
> > Python literal.
>
> My lexer is for my language, Decaf, which, in this particular, is the
> same as Python. Here's what I find at at python.org/ref: (2.4.4).
>
>     hexinteger ::= "0" ("x" | "X") hexdigit+
>
> Implementation differs from specification. In this case, I think the
> spec is more sensible.
>

I tend to consider "what the parser actually accepts" rather than
"what the grammar specifies" to be normative when writing pretty
printers or syntax highlighters.