[Python-Dev] Int literals and method calls

["Martin v. Löwis"]

Michael Walter wrote:
> I think it's inconsistent because it works for "list literals" but not
> for "integer literals". What do I miss?

That tokenization works consistently, using the "maximum match"
strategy.

If you meant to parse

1.__class__

as "<integer 1>" "." "<identifier __class__>", not as "<float 1.0>"
"<identifier __class_>", then, for consistency, you should also parse
the second line of

   s = 100
   prints

as "<keyword print>" "<identifier s>", not as "<identifier prints>".
Since the latter is certainly undesirable, the former must be
followed for consistency.

You easily derive the rule "a space is necessary between keyword
and identifier" from the second example; you should, for consistency,
also derive the rule "a space is necessary between an integer
literal and a dot".

As for "list literals": The Python grammar calls them "displays",
not "literals", as they don't (necessarily) denote a literal value,
e.g. in [1,2,x,y+5].

Regards,
Martin