invalid-token syntax hook (was Re: Hack request: rational numbers)

[Neelakantan Krishnaswami]

On Sun, 28 Jan 2001 10:26:03 +0100, Alex Martelli <aleaxit at yahoo.com> wrote:
>"Tim Peters" <tim.one at home.com> wrote in message
>news:mailman.980590234.29036.python-list at python.org...
>    [snip]
>> That said, I believe your only hope of getting new literal syntax is to
>> embed it in a general proposal.  Some notation for rational literals is
>> just one of an unbounded number of similar desires.  
>
> What I was thinking, too.  So what about the "invalid-token syntax
> error hook" idea -- whenever the compiler is about to raise a
> SyntaxError because of an invalid token ('3r','@@x',etc), if
> sys.invalidTokenHook (or something of that ilk) is set to some
> callable, that callable is invoked, passed the invalid-token string,
> and given a chance to either return None (confirming that, yes, it
> IS a syntax error) or a tuple (possibly empty or singleton) of
> tokens to be used instead.

Insufficiently general, IMO.If you make exceptions restartable (ala
Lisp and Smalltalk) in Python, and then you get the hook into
SyntaxError for free, without needing special-purpose machinery.


Neel