Significant whitespace

[Greg Ewing (using news.cis.dfn.de)]

Marcin 'Qrczak' Kowalczyk wrote:
> Having failed with significant indents, I tried to use significant line
> breaks in next incarnations of my language, which looked like a good
> compromise. ... I had
> troubles with designing a good syntax for some constructs, mainly
> if-then-else, being constrained to syntaxes which can be nicely split
> into lines.

While pondering that sort of problem one day I devised a scheme for
handling syntaxes where some line breaks are required and others are
optional.

The idea is that instead of the scanner returning line breaks as
tokens of their own, there is a flag associated with the current
token that indicates whether it is followed by a line break.

When parsing something that doesn't care whether it has line
breaks in it or not, the parser ignores this flag. But when it
gets to a point where a line break could be significant, it takes
notice of it.

For instance, when parsing

    a = b +
       c

the parser would ignore the line break flag while looking at
the '+' because it knows there must be more of the expression
coming. But when it gets to the 'c', it would check the line
break flag, find it set, and conclude that the expression is
finished.

This ought to allow extra line breaks to be placed wherever
they don't lead to any ambiguity.

I haven't had a chance to try this out, however, so there
may be some problems that I haven't anticipated!

-- 
Greg Ewing, Computer Science Dept,
University of Canterbury,	
Christchurch, New Zealand
http://www.cosc.canterbury.ac.nz/~greg