New to Python - block grouping (spaces)

[Ron Adam]

On 04/19/2015 05:42 PM, BartC wrote:
>
> So I'm aware of some of the things that are involved.
>
> (BTW that project worked reasonably well, but I decided to go in a
> different direction: turning "J" from a mere syntax into an actual language
> of its own.)

Something you might try with your new language is to have an editor that 
can parse the source text to a tokenised "text" data file.  It should be a 
fairly direct text to text translation, so putting it into the editor 
should be doable.

Once you get that far, you can add pluggable token parsers to the editor 
that can unparse the token files to a specific user format for editing, and 
reparse it back to the standardised token file for storage.  Now the users 
can choose a customised dialect of your language.  And the compiler will 
work on a single token file type.

This will also remove the need for pretty printers and formatters as they 
will be built into the editors pluggable token parser.  Just have the 
editor tokenise and then immediately untokenise and you just checked for 
syntax errors and reformatted the program.   The highlighter could also 
work with the token parser as well.  The only catch is how to handle 
compile and run time errors.  The editor will need to read an error file 
and translate it back to the source.  I think it's doable but it will be 
tricky to get it to work well.

The reason to make the split at tokens is, it's at a stage that most of 
what is needed is already in a text editor.

The token file format becomes the bridge that spans the gap between the 
user/editor and the compiler/interpreter.

Then your compiler/interpreter is all implementation details that work on a 
single standardised token file format rather than unformatted text files.

Ron