[New-bugs-announce] [issue22616] Allow connecting AST nodes with corresponding source ranges

[Aivar Annamaa]

New submission from Aivar Annamaa:

Currently lineno and col_offset attributes in AST nodes have confusing roles. According to documentation they indicate the starting position of node's source text but according to recent developments (#16795) they seem to indicate a position most suitable to use in error messages related to that node (rather narrow goal IMO).

Another problem is that currently the AST nodes don't contain any information about the end position of corresponding source text. Therefore it's very difficult to relate nodes with source. One could want to do this for example in advanced graphical debuggers (https://bitbucket.org/plas/thonny)

I propose adding new attributes to AST nodes which indicate the corresponding source range. If you want to keep nodes lightweight by default, then you could also introduce a flag in ast.parse for getting these attributes.

The range could be given either in token indices or in character positions (or both). This probably needs more discussion. (I would vote against pointers to UTF-8 bytes, as is the case with col_offset currently.)

----------
components: Interpreter Core
messages: 229124
nosy: Aivar.Annamaa
priority: normal
severity: normal
status: open
title: Allow connecting AST nodes with corresponding source ranges
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22616>
_______________________________________