[New-bugs-announce] [issue42086] AST: Document / re-design? the simple constructor nodes from sums

[Batuhan Taskaya]

New submission from Batuhan Taskaya <isidentical at gmail.com>:

Simple constructors (basically constructors with no fields) are currently 'cached' / pre-created and dispatched. What I mean by that is, when a user parses some code with `ast.parse()` and gets a tree object, all simple constructors with the same value would point to the same object.

>>> mod1 = ast.parse("a + b", mode="eval")
>>> mod1.body.left.ctx
<ast.Load object at 0x7f8695321d20>
>>> mod1.body.right.ctx
<ast.Load object at 0x7f8695321d20>
>>> mod1.body.left.ctx is mod1.body.right.ctx
True
>>> mod1.body.left.ctx.some_annotation_that_my_extra_process_tool_puts = 1
>>> mod1.body.right.ctx.some_annotation_that_my_extra_process_tool_puts
1

Even though I have no real evidence that, this was done on purpose, I believe this is some sort of 'enum' replication (caching singletons but not really, since this is only valid for the results of ast.parse)
>>> mod1.body.right.ctx is ast.Load()
False
>>> ast.parse("a + b", mode="eval").body.left.ctx is ast.parse("c + d", mode="eval").body.right.ctx
True



Obviously, we can not change these into enums like (ast.expr_ctx.Load / LOAD) since it would break theoretically most of the code that works with ast. So here is a tl;dr:

- Even though all ast objects are mutable (by default), we use the same objects when converting C AST into Python AST. So that mutations on one object is affecting the rest of the tree. 

- We can either;
  - Document this behavior and keep it
  - Return freshly constructed objects when converting C AST into Python AST
    This has a very slight (that I suspect no body uses) risk of breaking code, but as I implied, very slight. Which would occur in such a case

import ast
from collections import defaultdict

def collect_contexts(tree):
    contexts = defaultdict(list)
    for node in ast.walk(tree):
        if isinstance(node, ast.Name):
            contexts[node.ctx].append(node.id)
    return contexts
    
print(collect_contexts(ast.parse("a, b = c, d")))

This code can easily (and it makes it more reliable/robust) refactored into 
-             contexts[node.ctx].append(node.id)
-             contexts[type(node.ctx)].append(node.id)
but just mentioning in case of any question appears about backwards incompatability.

----------
components: Library (Lib)
messages: 379004
nosy: BTaskaya, lys.nikolaou, pablogsal, serhiy.storchaka
priority: normal
severity: normal
status: open
title: AST: Document / re-design? the simple constructor nodes from sums
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42086>
_______________________________________