[New-bugs-announce] [issue32483] Misleading reflective behaviour due to PEP 3131 NFKC identifiers normalization.

[Iago-lito -]

New submission from Iago-lito - <iago.bonnici at gmail.com>:

Consistent with [PEP 3131](https://www.python.org/dev/peps/pep-3131/) and NFKC normalization of identifiers, these two last lines yield an error, since `𝜏` (U+1D70F) is automatically converted to `τ` (U+03C4).

    class Base(object):
        def __init__(self):
            self.𝜏 = 5 # defined with U+1D70F

    a = Base()
    print(a.𝜏)     # 5             # (U+1D70F) expected and intuitive
    print(a.τ)     # 5 as well     # (U+03C4)  normalized version, okay.
    d = a.__dict__ # {'τ':  5}     # (U+03C4)  still normalized version
    print(d['τ'])  # 5             # (U+03C4)  consistent with normalization
    assert hasattr(a, 'τ')         # (U+03C4)  consistent with normalization
    # But if I want to retrieve it the way I entered it because I can type (U+1D70F)
    print(d['𝜏'])  # KeyError: '𝜏' # (U+1D70F) counterintuitive
    assert hasattr(a, '𝜏') # Fails # (U+1D70F) counterintuitive

I've described and undestood the problem in [this post](https://stackoverflow.com/questions/48063082/).

Nothing is unconsistent here. However, I am worried that:

- this behaviour might be counterintuitive and misleading, especially if it occurs that the character user can easily enter for some reason (e.g. U+1D70F) is not equivalent to its NFKC normalization (e.g. U+03C4)

- this behaviours makes it more difficult to enjoy python's reflective `__dict__`, `hasattr` and `getattr` features in this particular case.

Maybe it is user's responsibility to be aware of this limitation, and to keep considering utf-8 coding a bad practice. In this case, maybe this particular reflective limitation could be made explicit in PEP 3131.

Or maybe it is python's responsibility to ensure intuitive and consistent behaviour even in tricky-unicode-cases. So reflective features like `__dict__.__getitem__`, `hasattr` or `getattr` would NFKC-convert their arguments before searching just like `a.𝜏` does, so that:

    getattr(a, '𝜏') is gettatr(a, 'τ')

always yields True.

I actually have no idea of the philosophy to stick to. And the only purpose of this post is to inform the community about this particular, low-priority case.

Thank you for supporting Python anyway, cheers for your patience.. and happy 2018 to everyone :)


--
Iago-lito

----------
components: Unicode
messages: 309382
nosy: Iago-lito -, ezio.melotti, vstinner
priority: normal
severity: normal
status: open
title: Misleading reflective behaviour due to PEP 3131 NFKC identifiers normalization.
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue32483>
_______________________________________