[issue41842] Add codecs.unregister() to unregister a codec search function

[STINNER Victor]

New submission from STINNER Victor <vstinner at python.org>:

Writing an unit test on the Python codecs machinery is facing a practical problem: there is no C nor Python API to unregister a codec search function.

It's even documented in a note of the codecs.register() function:

"Note: Search function registration is not currently reversible, which may cause problems in some cases, such as unit testing or module reloading."

https://docs.python.org/dev/library/codecs.html#codecs.register

test_codecs contains a long comment about that:

        # There's no way to unregister a codec search function, so we just
        # ensure we render this one fairly harmless after the test
        # case finishes by using the test case repr as the codec name
        # The codecs module normalizes codec names, although this doesn't
        # appear to be formally documented...
        # We also make sure we use a truly unique id for the custom codec
        # to avoid issues with the codec cache when running these tests
        # multiple times (e.g. when hunting for refleaks)

See bpo-22166 which fixed memory leaks in test_codecs.

In 2011, a Python user requested the function
https://mail.python.org/pipermail/python-dev/2011-September/113588.html

Marc-Andre Lemburg explained:

"There is no API to unregister a codec search function, since deregistration
would break the codec cache used by the registry to speedup codec
lookup."

One simple solution would be to clear the cache (PyInterpreterState.codec_search_cache) when codecs.unregister() removes a search function. I expect that calling unregister() is an uncommon operation, so the performance is not a blocker issue.

----------
components: Library (Lib), Unicode
messages: 377377
nosy: ezio.melotti, vstinner
priority: normal
severity: normal
status: open
title: Add codecs.unregister() to unregister a codec search function
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue41842>
_______________________________________