[New-bugs-announce] [issue40495] compileall: option to hardlink duplicate optimization levels bytecode cache files

[Lumír Balhar]

New submission from Lumír Balhar <frenzy.madness at gmail.com>:

We would like to include a possibility of hardlink deduplication of identical pyc files to compileall module in Python 3.9. We've discussed the change [0] and tested it in Fedora RPM build system via implementation in the compileall2 module [1].

The discussion [0] contains a lot of details so I mention here only the key features:
* the deduplication can be enabled only if multiple optimization levels are processed at once
* it generates a pyc file (optimization level 0) as usual but if it finds that optimized files (optimization levels 1 and 2) have the same content, it uses hardlinks (os.link) to prevents duplicates
* the deduplication is disabled by default

We believe that this might be handy for more Pythonistas. In our case, this functionality lowers the installation size of Python 3.9 from 125 MiB to 103 MiB.

[0] https://discuss.python.org/t/compileall-option-to-hardlink-duplicate-optimization-levels-bytecode-cache-files/3014
[1] https://github.com/fedora-python/compileall2

----------
components: Library (Lib)
messages: 368022
nosy: frenzy
priority: normal
severity: normal
status: open
title: compileall: option to hardlink duplicate optimization levels bytecode cache files
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue40495>
_______________________________________