[issue41566] Include much faster DEFLATE implementations in Python's gzip and zlib libraries. (isa-l)

[Gregory P. Smith]

Gregory P. Smith <greg at krypto.org> added the comment:

fwiw, no PEP is needed for things like this.  it'd just be an alternative library implementing the core of the zlib and/or gzip modules behind the scenes.

normally this kind of thing would be done using a check for the availability of the library in configure.ac with conditional compilation via #ifdef's from the configure output that controls what is set in pyconfig.h in the zlib and gzip modules itself.

Within the stdlib, I'd focus only on using things that can be used in a 100% api compatible way with the existing modules.

Otherwise creating a new module and putting it up on PyPI to expose the functionality from the libraries you want makes sense and will be easier to make available to everyone on existing Python versions rather than waiting on getting something into CPython.

FWIW I tend to avoid software provided by Intel given any other choice.

Look instead at Chromium's zlib as discussed in https://github.com/madler/zlib/issues/346 or possibly at a project like https://github.com/zlib-ng/zlib-ng.  These are much more likely to be drop in zlib replacements making this simply a build time thing which is more a matter of making sure we get the relevant compiler flags correct when the faster libraries are detected by configure.ac as available.

There is a caveat to using any of these: how well maintained and security vetted are all of the code paths in the implementation?  zlib proper gets massive security attention.  Its low rate of change and staleness are a feature.  Chromium's zlib also gets proper security attention.  No idea about zlib-ng and even less about Intels self serving arm-ignoring oddity.

----------
nosy: +gregory.p.smith
stage: patch review -> needs patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue41566>
_______________________________________