[Python-Dev] Inclusion of lz4 bindings in stdlib?

[Antoine Pitrou]

On Wed, 28 Nov 2018 19:35:31 +0000
Jonathan Underwood <jonathan.underwood at gmail.com> wrote:
> On Wed, 28 Nov 2018 at 18:57, Antoine Pitrou <solipsis at pitrou.net> wrote:
> >
> > On Wed, 28 Nov 2018 10:43:04 -0800
> > "Gregory P. Smith" <greg at krypto.org> wrote:  
> [snip]
> > > I don't think adding lz4 to the stdlib is worthwhile.  It isn't required
> > > for core functionality as zlib is (lowest common denominator zip support).  
> >
> > Actually, if some people are interested in compressing .pyc files, lz4
> > is probably the best candidate (will yield significant compression
> > benefits with very little CPU overhead).
> >  
> 
> It's interesting to note that there's an outstanding feature request
> to enable "import modules from a library.tar.lz4", justified on the
> basis that it would be helpful to the python-for-android project:
> 
> https://github.com/python-lz4/python-lz4/issues/45

Interesting.  The tar format isn't adequate for this: the whole tar
file will be compressed at once, so you need to uncompress it all even
to import a single module.  The zip format is more adapted, but it
doesn't seem to have LZ4 in its registered codecs.

At least for pyc files, though, this could be done at the marshal level
rather than at the importlib level.

Regards

Antoine.