[Python-ideas] Enhancing Zipapp

[Brett Cannon]

Thanks for the ideas, Abdur-Rahmaan! Some feedback below.

On Mon, Jan 6, 2020 at 11:35 AM Abdur-Rahmaan Janhangeer <
arj.python at gmail.com> wrote:

> Note: draft simplified
>
> Abstract
> ======
>
> This extracts aims at proposing enhancements to the generated zipapp
> executable
>
> Rationale
> =======
>
> One area where there remains some difficulty in Python is packaging for
> end-user consumption. To that effect either the code is distributed in pure
> Python form with installers [1] or native executables are built for each
> target
> Os [2]. Currently by default, Python does not provide such utilities. This
> pro-
> posal aims at finalising a Python-specific archive as the default VM exec-
> utable built on zipapp.
>
> In simple terms, it proposes to enhance zipapp from plain archive to
> app-level
> archive.
>
> Advantages of archives
> ==================
>
> Archives provide a great way to publish software that needs to be
> distributed as
> a single file script but is complex enough to need to be written as a
> collection of
> modules [3]
>
> You can use archives for tasks such as lossless data compression,
> archiving,
> decompression, and archive unpacking. [4] Adding capabilities like digital
> signing is used to verify integrity and authenticity.
>
> Zip archives as apps
> ================
>
> If we are to treat zip archives as app, here are some recommended features
>
> - [x] Main entry point
>
> A main entry point specifies which file to launch. Zipapp already solves
> this
> problem by either having a __main__.py [5] or specifying the entry point
> at the
> commandline ENTRYPOINT_MODULE:ENTRYPOINT_FUNCTION [6]
>
> - [  ] App info file
>
> An info file can have info such as author name, archiving date, company
> name
> etc.
>

This would be a packaging detail so not something to be specified in the
stdlib.


>
> - [  ] Signing mechanism
>
> Mechanisms can be added to detect the integrity of the app. App hash can
> be
> used to check if the app has been modified and per-file hash can be used
> to
> detect what part has been modified. This can be further enhanced if needed.
>
> - [  ] Protecting meta data
>
> Metadata are not protected by basic signing. There existing ways to protect
> metadata and beyond [7]
>

This can be tricky because people want signing in specific ways that vary
from OS to OS, case by case. So unless there's a built-in signing mechanism
the flexibility required here is huge.


>
>  - [x] Pure-Python 3rd party package bundling
>
> In Python, as long as the 3rd party packages are pure-python packages, we
> can bundle and use them [6]. The user can maybe just include a
> requirements.txt
>
> - [  ] C-based 3rd party packages
>
> Zipapp by default was not meant to include packages at all.
>
> << The executable zip format is specifically designed for standalone use,
> without needing to be installed. They are in effect a multi-file version
> of a
> standalone Python script >>
>
> Though the previous point shows that this can be done. Now remains the
> issue of C-based packages. Distributing wheels might be the answer [8].
> A zip archive is supposed to be standalone. A possible  solution might be
> to
> include wheels and the wheels are installed in a site-packages folder.
>
> When running such an app, the interpreter will check first if the
> app-specific
> site-packages folder is empty, if not, install the wheels. This provides
> package-
> freezing ability. The only downside is longer first-run time.
>

Install the wheels where? You can't do that globally. And you also have to
worry about the security of doing the install implicitly. And now the user
suddenly has stuff on their file system they may not have asked for as a
side-effect which may upset some people who are tight on disk space
(remember that Python runs on some low-powered machines).

-Brett


>
> Only specifying packages to be installed is not an option as if you really
> want
> stand-alone apps, using the internet etc defeats the purpose.
>
> FAQ
> ====
>
> Why not a package manager?
> ---------------------------------------
> The zipapp pep was introduced for a reason, for easing the running of
> archives.
> Maybe the package manager idea came from listening to people talking about
> packaging and pex and comparing it with package-managers like homebrew
> and concluded that pex and hence zipapp is not worth it and people would
> better off not complicate their lives with some zip utility.
>
> This proposal is not solving any problem at all
> --------------------------------------------------------------
> This proposal aims at enhancing zipapp. Zipapp solved the problem. Zipapp
> had an aim. This proposal aims at helping zipapp better accompplish it's
> aim.
>
>
> References
>
> [1] https://pynsist.readthedocs.io/en/latest/
>
> [2] https://www.pyinstaller.org
>
> [3] https://www.python.org/dev/peps/pep-0441/
>
> [4]
> https://docs.oracle.com/javase/tutorial/deployment/jar/basicsindex.html
>
> [5] https://docs.python.org/3/library/zipapp.html
>
> [6] https://gist.github.com/lukassup/cf289fdd39124d5394513a169206631c
>
> [7] https://source.android.com/security/apksigning
>
> [8] https://pythonwheels.com
>
>
>
> Yours,
>
> Abdur-Rahmaan Janhangeer
> pythonmembers.club <http://www.pythonmembers.club/> | github
> <https://github.com/Abdur-rahmaanJ>
> Mauritius
> _______________________________________________
> Python-ideas mailing list -- python-ideas at python.org
> To unsubscribe send an email to python-ideas-leave at python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/EMP4PMT7AHUJPLYD2DN4MXKCMGDK3QSO/
> Code of Conduct: http://python.org/psf/codeofconduct/
>