[Python-Dev] Support of the Android platform
Victor Stinner
victor.stinner at gmail.com
Wed Dec 13 16:56:10 EST 2017
Hi Xavier,
I looked at your scripts to build Android but I failed to use them.
Anyway, I'm not sure why these scripts have to be part of the CPython
git repository.
Technically, is there a reason to put it aside the source code and
Unix build scripts (configure/Makefile/setup)?
Your https://github.com/python/cpython/pull/1629 only adds new files
without touching existing files.
I suggest to create new Git project. It may be in the python
organization, or you may start with your GitHub account.
Cross-compilation is hard, and I'm not sure that it's possible to
build a single recipe for all Android API versions, all configuration,
any set of libraries, etc. For Android, it seems like each developer
might want a subtle different configuration which might not be easy to
support.
Having a separated Git project would allow people to contribute more
easily, experiment their fork, etc.
What do you think?
I'm only talking about the proposed Android/ directory and
https://github.com/python/cpython/pull/1629. Everything else is fine
:-)
Victor
2017-12-10 15:19 GMT+01:00 Xavier de Gaye <xdegaye at gmail.com>:
> The following note is a proposal to add the support of the Android platform.
>
> The note is easier to read with clickable links at
> https://github.com/xdegaye/cagibi/blob/master/doc/android_support.rst
>
> Motivations
> ===========
>
> * Android is ubiquitous.
> * This would be the first platform supported by Python that is
> cross-compiled,
> thanks to many contributors.
> * Although the Android operating system is linux, it is different from most
> linux platforms, for example it does not use GNU libc and runs SELinux in
> enforcing mode. Therefore supporting this platform would make Python more
> robust and also would allow testing it on arm 64-bit processors.
> * Python running on Android is also a handheld calculator, a successor of
> the
> slide rule and the `HP 41`_.
>
> Current status
> ==============
>
> * The Python test suite succeeds when run on Android emulators using
> buildbot
> strenuous settings with the following architectures on API 24: x86,
> x86_64,
> armv7 and arm64.
> * The `Android build system`_ is described in another section.
> * The `buildmaster-config PR 26`_ proposes to update ``master.cfg`` to
> enable
> buildbots to run a given Android API and architecture on the emulators.
> * The Android emulator is actually ``qemu``, so the test suites for x86 and
> x86_64 last about the same time as the test suite run natively when the
> processor of the build system is of the x86 family. The test suites for
> the
> arm architectures last much longer: about 8 hours for arm64 and 10 hours
> for
> armv7 on a four years old laptop.
> * The changes that have been made to achieve this status are listed in
> `bpo-26865`_, the Android meta-issue.
> * Given the cpu resources required to run the test suite on the arm
> emulators,
> it may be difficult to find a contributed buildbot worker. So it remains
> to
> find the hardware to run these buildbots.
>
> Proposal
> ========
>
> Support the Android platform on API 24 [1]_ for the x86_64, armv7 and arm64
> architectures built with NDK 14b.
>
> *API 24*
> * API 21 is the first version to provide usable support for wide
> characters
> and where SELinux is run in enforcing mode.
>
> * API 22 introduces an annoying bug on the linker that prints something
> like
> this when python is started::
>
> ``WARNING: linker: libpython3.6m.so.1.0: unused DT entry: type
> 0x6ffffffe arg 0x14554``.
>
> The `termux`_ Android terminal emulator describes this problem at the
> end
> of its `termux-packages`_ gitlab page and has implemented a
> ``termux-elf-cleaner`` tool to strip the useless entries from the ELF
> header of executables.
>
> * API 24 is the first version where the `adb`_ shell is run on the
> emulator
> as a ``shell`` user instead of the ``root`` user previously, and the
> first
> version that supports arm64.
>
> *x86_64*
> It seems that no handheld device exists using that architecture. It is
> supported because the x86_64 Android emulator runs fast and therefore is a
> good candidate as a buildbot worker.
>
> *NDK 14b*
> This release of the NDK is the first one to use `Unified headers`_ fixing
> numerous problems that had been fixed by updating the Python configure
> script
> until now (those changes have been reverted by now).
>
> Android idiosyncrasies
> ======================
>
> * The default shell is ``/system/bin/sh``.
> * The file system layout is not a traditional unix layout, there is no
> ``/tmp`` for example. Most directories have user restricted access,
> ``/sdcard`` is mounted as ``noexec`` for example.
> * The (java) applications are allocated a unix user id and a subdirectory on
> ``/data/data``.
> * SELinux is run in enforcing mode.
> * Shared memory and semaphores are not supported.
> * The default encoding is UTF-8.
>
> Android build system
> ====================
>
> The Android build system is implemented at `bpo-30386`_ with `PR 1629`_ and
> is documented by its `README`_. It provides the following features:
>
> * To build a distribution for a device or an emulator with a given API level
> and a given architecture.
> * To start the emulator and
> + install the distribution
> + start a remote interactive shell
> + or run remotely a python command
> + or run remotely the buildbottest
> * Run gdb on the python process that is running on the emulator with python
> pretty-printing.
>
> The build system adds the ``Android/`` directory and the
> ``configure-android``
> script to the root of the Python source directory on the master branch
> without
> modifying any other file. The build system can be installed, upgraded (i.e.
> the
> SDK and NDK) and run remotely, through ssh for example.
>
> The following external libraries, when they are configured in the build
> system,
> are downloaded from the internet and cross-compiled (only once, on the first
> run of the build system) before the cross-compilation of the extension
> modules:
>
> * ``ncurses``
> * ``readline``
> * ``sqlite``
> * ``libffi``
> * ``openssl``, the cross-compilation of openssl fails on x86_64 and arm64
> and
> this step is skipped on those architectures.
>
> The following extension modules are disabled by adding them to the
> ``*disabled*`` section of ``Modules/Setup``:
>
> * ``_uuid``, Android has no uuid/uuid.h header.
> * ``grp`` some grp.h functions are not declared.
> * ``_crypt``, Android does not have crypt.h.
> * ``_ctypes`` on x86_64 where all long double tests fail (`bpo-32202`_) and
> on
> arm64 (see `bpo-32203`_).
>
> .. [1] On Wikipedia `Android version history`_ lists the correspondence
> between
> API level, commercial name and version for each release. It also provides
> information on the global Android version distribution, see the two
> charts
> on top.
>
> .. _`README`:
> https://github.com/xdegaye/cpython/blob/bpo-30386/Android/README.rst
> .. _`bpo-26865`: https://bugs.python.org/issue26865
> .. _`bpo-30386`: https://bugs.python.org/issue30386
> .. _`bpo-32202`: https://bugs.python.org/issue32202
> .. _`bpo-32203`: https://bugs.python.org/issue32203
> .. _`PR 1629`: https://github.com/python/cpython/pull/1629
> .. _`buildmaster-config PR 26`:
> https://github.com/python/buildmaster-config/pull/26
> .. _`Android version history`:
> https://en.wikipedia.org/wiki/Android_version_history
> .. _`termux`: https://termux.com/
> .. _`termux-packages`: https://gitlab.com/jbwhips883/termux-packages
> .. _`adb`: https://developer.android.com/studio/command-line/adb.html
> .. _`Unified headers`:
> https://android.googlesource.com/platform/ndk.git/+/ndk-r14-release/docs/UnifiedHeaders.md
> .. _`HP 41`: https://en.wikipedia.org/wiki/HP-41C
> .. vim:filetype=rst:tw=78:ts=8:sts=2:sw=2:et:
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
More information about the Python-Dev
mailing list