[Python-Dev] Introducing Python for CloudABI

[Alex Willmer]

Morning all, I'm writing to introduce myself and a port of CPython 3.6
to a CloudABI.

The port is reaching the point where it might be of interest to
others. Namely it ran it's first .py script yesterday during the
EuroPython scripts. Having said that it's still very early days, the
patches do horrible things - particularly to the import machinery.

I writing this to raise awareness, and open discussions. I'd love to
answer any questions/comments you might have.

Background:

# What is CloudABI?
CloudABI is a POSIX-like platform, with Capability based Security
applied. At the syscall/libc layer functions which perform IO or
acquire resources without a pre-existing file descriptor (e.g. open(),
stat(), bind() etc) are removed. All IO operations must be performed
through functions that accept a file descriptor, or a path relative to
an fd.

In this way descriptors server as capability tokens. All such tokens
are provided to a process when it is spawned. If none are provided
then the program in question is limited to just pure computation &
memory allocation. Even stdin, stdout & stderr are not provided by
default.

# Why bother with CloudABI?
It makes it possible to isolate programs from the OS, without
resorting to e.g. containers. Possibly even to run untrusted binaries.
A compromised CloudABI process can only damaged the things it has
access to e.g. a transcoding job can only read the provided input and
write to the provided output. It couldn't read /etc/passwd, or try to
brute force SSH. This kind of isolation is still possible with UNIX,
but it's not the default - which makes it rare.

Personally, I find it interesting. I like the fact that CloudABI
processes can be run by unprivileged users - unlike containers. The
no-default-global-resources nature makes it easier to write code that
can be tested. The fd provided to a webapp doesn't have to be a TCP
socket, it could be a domain socket, or just a file stream.

# What is the state of Python for CloudABI?
Python for CloudABI is a proof of concept. The port takes the form of
a number of patches to CPython 3.6.0a3. These mostly add autoconf &
#ifdef entries for POSIX API functions that CloudABI deliberately does
not support.

A few differences make their way through Python code, for instance
 - sys.path is a list of file descriptors, rather than a  list of strings
 - sys.executable is None
 - sys.argv is not present
 - The uid and gid entries of stat tuples are set to None (like on Windows)

I got print('Hello World', file=...) working about a month ago, and
executed my first .py file yesterday (commit pending).

The current TODO list is
 - Finish script support
- Module execution (python -m) support
- zipimport support for file descriptors
- ssl support
- patch cleanup
- try to run test suite

There is no Python 2.x support, and I don't plan to add any.

# What's the state of CloudABI?
CloudABI runs on FreeBSD, NetBSD, macOS and Linux. For now it requires
a patched kernel on Linux; FreeBSD 11 will include it out the box.
Various libraries/packages have been ported (e.g. curl, libpng, x265,
lua, libressl).

# What's the history of CloudABI?
The project started about 2 years ago. Ed Schouten is the project
leader & creator. I became involved this year, having seen a talk by
Ed at CCC around new year.

# Where can I get more info?
- https://nuxi.nl - CloudABI homepage, including Ed Schouten's CCC talk
- http://slides.com/alexwillmer/cloudabi-capability-security - My EP2016 talk
- https://www.youtube.com/watch?v=wlUtkBa8tK8&feature=youtu.be&t=49m
- https://github.com/NuxiNL/cloudlibc
- https://github.com/NuxiNL/cloudabi-ports
- https://github.com/NuxiNL/cloudabi-ports/tree/master/packages/python
- #cloudabi on Efnet IRC

Regards, Alex
-- 
Alex Willmer <alex at moreati.org.uk>