[Distutils] Incorrect detection of 32/64-bit arch on Linux for old 32-bit VMs migrated to 64-bit kernels

[Nathaniel Smith]

Here's an interesting bug I just discovered on the old and poorly
maintained VM that I keep for running random junk:

# 64-bit kernel
$ uname -a
Linux roberts 4.1.5-x86_64-linode61 #7 SMP Mon Aug 24 13:46:31 EDT
2015 x86_64 x86_64 x86_64 GNU/Linux

# 32-bit userland
$ file /usr/bin/python2.7
/usr/bin/python2.7: ELF 32-bit LSB executable, Intel 80386, version 1
(SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24,
BuildID[sha1]=0xf481c2b1f8b4328b2f56b642022cc35b4ad91b61, stripped

$ /usr/bin/python2.7

# Yep, definitely 32-bit
>>> import sys; sys.maxint
2147483647

# Uh oh.
>>> import distutils.util; distutils.util.get_platform()
'linux-x86_64'

# Yeah, this is bad
>>> import distlib.wheel; distlib.wheel.COMPATIBLE_TAGS
set([(u'cp21', u'none', u'any'), (u'py21', u'none', u'any'), (u'py23',
u'none', u'any'), (u'cp24', u'none', u'any'), (u'py2', u'none',
u'any'), (u'cp27', u'none', u'any'), (u'cp20', u'none', u'any'),
(u'py22', u'none', u'any'), (u'cp26', u'none', u'any'), (u'cp27',
u'cp27mu', u'linux_x86_64'), (u'py25', u'none', u'any'), (u'cp23',
u'none', u'any'), (u'py24', u'none', u'any'), (u'cp25', u'none',
u'any'), (u'py27', u'none', u'any'), (u'cp22', u'none', u'any'),
(u'cp2', u'none', u'any'), (u'cp27', u'none', u'linux_x86_64'),
(u'py26', u'none', u'any'), (u'py20', u'none', u'any')])

In the past this has never mattered, because there were no linux
wheels on pypi, so even if pip was using the wrong platform tag it
still wouldn't download the wrong thing. But once manylinux1 goes
live, any systems configured like this will start downloading and
installing totally wheels that will crash on import.

The problem seems to be that distutils.util.get_platform() assumes
that the architecture is whatever uname's "machine" field says
(equivalent to uname -m). Unfortunately, this gives the architecture
of the kernel, not of the Python interpreter.

I think the fix is that we should add some check like

if osname == "linux" and machine == "x86_64" and sys.maxsize == 2147483647:
    machine = "i686"

to distutils (in the branches where it's still maintained), and also
need to add a similar workaround to distlib? (With the later needing
to land before or together with manylinux1 enablement.)

-n

-- 
Nathaniel J. Smith -- https://vorpus.org