[New-bugs-announce] [issue44689] MacOS: Python binaries not portable between Catalina and Big Sur

[Tobias Bergkvist]

New submission from Tobias Bergkvist <tobias at bergkv.ist>:

Python-binaries compiled on either Big Sur or Catalina - and moved to the other MacOS-version will not work as expected when code depends on ctypes.util.find_library.

Example symptom of this issue: https://github.com/jupyterlab/jupyterlab/issues/9863
I have personally faced this when using Python from nixpkgs - since nixpkgs Python has been built on Catalina - and I'm using Big Sur.


Scenario 1: Compile on Catalina, copy binaries to BigSur, and call ctypes.util.find_library('c')
Python 3.11.0a0 (heads/main:635bfe8162, Jul 19 2021, 08:09:05) [Clang 12.0.0 (clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes.util import find_library; print(find_library('c'))
None

Scenario 2: Compile on Big Sur, copy binaries to Catalina, and call ctypes.util.find_library('c'):
Python 3.11.0a0 (heads/main:635bfe8162, Jul 19 2021, 08:28:48) [Clang 12.0.5 (clang-1205.0.22.11)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes.util import find_library; print(find_library('c'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.11/ctypes/__init__.py", line 8, in <module>
    from _ctypes import Union, Structure, Array
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: dlopen(/usr/local/lib/python3.11/lib-dynload/_ctypes.cpython-311-darwin.so, 2): Symbol not found: __dyld_shared_cache_contains_path
  Referenced from: /usr/local/lib/python3.11/lib-dynload/_ctypes.cpython-311-darwin.so (which was built for Mac OS X 11.4)
  Expected in: /usr/lib/libSystem.B.dylib
 in /usr/local/lib/python3.11/lib-dynload/_ctypes.cpython-311-darwin.so

----------
components: ctypes
messages: 397916
nosy: bergkvist
priority: normal
pull_requests: 25816
severity: normal
status: open
title: MacOS: Python binaries not portable between Catalina and Big Sur
type: behavior
versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue44689>
_______________________________________