[issue44689] ctypes.util.find_library() does not find macOS 11+ system libraries when built on older macOS systems

[Gregory Szorc]

Gregory Szorc <gregory.szorc at gmail.com> added the comment:

I spoke too soon: you can reproduce this with CPython's build system and I think this is a legit regression.

I think the function sniffing logic is subtly wrong.

Here is the logic as written:

#ifdef __APPLE__
  #ifdef HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH
    #define HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH_RUNTIME \
      __builtin_available(macOS 11.0, iOS 14.0, tvOS 14.0, watchOS 7.0, *)
  #else
    static bool (*_dyld_shared_cache_contains_path)(const char *path);

  #define HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH_RUNTIME \
      _dyld_shared_cache_contains_path != NULL
  #endif
#endif

The fundamental problem is that HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH comes from configure. Configure is using the target settings to probe for function presence. If I set the deployment target to 10.9, it says `checking for _dyld_shared_cache_contains_path... no`. But if I set the deployment target to 11.0, it says `checking for _dyld_shared_cache_contains_path... yes`. Because we may be targeting a macOS without the function, the function appears as not available to configure. It may exist in the SDK, but it is masked behind availability checks for the test compile.

The logic as written assumes that !HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH means the SDK doesn't contain the function at all. This just isn't true for SDKs >= 11.0.

If you look at posixmodule.c, the logic for checking for function availability is typically this pattern:

#ifdef __APPLE__
  #ifdef HAVE_BUILTIN_AVAILABLE
    #define HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH_RUNTIME \
      __builtin_available(macOS 11.0, iOS 14.0, tvOS 14.0, watchOS 7.0, *)
  #else
    #ifdef HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH 
      #define HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH_RUNTIME (_dyld_shared_cache_contains_path != NULL)
    #endif
  #endif
#endif

This other pattern also reveals another regression with this patch: __builtin_available() may not be available on older compilers. See issue42692. I'm unsure what "older compilers" actually is. But someone is bound to complain about this (if they haven't already).

Do these build regressions warrant an unplanned 3.9.8 release?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue44689>
_______________________________________