[Distutils] The Wheel specification and compatibility tags on Windows

[Paul Moore]

On 28 February 2013 06:07, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Thu, Feb 28, 2013 at 5:47 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>> It's certainly true that the ABI flags are incomplete (there's also
>> the whole stable ABI to consider).
>
> The stable ABI is covered in PEP 425: the stable ABI compatibility tag
> is "abi3", and you use the Python version tag to indicate the minimum
> required Python version. What part do you feel we missed?

You're confusing the tags here. cpXX is the "python" tag, which would
be the same for both stable and normal ABI. The ABI tag for the normal
ABI is "none". I'd missed the fact that the PEP specifies "abi3" for
the stable ABI.

The main issue is that the current tools (wheel and distlib) don't
generate it on Windows (the PEP says "The CPython stable ABI is abi3
as in the shared library suffix", but imp.get_suffixes() doesn't show
a .abi3 suffix on Windows - indeed, PEP 384 doesn't specify a naming
convention for stable ABI compatible extensions on Windows. So a wheel
with the API tag 'abi3' would never be accepted as compatible on
Windows.

For that matter, I don't believe it's even possible to tell if an
extension only uses the stable ABI without looking at the source (at
least on Windows). That's the main problem, as if I can't tell then
how do I know that it's OK to use the extension in a version of Python
it wasn't built for?

Mainly, though, I just think that the tools need to properly generate
a list of supported tags that includes the stable ABI on Windows. The
rest is largely just about whether anyone is actually using the stable
ABI, and how (as an end user) I can tell given the need to manually
adjust tags until more sophisticated builders become available.

>> But they are certainly good enough
>> for the foreseeable future, and if & when we want to implement a more
>> complete solution, automatically converting to a new scheme is trivial
>> (as we can determine the MSVCRT version from the Python version for
>> all existing cases).
>
> Indeed - the intent of PEP 425 is that all Windows wheel binaries
> using "cpXY" or "abi3" ABI tags *must* be compatible with the
> corresponding official CPython release from python.org. If they're
> not, then the tool doing the publishing is using the wrong tags
> (perhaps by misrepresenting itself at runtime as a normal CPython
> distribution).

As I said, at least on Windows, tools can't tell if the stable ABI was
used, so they *have* to publish wheels as cpXY-none, which is tied to
a specific minor version. An extension using the stable ABI would need
to specify cpX-abi3 and neither distlib or wheel generates that
combination when generating the list of "supported" tags for a Python
implementation.

> Any Python version linked against a different version of the MSVCRT is
> *not* considered the same implementation from an ABI perspective, and
> thus should *not* use the "cpXY" or "abi3" ABI tags. They can really
> use whatever other tags they like, though (for example "cp27_vc6").

I'm confused here, because you're mixing up tags. I agree that the ABI
tag would have to be different. But cpXY is the *python* tag. Are you
saying that extensions for Python compiled with a different MSVCRT
should be considered as for a different implementation of Python? I
can see an argument for that, but I'm not sure it's strong enough to
justify changing that as well as the ABI tag.

Paul.