[Distutils] Towards a simple and standard sdist format that isn't intertwined with distutils

[Nathaniel Smith]

On Wed, Oct 7, 2015 at 1:28 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> The disconnect here seems to be that I view all of those wheels as
> being numpy 1.9.X wheels (or whatever). They differ in terms of
> compatibility details, but they are all wheels for the same
> project/version. So there's no problem with them all being built from
> the same source wheel. I also have no problem with it being possible
> to configure the build differently from a single source wheel, to
> generate all those wheels. The configuration isn't metadata, it's
> "just" settings for the build.

But the different builds for the different configurations end up with
different metadata. If I'm understanding right, the whole point of
"source wheels" is that they have all the static metadata that pip
needs in order to make decisions, and this has to match the resulting
wheels -- right?

The way I'm imagining it is that there are multiple levels of metadata
staticness:

package name, author, description, ...
  static in: VCS checkouts, source releases, source wheels, wheels
package version
  static in: source releases, source wheels, wheels
package dependencies
  static in: source wheels, wheels
environment tag
  static in: wheels

> Of course, there *is* an unsolved issue here, which is how we manage
> compatibility for wheels at the level needed for numpy. But I thought
> the discussion on that was ongoing? I'm concerned that this proposal
> is actually about bypassing that discussion, and instead trying to
> treat incompatibly linked wheels as "different" in terms of project
> metadata, which I think is the wrong way of handling things. I note
> that Christoph Gohlke's numpy builds are tagged with a "+mkl" local
> version modifier - that's presumably intended to mark the fact that
> they are built with an incompatible runtime - but that's a misuse of
> local versions (and I've found it causes niggling issues with how pip
> recognises upgrades, etc).

Yeah, that's not a good long term solution -- it needs to be moved
into the metadata (probably by creating an MKL wheel and then making
the numpy wheel depend on it). That's exactly the problem :-)

> So, in summary: Your points above don't seem to me to in any way
> preclude having a single numpy source wheel, and a number of (mutually
> incompatible, but the same in terms of project and version) binary
> wheels.

Maybe I have misunderstood: does it actually help pip at all to have
static access to name and version, but not to anything else? I've been
assuming not, but I don't think anyone's pointed to any examples yet
of the problems that pip is encountering due to the lack of static
metadata -- would this actually be enough to solve them?

-n

-- 
Nathaniel J. Smith -- http://vorpus.org