[Distutils] Towards a simple and standard sdist format that isn't intertwined with distutils

[Paul Moore]

On 2 October 2015 at 22:45, Nathaniel Smith <njs at pobox.com> wrote:
>> I'm uncomfortable with the fact that the proposed sdist format has
>> more or less no metadata of its own (even the filename format is only
>> a recommendation) so (for example) if someone does "pip install
>> foo==1.0" I don't see how pip can find a suitable sdist, if no wheel
>> is available.
>
> About the filename thing:
>
> The reason that the draft makes the inclusion of package/version info
> a SHOULD instead of a MUST is that regardless of what the spec says,
> all decent installation tools are going to support doing things like
>
>     curl https://github.com/numpy/numpy/archive/master.zip -O numpy-master.zip
>     pip install numpy-master.zip
>
> So we can either handle that by saying that "numpy-master.zip" is an
> sdist, just not one that we would allow on PyPI (which is what the
> current draft does), or we could handle it by saying that
> numpy-master.zip is almost-but-not-quite an sdist, and handling it is
> a commonly supported extension to the standard. Doesn't really matter
> that much either way -- just a matter of terminology. Either way the
> sdists are PyPI are obviously going to be named
> <package>-<version>.<ext>.

OK, that's a good point, and I never felt it was crucial that the
name/version be encoded in the filename. But having them in some form
of static metadata should be mandatory. Your _pypackage.cfg doesn't
contain the package name or version, so how would I get them without
running code? That's my real point.

> Given that the version number in
> particular is something that usually would need to be computed at
> sdist-build-time (from a __version__.py or whatever -- it's very
> common that the source-of-truth for version numbers is not static),
> then leaving it out of the static metadata is nice because it makes
> sdist-building-code much simpler -- 90% of the time you could just
> keep the static metadata file instead of having to rewrite it for each
> sdist. But that's just an engineering trade-off, it's not crucial to
> the concept.

I'm willing to allow for it being non-static in the source tree, but
not in the sdist.

>> I would rather see an sdist format that can be introspected *without*
>> running code or a build tool. Installers and packaging tools like pip
>> need to be able to do that - one of the biggest issues with pip's
>> current sdist handling is that it can't make any meaningful decisions
>> before building at least the egg-info.
>
> Another way to look at this is to say that pip's current handling is
> proof that the build-to-get-metadata strategy is viable :-).

Not if you look at the bug reports for pip that can be traced back to
needing to run setup.py egg-info to get metadata, or other variations
on not having static introspectable metadata in sdists.

Paul