[Distutils] distlib updated with "locators" API

[Daniel Holth]

On Mon, Oct 15, 2012 at 2:41 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Daniel Holth <dholth <at> gmail.com> writes:
>
>> When you are using setuptools, setup.py can be used to generate
>> declarative metadata with the command "python setup.py egg_info". You
>> can add --egg-base <dir> to put the static metadata in a different
>> directory. Setuptools even makes it easy to add new metadata files to
>> the .egg-info folder by using a plugin to write each file. The
>> majority of the sdists on pypi contain pre-built versions of this
>> static metadata in .egg-info folders.
>
> Can it be correctly said that all of the metadata passed to setup() is in
> .egg-info? How far can we rely on the accuracy of the content of .egg-info
> dirs in sdists on PyPI? I have found hundreds of projects where even basic
> metadata is missing or incorrect - for example, the version.

You can't trust the bundled .egg-info folder, that is why pip rebuilds
it during dependency resolution. You can trust the fresh one that
you've just built.

The .egg-info contains the three things you need during install: Name,
Version, requirements.

It's always been a mystery to me that we don't just edit PKG-INFO directly.

>> Unfortunately setup.py is used to build not one, but many different
>> dists (the most common difference of course being different
>> requirements) depending on any number of factors, so pip rebuilds the
>> static metadata before using it. Metadata 1.2+ of course allow us to
>> put the most common "reasons for different dependencies" conditions in
>> the static metadata itself.
>
> Unfortunately, that doesn't help with nearly all the stuff currently on PyPI.

Nope, it's a ~25k-package hard problem. Or ~1.5k if you discard the simple ones.