Change in cache tag in Python 3.8 - pip confused

[Chris Angelico]

On Wed, May 8, 2019 at 8:09 AM Paul Moore <p.f.moore at gmail.com> wrote:
>
> On Tue, 7 May 2019 at 22:26, Chris Angelico <rosuav at gmail.com> wrote:
> > So the next question is: Is this actually a problem? If it's something
> > that can only ever happen to people who build prerelease Pythons, it's
> > probably not an issue. Is there any way that a regular installation of
> > Python could ever change its cache_tag? What I experienced would be
> > extremely confusing if it ever happened live ("but it IS installed, no
> > it ISN'T installed...").
>
> I'm probably not the person to answer, because I know *way* to much
> about how pip works to judge what's confusing :-)
>
> However, I'd say that:
>
> 2. In a normal release, Python wouldn't break compatibility like this,
> so this would be a non-issue in a standard x.y.z Python release.

Good.

> 3. Python doesn't guarantee ABI compatibility across minor releases,
> and the site-packages locations would normally be different anyway (I
> know they are on Windows, and I assume they would be on Unix) so you
> won't have this issue in anything other than X.Y.Z and X.Y.W (where W
> != Z). I'd normally recommend reinstalling everything when changing a
> point release too, but I know not everyone does that. ABI
> compatibility between point releases is maintained, though, so it
> shouldn't be a problem.

Yeah, also good. (My personal policy isn't to reinstall everything,
but simply to leave the old packages behind and then install things as
I have need, but same diff.)

> Short answer, it's only ever going to be a problem for people building
> their own Python from git, and they are probably going to know enough
> to debug the issue themselves (even if, like you, they find it hard to
> work out what happened...)

Cool. Good to know.

> > When pip installed python-lzo from source, it must have constructed
> > the file name somehow. I've tried digging through pip's sources, but
> > searching for "cache_tag" came up blank, and I'm not sure where it
> > actually figures out what name to save the .so file under. There's a
> > lot of code in pip, though, and I freely admit to having gotten quite
> > lost in the weeds :|
>
> The build is done in distutils (or maybe, setuptools) so that's where
> you should probably look. Pip just gets a bunch of files from
> setuptools, and installs them, at least for up to date projects and
> versions of pip - it's the "PEP 517" behaviour if you're into recent
> packaging standards. For older pip versions, or projects not updated
> to recent standards, pip just says "hey, setuptools, install this
> stuff and tell me the names of all the files you created".
>
> Good luck digging into the setuptools/distutils code - take rations,
> torches and emergency flares, it's dangerous in there :-)

Eww. Well, that explains why search-across-files for "cache_tag"
didn't come up with anything, but I think... at this point... I don't
need to know any more. Many thanks for your information. I'll pass on
exploring distutils!

ChrisA