Change in cache tag in Python 3.8 - pip confused

[Chris Angelico]

On Wed, May 8, 2019 at 7:13 AM Paul Moore <p.f.moore at gmail.com> wrote:
>
> On Tue, 7 May 2019 at 21:53, Chris Angelico <rosuav at gmail.com> wrote:
> >
> > I've been building Python 3.8 pre-alphas and alphas for a while now,
> > and ran into a weird problem. Not sure if this is significant or not,
> > and hoping to get other people's views.
> >
> > It seems that the value of sys.implementation.cache_tag changed from
> > "cpython-38m" to just "cpython-38" at some point. That means that a
> > statement like "import lzo" will no longer find a file called
> > "lzo.cpython-38m-x86_64-linux-gnu.so", which I had had prior to the
> > change. The trouble is, pip *did* recognize that file, and said that
> > the python-lzo package was still installed.
> >
> > Solution: "pip uninstall python-lzo" and then reinstall it. Now it's
> > created "lzo.cpython-38-x86_64-linux-gnu.so" and all is well.
> >
> > Does anyone else know about how pip detects existing files, and
> > whether it could be brought more in sync with the import machinery?
>
> I don't know if this is what you were after, but pip decides if a
> project FOO is installed by looking for the FOO-X.Y.Z.dist-info
> directory in site-packages. The RECORD file in that directory contains
> a list of all files installed when the package was originally
> installed (that's what pip uses to decide what to uninstall).
>
> Paul

Ah ha, thank you. That is indeed helpful.

So the next question is: Is this actually a problem? If it's something
that can only ever happen to people who build prerelease Pythons, it's
probably not an issue. Is there any way that a regular installation of
Python could ever change its cache_tag? What I experienced would be
extremely confusing if it ever happened live ("but it IS installed, no
it ISN'T installed...").

When pip installed python-lzo from source, it must have constructed
the file name somehow. I've tried digging through pip's sources, but
searching for "cache_tag" came up blank, and I'm not sure where it
actually figures out what name to save the .so file under. There's a
lot of code in pip, though, and I freely admit to having gotten quite
lost in the weeds :|

ChrisA