[Distutils] some questions about PEP470

[holger krekel]

Hi Carl, Paul, all,

On Sat, Oct 11, 2014 at 18:48 -0600, Carl Meyer wrote:
> Hi Holger,
> 
> On 10/11/2014 12:31 AM, holger krekel wrote:
> > I understand that as a fairly generic security statement.  But I was trying to
> > rather ask about use cases and scenarios where precisely the
> > --extra-index-url option is useful and to be recommended.
> > I'd be grateful if Nick or you could still describe use cases,
> > especially outside PEP470 external links context (the option existed 
> > before so i presume there must be some use cases).
> 
> I don't use it anymore (because these days for everything other than
> interactive playing around, I install only from a curated local index
> specifically limited to each project's dependencies using --no-index and
> --find-links), but I used to use it. My use case was this: generally
> dependencies were installed from PyPI, but occasionally I would need to
> patch a dependency, so I would create an sdist with a patched version
> number (e.g. if I patched 2.0.1, I would create an sdist for version
> 2.0.1.obc1, where "obc" is a tag based on my company name or the
> project) and add this patched sdist to my own index, which I would add
> to my installs with --extra-index-url.
> 
> Because I used a patched version number and pinned all dependencies
> exactly, it didn't matter to me that both PyPI and my "extra index" were
> considered for installation; in fact that was convenient, since it meant
> I could very easily upgrade to a newer PyPI release.
> 
> I never used it for "private" non-PyPI packages.

Right, makes sense and is in line with what Paul noted as his use case
(adding wheels to existing pypi sdists): I think it's good and safe to use it
when you are adding/patching things wrt existing projects on
pypi.python.org.

However, many people don't realize that using --extra-index-url
to install private packages is a bad idea unless you register every
private package as an empty pypi package.  But the latter restriction
is virtually never mentioned (and is an unrealistic recommendation in my
opinion), see examples here:

http://devcenter.gemfury.com/articles/pypi-server.html

http://exhuma.github.io/mypi/index-config.html

and I just noted that the very Python guide on packaging is advertising
using plain --extra-index-url for private packages as well:

http://docs.python-guide.org/en/latest/shipping/packaging/#personal-pypi

and, besides the need for fixing the various discussions/pages, i think
that PEP470 should contribute to a more careful discussion of the feature 
(it's fine for the actual external linking to existing pypi projects 
usecase, mind you).  And i guess pip should have a warning note in
the option help to help educating users.

best,
holger