[Distutils] Outdated packages on pypi

Fri Jul 22 13:47:36 EDT 2016

We've been discussing here at least two different problems related to
package maintainership:

1. Abandoned/no-longer-maintained, but previously useful packages

2. namespace and package idea space pollution due to tests/aborted
attempts/packaginginexperience.

I don't have a good idea about 1, and it's clear from this discussion that
curation as a solution for 2 is currently non-workable/undesirable due to
both lack of manpower and a desire not to discourage contributions.

Even if curation is not involved, we also don't want to up the barrier to
entry such that we discourage new entrants into the ecosystem.

A level of namespacing on the project name (but not (necessarily) on the
python module/package level) was suggested as a way to reduce 2.

I just thought of another, orthogonal, suggestion for 2:

There is a TestPyPI server but it feels like few people use it (I certainly
never did).

What if you were only allowed to push a new project onto PyPI proper if you
already have similarly named project on TestPyPI and at least one release
of that project there?

Notice that I'm not suggesting that all files need to touch TestPyPI before
going to PyPI, although that could be considered in the future.

I'm also not suggesting a waiting period between publishing on TestPyPI and
PyPI, but this could also be considered if we see people abusing the fact
that dumping some garbage on TestPyPI is the only step necessary before
dumping some garbage on PyPI.

And we could make it so TestPyPI is a bit more flexible with files than
PyPI itself. For instance we could allow deletion and re-upload of files
with the same name there that is currently disallowed on PyPI so people can
test-drive their releases before pushing  files onto PyPI proper (don't
know if this is already the case).

"Hey!" I hear you saying "If we're just moving the namespace reservation
from PyPI to TestPyPI" aren't we just moving problem number 2 around
without really solving it?"

Well, if someone does some test on TestPyPI and later abandons it, we don't
need to have as many qualms about removing the project than we do with a
project on PyPI proper.

We could even have an official grace period, like a few months, before we
consider that a project on TestPyPI without a PyPI equivalent can be
considered abandoned.

And if we do implement a minimum period on TestPyPI before graduation to
PyPI, this would also give us a measure of protection from malicious typo
squatting:

http://incolumitas.com/2016/06/08/typosquatting-package-managers/

Regards,

Leo

On 22 July 2016 at 13:39, Donald Stufft <donald at stufft.io> wrote:

>
> > On Jul 22, 2016, at 11:47 AM, Chris Barker - NOAA Federal <
> chris.barker at noaa.gov> wrote:
> >
> >
> > If the core devs think it's fine and dandy like it is, we can all stop
> > talking about it.
>
> I think they’re certainly a problem. The current solutions that have been
> proposed have their own problems of course, and problems enough that I
> don’t feel comfortable implementing them. Personally I don’t currently have
> the time to work on a better solution but if someone did that’d be fine
> with me.
>
> I would mention though that it’s possible there *is* no solution to this
> problem that doesn’t bring with it it’s own problems that are worse then
> the problem at hand. I’m not saying that’s the case, but just mentioning
> that it may be so.
>
> >
> > By the way, there is an experiment underway with a "curated" community
> > package repository for conda packages:
> >
> > https://conda-forge.github.io
> >
> > It's semi-curated, in the sense that only the core devs can approve a
> > package being added, but once added, anyone can maintain it.
> >
> > It's going very well, but I'm not sure how it's going to scale. So
> > far, 900 or so packages, 80 or so maintainers. Which I think is very
> > impressive for a young system, but still a lot smaller than it could
> > be.
> >
> > But I think PyPA should keep an eye on it -- I'm sure there will be
> > lessons learned.
>
> conda-forge and PyPI are not really similar. For conda-forge they’re
> largely
> just repackaging other software for the conda system. This makes it
> function
> better when you have just anyone of the core team able to work on stuff,
> because all they’re doing to adjusting the build, upgrading versions etc.
> They’re not working on the projects themselves.
>
> Curated also goes against the sort of “spirit” of PyPI. It’s a place where
> anyone
> can release something and immediately be able to be part of the ecosystem.
> Adding
> curation on top of that would change the nature of PyPI, maybe for the
> better or
> maybe for the worst, but it would fundamentally change PyPI.
>
> —
> Donald Stufft
>
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160722/22fa67a1/attachment-0001.html>