[Catalog-sig] V2 pre-PEP: transitioning to release file hosting on PYPI

[holger krekel]

Hi Carl,

On Tue, Mar 12, 2013 at 10:48 -0600, Carl Meyer wrote:
> Hi Holger,
> 
> I am confused about the discrepancy between the title of this pre-PEP
> ("transition to release file hosting on PyPI") and the contents of the
> PEP, which describe a transition to not crawling _HTML pages_ on
> external sites looking for distribution download links. These are not
> the same thing at all.

I agree the title is not quite right at the moment.

> Current installer tools will only crawl external HTML pages if they are
> rel="download" or rel="homepage", but they will use any link they find
> in the simple index (regardless of rel attr) if the target of the link
> appears to be a distribution file (as determined by filename
> pattern-matching or #egg fragment).

Right.

> At the end of the process you describe, if all packages migrate to
> "nocrawl", the rel-link HTML spidering will no longer happen. This is a
> good first step: it will speed up installation somewhat, and reduce the
> frustration of some package owners when installers find files linked
> from their project homepage that they never intended for automated
> installation. But installers will still find and download release
> packages that are not hosted on PyPI, if those package files are linked
> directly in the simple index. This is still surprising behavior to many
> new Python users, and still carries the security and reliability
> concerns that this PEP claims to address.

Yes, and here the installers should move to give clear warnings
and change defaults.

> I'm honestly not sure whether the title or the content more accurately
> reflects the intent of this PEP; depending which it is, I suggest one of
> the following:
> 
> 1) Add to the PEP a description of a further step in the migration
> process, which actually does transition away from automated installation
> of non-PyPI-hosted release files (as the default behavior of
> installation tools); or

This makes sense to me.  Do you feel like opening a pull request on

    https://bitbucket.org/hpk42/pep-pypi

to help refine this aspect?  I am also on IRC for co-ordination (also
about the title) as i intend to create the PEP submission for
python-ideas and maybe already the pep-editors (?!).  In any case, it
wouldn't mean the PEP's discussion is finalized, of course, and i'd
continue to post here new versions and ask for feedback.

cheers,
holger

> 2) Change the title of the PEP to something like "Transitioning away
> from non-PyPI HTML crawling" and add a paragraph to the PEP clarifying
> that this PEP does not address the issue of actual release files hosted
> off-PyPI.


> Carl
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
>