[Catalog-sig] PyPI's external packages

[Richard Jones]

On 13 May 2011 06:56,  <exarkun at twistedmatrix.com> wrote:
> On 07:21 pm, ziade.tarek at gmail.com wrote:
>> 2011/5/12  <exarkun at twistedmatrix.com>:
>>>
>>> On 03:57 pm, ziade.tarek at gmail.com wrote:
>>>>
>>>> Hey,
>>>>
>>>> I think some people are unaware of the fact that hosting themselves
>>>> their packages can lead to problems when their websites are down.
>>>>
>>>> I'd like to propose these two very simple changes:
>>>>
>>>> - in packaging/distutils2, when the register command is called, just
>>>> state that uploading the package would be a good idea  :)
>>>> - in pypi.python.org, on a project page that has no file uploaded, if
>>>> the user connected is the project owner/maintainer, add a small
>>>> message explaining why it's a good idea
>>>>
>>>> Maybe that could help reducing the number of external packages
>>>>
>>>> I'll definitely do something in distutils2 but maybe someone has a
>>>> better
>>>> idea ?
>>>
>>> Make it easier to upload packages to PyPI.  For example, add an scp-
>>> based
>>> interface
>>
>> I think Martin added some ssh capability lately. Would make sense to
>> add it in distutils2.
>
> It's weird ssh stuff that so far hasn't seemed to make anything easier.

http://pypi.python.org/pypi/pypissh was developed to allow the
distutils "upload" command to transmit the upload over ssh. Its
intention isn't to make anything easier. It involves submitting an SSH
key to PyPI but other than that it should just work - certainly not
make anything harder.

You're right about it being weird though - well, the heavy
monkey-patching it does of distutils is anyway :-)


> I'm not entirely sure what its goal is.

How would your scp interface work? Do you have an existing
implementation that you could refer to as a model?


>>>  or make "upload" work even if the package files exist on the
>>> filesystem somewhere already.
>>
>> I am not sure to get that one.  Like
>>
>> $ python setup.py upload /any/random/file  ?
>
> Yes, like that.  There are already server-side checks (which are too strict
> in at least one place, preventing legitimate files from being uploaded), so
> I don't see how it's a problem.

I'm not currently aware of any legitimate files being blocked at.
There have been some issues in the past but I believe I'd be correct
in saying that I can count the number of issues I've had to deal with
on one hand.

I do not believe we should allow uploading of arbitrary content as
packages to PyPI. I'm not entirely comfortable with hosting the
arbitrary content in the docs side of things, but that's because I'm
way too paranoid about such things.

Preventing re-uploading of files with the same name was done
intentionally very early on to avoid end-user confusion and spurious
bug reports (that is, people with distribution files of the same name
but with different contents).


> Plus, if I really want to dump garbage onto
> PyPI, then I can still use the web interface.  Making uploading inconvenient
> isn't a strategy for keeping trouble away.

The web form for uploading packages is subject to the same file
legitimacy tests as the distutils upload command. They both use the
same HTTP call on PyPI.


     Richard