[Distutils] dependencies, pip and non-PyPI-hosted packages

[Ernesto Posse]

Thanks. I have tried it with distribute, and it's quite nice, but I
have a couple of questions:

On Tue, Apr 19, 2011 at 2:26 PM, Carl Meyer <carl at oddbird.net> wrote:
> Hi Ernesto,
>
> On 04/19/2011 12:42 PM, Ernesto Posse wrote:
>> Hi I have a couple of packages that I don't want to upload to PyPI
>> just yet but they are hosted some other place, and one (project2)
>> depends on the other (project1). I've packaged them using the standard
>> distutils setup.
>>
>> For project1, setup.py contains:
>>
>>   setup(name='project1', ..., requires=[], provides=['project1'], ...)
>>
>> and for project2, setup.py contains:
>>
>>   setup(name='project2', ..., requires=['project1'], provides=['project2'], ...)
>>
>> I put them in some host https://my.host.org/repository, but when I run
>>
>>   pip install -f https://my.host.org/repository project2
>>
>> installs project2 correctly, but it does not install project1.
>>
>> So my first question is: is the 'requires' keyword in setup.py
>> actually used when installing packages? or it only works for projects
>> hosted on PyPI?
>
> The "standard" requires and provides keywords defined in PEP 314 and
> accepted by distutils are, AFAIK, not actually used by any existing
> tools. They are also deprecated by the newer PEP 345 metadata standard.
>
> (The reason for this, as I understand it, is that those keywords were
> defined by PEP 314 to refer to importable module/package names rather
> than project names. This makes them almost impossible for an installer
> to use, given that we have no global mapping available for translating
> an importable module/package name to an installable project name.)
>
> In order to have pip (or any other tool I know of) respect dependencies
> specified in setup.py, you need to use setuptools (or the "distribute"
> fork) in your setup.py and use its install_requires keyword argument
> instead. See
> http://peak.telecommunity.com/DevCenter/setuptools#new-and-changed-setup-keywords
> for details.

1) using distribute and

  setup(..., install_requires=['project1'],...,
dependency_links=['http://my.host.org/repository/'],...)

installs as expected both project1 and project2, but

  pip uninstall project2

does not uninstall project1. This is quite disappointing, as a user
may be unaware of dependencies automatically installed, and thus, the
uninstall leaves behind something that was installed with the bundle.
I imagine that the idea is that the user may install some other
package that depends on 'project1' and pip takes the conservative
approach (is that the case?) but I would have expected for pip or
distribute or setuptools or distutils to keep some dependency
reference counter. Does any of these tools have something like that?
or is it going to be addressed in distutils2?

2) if install_requires is missing a dependency (project1), the package
gets installed without that dependency, but if I add a dependency and
the user attempts an install (project2) with the updated setup.py
(listing the new dependency) pip will say that the package (project2)
is already installed and won't attempt to install the dependencies. Is
this correct? If so, is there a way to tell pip to install project2's
dependencies?

Thanks



-- 
Ernesto Posse

Modelling and Analysis in Software Engineering
School of Computing
Queen's University - Kingston, Ontario, Canada