[Distutils] How to implement ‘setup.py’ functionality that itself needs third-party distributions (was: Module from install breaks subsequent install of different distribution)

[Daniel Holth]

Have you had a go with https://bitbucket.org/dholth/setup-requires ? I
wrote it because it has always been problematic putting setup_requires
in setup.py to try to get something that setup.py must import into the
global namespace before it can run. I'd love to see some build helpers
that take advantage of this feature.

My version of setup-requires replaces setup.py with a short script
that uses pip to install the setup requirements as specified in a
special section in setup.cfg file, and then runs the real setup.py
which does not have to mention any setup requirements, because they
have already been installed by the setup-requires script. The setup
requirements are installed into a subdirectory and are only available
for the duration of the install. Then real-setup.py can use those
requirements to accomplish whatever it has to do.

On Mon, Jan 19, 2015 at 9:14 PM, Donald Stufft <donald at stufft.io> wrote:
>
>> On Jan 19, 2015, at 8:50 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
>>
>> Tres Seaver <tseaver at palladion.com> writes:
>>
>>> setuptools itself is extensible by means of entry points. Both entry
>>> points in your example (as cited by Marius) demonstrate that: one adds
>>> support for a new keyword argument to 'setup()'[1], and the other
>>> defines a new "writer" for 'egg-info'[2]. By design, both of those are
>>> supposed to be loaded / available for any invocation of 'setup()' in a
>>> Python where the are installed (not merely for packages which
>>> "mention" them).
>>
>> What recourse do I have, then?
>>
>> I'm using entry points because it seems to be the only way I can declare
>> functionality that resides in a module alongside the ‘setup.py’ which
>> itself needs third-party packages.
>>
>> * During the distribution build stage (‘./setup.py build’ or earlier),
>>  I want to parse a reST document and derive some of the distribution
>>  metadata from that.
>>
>> * The ‘setup.py’ is the wrong place for this; it's complex and deserves
>>  its own module which can be imported for unit testing.
>>
>> * This also is totally unrelated to the functionality this distribution
>>  installs, so it doesn't belong in any of the packages to distribute
>>  for installation.
>>
>> * So I place it in a separate top-level module, ‘version’, only for use
>>  within ‘setup.py’.
>>
>> * That module itself needs a third-party distribution (‘docutils’) from
>>  PyPI. So I need that distribution to be installed *before* the
>>  ‘version’ module can be used. Apparently the ‘setup_requires’ option
>>  is the only way to do that.
>>
>> * Then, in order to declare how that functionality is to be used for
>>  commands like ‘./setup.py egg_info’, I have no other way than
>>  declaring a Setuptools entry point.
>>
>> * Declaring a Setuptools entry point makes Setuptools think the
>>  distribution-specific module must be imported by every other
>>  distribution in the same run. Of course it's not available there so
>>  those distributions fail to install.
>>
>> * It even thwarts the installation of ‘docutils’, the very distribution
>>  that is needed for ending this circular dependency.
>>
>> What am I missing? How can I implement complex functionality specific to
>> packaging this distribution, without making an utter mess?
>>
>> --
>> \            “The whole area of [treating source code as intellectual |
>>  `\    property] is almost assuring a customer that you are not going |
>> _o__)               to do any innovation in the future.” —Gary Barnett |
>> Ben Finney
>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>
> I think the thing you’re missing here is that the entry point information
> is used to say “this project supplies these entry points”. setuptools does
> not offer the ability to have only build time entry points, it assumes
> the installed project supplies those entry points. The right way to handle
> this is to either import it in setup.py and do it there, or spin it out
> into it’s own thing that can be installed on it’s own.
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig