Distributing multiple packages with on setup.py

[Cameron Simpson]

On 30Sep2017 00:52, Jimmy Thrasibule <thrasibule.jimmy at gmail.com> wrote:
>I've reorganized my Python project to be under a same name umbrella.
>My project can now be seen as multiple subsystems than can depend on
>each other. That means that every sub-package can now be distributed
>alone so that only required dependencies can be installed.
[...]
>The new structure:
>    ├─ myproj/
>    │  ├─ common/
>    │  │  └─ mod1.py
>    │  ├─ subpackage1/
>    │  ├─ subpackage2/
>    │  └─ __init__.py
>    └─ setup.py
>
>
>As you can see not much has changed except that `myproj` is now a
>`namespace package
><https://packaging.python.org/guides/packaging-namespace-packages/>`_
>and that sub-packages ``common``, ``subpackage1`` and ``subpackage2``
>can now be distributed independently.
>
>Is it possible, still keeping one unique ``setup.py`` file, to create
>3 independent packages?
>
>* ``myproj.common``
>* ``myproj.subpackage1``
>* ``myproj.subpackage2``
>
>Also I'd like to specify that when installing ``myproj.subpackage1``,
>``myproj.common`` is required or that ``myproj.subpackage2`` will
>require both ``myproj.common`` and ``myproj.subpackage1``.

I do this with my stuff, but instead of keeping a common setup.py I have an 
elaborate and clumsy system that rolls a package or module distro on the fly, 
writing a setup.py file in the process.

So each package/module I publish has a dict names "DISTINFO" in the top level 
file, looking like this:

  DISTINFO = {
      'keywords': ["python2", "python3"],
      'classifiers': [
          "Programming Language :: Python",
          "Programming Language :: Python :: 2",
          "Programming Language :: Python :: 3",
      ],
      'install_requires': [
          'cs.app.flag',
          'cs.env',
          'cs.logutils',
          'cs.pfx',
          'cs.psutils',
      ],
      'entry_points': {
          'console_scripts': [
              'svcd = cs.app.svcd:main'
          ],
      },
  }

This gets fleshed out with a README file containing the module docstring etc.  
So instead of tediously maintaining some uber setup.py I roll a distribution 
directory suitably filled and push that.

This lets me keep the code in a nice shared tree like yours without a heap of 
setup.py files.

Then I have a script to push the latest tagged revision of the module; it makes 
a scratch directory, checks out the relevant files as of the release tag, makes 
the setup.py and README etc, pushes it, then tossed the scratch directory.

I keep a tag for each module release, eg:

  cs.app.svcd 20170906

for the above. A more published module accrues more tags:

  cs.app.megacli 20150118
  cs.app.megacli 20150118.2
  cs.app.megacli 20150118.3
  cs.app.megacli 20150118.4
  cs.app.megacli 20150118.5
  cs.app.megacli 20150801
  cs.app.megacli 20150801.1
  cs.app.megacli 20160225
  cs.app.megacli 20160226
  cs.app.megacli 20160310

The release script looks for the latest one.

Note that this is all one code repo at my end; the script knows how to assemble 
just the relevant files for a particular module.

Cheers,
Cameron Simpson <cs at cskk.id.au> (formerly cs at zip.com.au)