python package management confusion

[dcs3spp]

On Tuesday, 15 January 2019 07:48:57 UTC, Chris Angelico  wrote:
> On Tue, Jan 15, 2019 at 6:18 PM dieter <dieter at handshake.de> wrote:
> >
> > dcs3spp via Python-list <python-list at python.org> writes:
> > > I am a newbie completely confused with python package management.
> > >
> > > I have a setup.py file (listed below) and have setup pip and setup.cfg to install my own dependencies  from a local devpi repository.
> > >
> > > Can setup.py reference a git repository so that I can install from that url?
> >
> > I doubt it:
> > A primary goal of the Python package management is to allow users
> > to easily install prepackaged components (published in a repository
> > like PyPI) - not to integrate transparently with source code control
> > systems.
> 
> You can use pip to install from a git repo, but I don't know the details.
> 
> ChrisA

Ok cheers all for responding, appreciated....

So to manage the development of private packages, e.g. wheels, I would have to use my own private repository (something like devpi or a an alternative cloud pypi subscription service) to store each private dependency that I have written.  Alternatively, I would rely on pip requirement files. Will have to investigate zbuildout...

The package source for each dependency package could be managed in source control (git, gitlab etc.) and tested in CI build .... gitlab-runner/Jenkins.
In the local development environment, packages that use the dependency package could use the local devpi repository. This would work in a private bare metal environment. 

However, if I wanted to take a step further and run a CI build using cloud services(e.g. in a private gitlab.com repository) for a package that uses the private packages, then presumably there is no access to the devpi repository on my local system? So, alternatively when developing private Python packages I either use requirements.txt or pay subscription for a private pypi cloud repository and configure pip, setup.cfg on gitlab.com CI to reference it in config files. When the CI build completes it pushes the package to the private pypi repository. 

Alternatively:
1. Avoid cloud CI services when developing private Python packages and use private bare metal CI server, e.g. gitlab, Jenkins etc.  

2. Use one monolithic package.