From nawkboy at gmail.com  Sat May 18 23:31:47 2013
From: nawkboy at gmail.com (James Carpenter)
Date: Sat, 18 May 2013 16:31:47 -0500
Subject: [Catalog-sig] Continuous deployment,
	in-house PyPi repo and artifact promotion
In-Reply-To: <CAAndj4vty6EGDdgYKHtGEkxJZtMi10QKm+-YTvwUqrn5DC-8_Q@mail.gmail.com>
References: <CAAndj4vty6EGDdgYKHtGEkxJZtMi10QKm+-YTvwUqrn5DC-8_Q@mail.gmail.com>
Message-ID: <CAAndj4sVHbp8L+gPYo8zXmPe8EBuT=PAFdzeqVSn8qkRiZWT5Q@mail.gmail.com>

Catalog-sig is now dead, but I felt it made sense to tie off this thread by
mentioning our new solution.:
===========================
Defend Against Fruit is focused on providing a pragmatic, continuous
deployment style build system for Python. Current Python build systems do
not properly account for the needs of effective continuous deployment. This
package extends the Python tooling to add the missing pieces, including
integration with Artifactory.

With an eye to agile development principles and fast-feedback, we want a
build system which satisfies the following goals:

* Every SCM change-set committed should result in a potentially shippable
release candidate.

* When a defect is introduced, we want to immediately detect and isolate the
offending SCM change-set. This is true  even if the defect was introduced
into a library we depend upon.

* Library management should be so easy as to never impede code changes, even
in multi-component architecture.

More details available at:  http://teamfruit.github.io/defend_against_fruit/
<http://teamfruit.github.io/defend_against_fruit/>

License: Apache Public License v2

Authors:

James Carpenter
jcarpenter621 at yahoo.com
LinkedIn:  http://www.linkedin.com/in/jamescarpenter1
<http://www.linkedin.com/in/jamescarpenter1>

Matthew Tardiff
mattrix at gmail.com
LinkedIn:  http://www.linkedin.com/in/matthewtardiff
<http://www.linkedin.com/in/matthewtardiff>


On Thu, Feb 21, 2013 at 5:21 PM, James Carpenter <nawkboy at gmail.com> wrote:

> 1) Which PyPi repository servers are the most mature and feature rich, and
> therefore the best choice at the moment?
>
> 2) Do any of the PyPi repository servers support managing multiple local
> repositories? If yes, which ones? If no, do you have any recommendations on
> how to most easily support the artifact promotion needs of continuous
> deployment within the Python build ecosystem?
>
> =================================
> Background:
> As part of an effort to implement continuous deployment a fellow traveler
> and I are attempting to workout a good solution for an in-house PyPI
> repository manager. Support of the PyPI XML-RPC API and artifact promotion
> appear to be critical requirements of any solution.
>
> We have tried using Artifactory, but unfortunately this only gives us
> support for "simple" PyPI, without the XML-RPC API distutils/pip uses to
> support searches.
> This is disappointing, because in other respects Artifactory and Nexus
> provide very complete mature support. Most importantly Artifactory and
> Nexus support the ability to host multiple local repositories and proxy
> external ones. By leveraging virtual repositories within Artifactory or
> Nexus one can easily inter-weave results from various local repositories as
> per configurable precedence rules.
>
> The ability to have the build tooling pull artifacts from multiple
> internal repositories turns out to be very important from a continuous
> deployment standpoint. Effective continuous deployment typically treats
> every build artifact as a potential release candidate. As a release
> candidate marches through each gauntlet of tests (unit, integration, load,
> manual, etc.) it is promoted to the next internal repository. This is in
> contrast to the approach of using SNAPSHOTS or the like which fails the
> rule of giving each code check-in the traceability necessary to be a
> potential release candidate. Whether the build tool itself (distutils, Ivy,
> Maven, etc.), the repository manager (Artifactory, Nexus, CheeseShop, etc.)
> or something else knows how to overlay artifacts is an implementation
> detail, but some tool has to do the job. Similarly the details of how a
> build artifact is promoted from one local repository to another is also an
> implementation detail with a variety of possible solutions.
>
> Recognizing support for the PyPI XMLRPC API tends to be a big deal at
> scale, we are looking into using CheeseShop,  Crate.io, or similar to
> address our needs. Another choice may be to place a read-only solution in
> front of Artifactory that adds support for the PyPI XMLRPC API. I am hoping
> members of this reading list can help point us in the right direction.
>
> Thanks for the time and effort you have spent reading this rather long
> post. I look forward to your responses.
>
> Sincerely,
> James Carpenter
> jcarpenter621 at yahoo dot com
>
> P.S.: As you may have guessed I am a visitor from the land of Java where
> the Sun has now set and the Seth lord reigns. (I know a Monty Python
> reference would be better, but I am a bit ignorant in that regard.)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20130518/1a378b2a/attachment.html>