[Distutils] Warehouse Migration Plan

[holger krekel]

On Wed, Jul 31, 2013 at 00:15 -0400, Donald Stufft wrote:
> So, in the spirit of not treating distutils-sig like an adversary, here's
> the main thing I've been working on lately with regards to PyPI. None
> of this is set in stone but this is the general gist of the plan for moving
> things to be developed in a modern framework, as well as cleaning
> up the code and getting repeatable deployments.

Is warehouse a re-implementation or did it start from the current code base?

> Warehouse Migration Plan
> ------------------------------------
> 
> Warehouse is currently primarily besides modeling for user accounts. It
> will be deployed alongside pypi-legacy at next.pypi.python.org in the near
> future. Initially it will have zero user facing value.
> 
> As time goes on the database tables will be migrated from being "owned"
> by pypi-legacy to being "owned" by Warehouse. This primarily means that
> the schema definition and migration of those tables will be controlled by
> Warehouse. As tables are migrated to Warehouse ownership the PyPI code
> will be updated to reflect any changes in schema (without modification to
> what end users see).

Am a bit skeptical if sharing databases is a good approach.
Certainly has potential for disrupting pypi.python.org and making
it harder for next.

> Once all tables that are going to be kept have been migrated, we will have
> a shared database that is accessible from both pypi-legacy and Warehouse.
> From this point Warehouse will begin evolving "legacy" views such as the
> simple and other pieces of API. The UX itself will continue to be ignored and
> focus will be on getting feature parity for automated tooling.
>
> Changes in behavior on these legacy views should be minimal and
> discussed on distutils-sig.

Having a doc/spec of those interactions would indeed help and contribute
to "not defined by implementation" as you state below.

> Once the legacy views are finished people will be encouraged to test their
> real world workloads against those reimplemented legacy APIs. As changes
> in behaviors, missing functionality, or bugs are found they will be rectified or
> declared unsupported.
>
> During this time work on the UI of Warehouse will begin focusing on maintaing
> feature parity but with no promises of no changes to the url structure or UX.
> 
> Once Warehouse gains feature parity with PyPI and has gotten enough testing
> against it's APIs then pypi-legacy will be retired and Warehouse will move from
> next.pypi.python.org to pypi.python.org. From there it will evolve on it's own without
> needing to keep pypi-legacy in sync.
> 
> Specification & Acceptance Testing
> ------------------------------------------------
> 
> I do not want a packaging index server to be specified by implementation, so as
> the legacy API is ported over to Warehouse a specification will be drafted. This
> spec will represent the "promise" that PyPI makes about the API it presents to be
> consumed by machines. During the migration any behavior not codified inside of
> the spec is considered implementation defined behavior and backwards compatibility
> for that behavior will not be promised.
> 
> In addition to a defined specification A repository of acceptance tests will be developed.
> These tests will be part of the test framework for any future changes to Warehouse
> but will be maintained separately alongside the specification. They will also allow
> any alternative implementation (such as DevPI) to test themselves against the spec.

I'd be happy to discuss if we can collaborate or even merge some of our 
efforts.

cheers,
holger

> 
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> 



> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130731/b1863054/attachment.pgp>