[Catalog-sig] pypi mirrors in bad shape

[Jannis Leidel]

On Wed, Jan 23, 2013 at 10:59 PM, Donald Stufft <donald.stufft at gmail.com>wrote:

> On Wednesday, January 23, 2013 at 4:52 PM, Christian Theune wrote:
>
>
> My proposal would be to sit down at PyCon to dig a bit more into the
> code to make it more robust. My feeling is that the current client
> tries way to hard on a very low-level API (httplib) for a lot of
> mechanics. I'd be happy to refactor and provide tests, I think.
>
> Also, a script to determine internal consistency and consistency
> compared to PyPI would be nice. Then again, the rsync idea might not be
> that far off regarding the amount of work and the problems we're
> dealing with …
>
> Crate.io uses it's own mirroring code that seems to handle things
> a bit better than pep381client does (infact I originally wrote it because
> of that). However it creates a full mirror with metadata and all and is
> currently being refactored to be a ton simpler.
>
> I still think for the "dumb" mirrors that setting up PyPI mirroring
> protocol to work via rsync is the right direction to go in. Rsync
> has way more manhours invested into it and a lot more testing
> for what pep381client essentially boils down to which is "keep
> these files and those files in sync".
>

Agreed.

But till that rsync API is set up, does anyone know what changed on PyPI
(or wherever) recently so that the mirrors so often go AWOL?

Jannis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20130124/00286db8/attachment.html>