From aclark at aclark.net  Thu Apr  5 03:56:14 2012
From: aclark at aclark.net (Alex Clark)
Date: Wed, 04 Apr 2012 21:56:14 -0400
Subject: [Catalog-sig] ConfigParser.NoSectionError: No section: 'database'
Message-ID: <jliu3v$gic$1@dough.gmane.org>

Hi

Looks like there may have been a regression with the recent PyPI code 
release and package releases made via pypissh:

- 
https://sourceforge.net/tracker/?func=detail&atid=513503&aid=3515034&group_id=66150


Getting consistent failures with pythonpackages.com (i.e. pypissh) releases.



Alex




-- 
Alex Clark ? http://pythonpackages.com


From kencochrane at gmail.com  Thu Apr  5 16:38:00 2012
From: kencochrane at gmail.com (ken cochrane)
Date: Thu, 5 Apr 2012 10:38:00 -0400
Subject: [Catalog-sig] PyPI mirrors are out of date
Message-ID: <CACBKDnj5N1ra=1BwJfFKxj5P59KWAWp5oXRBHq+rQ38EdAvLgg@mail.gmail.com>

Half of the PyPI mirrors are out of date, I have submitted a ticket here.

https://sourceforge.net/tracker/?func=detail&atid=513503&aid=3515213&group_id=66150

Is there anything I can do to help get this resolved?

Thanks,
Ken Cochrane

From jannis at leidel.info  Thu Apr  5 18:50:10 2012
From: jannis at leidel.info (Jannis Leidel)
Date: Thu, 5 Apr 2012 18:50:10 +0200
Subject: [Catalog-sig] PyPI mirrors are out of date
In-Reply-To: <CACBKDnj5N1ra=1BwJfFKxj5P59KWAWp5oXRBHq+rQ38EdAvLgg@mail.gmail.com>
References: <CACBKDnj5N1ra=1BwJfFKxj5P59KWAWp5oXRBHq+rQ38EdAvLgg@mail.gmail.com>
Message-ID: <94BBE508-3110-4006-B799-E98574E9D164@leidel.info>


On 05.04.2012, at 16:38, ken cochrane wrote:

> Half of the PyPI mirrors are out of date, I have submitted a ticket here.
> 
> https://sourceforge.net/tracker/?func=detail&atid=513503&aid=3515213&group_id=66150
> 
> Is there anything I can do to help get this resolved?

FYI, I just fixed d.pypi.python.org which was stuck with a non-working pep381client process.

Jannis

From kencochrane at gmail.com  Thu Apr  5 18:52:24 2012
From: kencochrane at gmail.com (ken cochrane)
Date: Thu, 5 Apr 2012 12:52:24 -0400
Subject: [Catalog-sig] PyPI mirrors are out of date
In-Reply-To: <94BBE508-3110-4006-B799-E98574E9D164@leidel.info>
References: <CACBKDnj5N1ra=1BwJfFKxj5P59KWAWp5oXRBHq+rQ38EdAvLgg@mail.gmail.com>
	<94BBE508-3110-4006-B799-E98574E9D164@leidel.info>
Message-ID: <CACBKDniMGa4+HdxJCi-M4fkY7JEjgtzFD+kNbod6tk026=rnDQ@mail.gmail.com>

Jannis,

Awesome, thank you so much for your help.

Ken


On Thu, Apr 5, 2012 at 12:50 PM, Jannis Leidel <jannis at leidel.info> wrote:
>
> On 05.04.2012, at 16:38, ken cochrane wrote:
>
>> Half of the PyPI mirrors are out of date, I have submitted a ticket here.
>>
>> https://sourceforge.net/tracker/?func=detail&atid=513503&aid=3515213&group_id=66150
>>
>> Is there anything I can do to help get this resolved?
>
> FYI, I just fixed d.pypi.python.org which was stuck with a non-working pep381client process.
>
> Jannis

From sdouche at gmail.com  Thu Apr  5 19:32:12 2012
From: sdouche at gmail.com (Sebastien Douche)
Date: Thu, 5 Apr 2012 19:32:12 +0200
Subject: [Catalog-sig] PyPI mirrors are out of date
In-Reply-To: <CACBKDnj5N1ra=1BwJfFKxj5P59KWAWp5oXRBHq+rQ38EdAvLgg@mail.gmail.com>
References: <CACBKDnj5N1ra=1BwJfFKxj5P59KWAWp5oXRBHq+rQ38EdAvLgg@mail.gmail.com>
Message-ID: <CAAGHeXEqpxpL6cMREPKQxTMbmzBFKo-wNMoxhkHg7QrDxKs_nA@mail.gmail.com>

On Thu, Apr 5, 2012 at 16:38, ken cochrane <kencochrane at gmail.com> wrote:
> Half of the PyPI mirrors are out of date, I have submitted a ticket here.
>
> https://sourceforge.net/tracker/?func=detail&atid=513503&aid=3515213&group_id=66150
>
> Is there anything I can do to help get this resolved?

It could be great to have a similar page to
http://mirrors.cpan.org/


-- 
Sebastien Douche <sdouche at gmail.com>
Twitter: @sdouche / G+: +sdouche

From kencochrane at gmail.com  Thu Apr  5 19:56:11 2012
From: kencochrane at gmail.com (ken cochrane)
Date: Thu, 5 Apr 2012 13:56:11 -0400
Subject: [Catalog-sig] PyPI mirrors are out of date
In-Reply-To: <CAAGHeXEqpxpL6cMREPKQxTMbmzBFKo-wNMoxhkHg7QrDxKs_nA@mail.gmail.com>
References: <CACBKDnj5N1ra=1BwJfFKxj5P59KWAWp5oXRBHq+rQ38EdAvLgg@mail.gmail.com>
	<CAAGHeXEqpxpL6cMREPKQxTMbmzBFKo-wNMoxhkHg7QrDxKs_nA@mail.gmail.com>
Message-ID: <CACBKDniohLzbMEPrW1Yq_6Qpk_q29yKJ6Q_mkb1VfxJsTw2ThQ@mail.gmail.com>

Yeah that is pretty cool, and it wouldn't be that hard to build, you
would just need a list of all mirrors and then ping each pypi mirrors
last-modified date url every so often and update the page. making it
look nice might be the most complex part.

Ken



On Thu, Apr 5, 2012 at 1:32 PM, Sebastien Douche <sdouche at gmail.com> wrote:
> On Thu, Apr 5, 2012 at 16:38, ken cochrane <kencochrane at gmail.com> wrote:
>> Half of the PyPI mirrors are out of date, I have submitted a ticket here.
>>
>> https://sourceforge.net/tracker/?func=detail&atid=513503&aid=3515213&group_id=66150
>>
>> Is there anything I can do to help get this resolved?
>
> It could be great to have a similar page to
> http://mirrors.cpan.org/
>
>
> --
> Sebastien Douche <sdouche at gmail.com>
> Twitter: @sdouche / G+: +sdouche
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig

From a.badger at gmail.com  Thu Apr  5 20:36:47 2012
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 5 Apr 2012 11:36:47 -0700
Subject: [Catalog-sig] Restart discussion on GNU Public License with
 version classifiers
In-Reply-To: <20120329150159.GF11151@unaka.lan>
References: <20120315203835.GB11151@unaka.lan>
	<CAJ5sox7SyGCEF-69NNQ_McSouEjYs4XGZQzUt6HMs+Na2w0pcw@mail.gmail.com>
	<20120329150159.GF11151@unaka.lan>
Message-ID: <CABVPEKoFY10sgzCY3eaSCvb=erjdEcNnKxYsLcoxZP88MzUE6A@mail.gmail.com>

Ping, Richard, Martin, somebody?

Thanks,
Toshio

On Thu, Mar 29, 2012 at 8:01 AM, Toshio Kuratomi <a.badger at gmail.com> wrote:
>
> Two weeks with no objections raised. ?Could we have the following added?
>
> License :: OSI Approved :: GNU Lesser General Public License v2 (LGPLv2)
> License :: OSI Approved :: GNU Lesser General Public License v3 (LGPLv3)
> License :: OSI Approved :: GNU Lesser General Public License v2 or later (LGPLv2+)
> License :: OSI Approved :: GNU Lesser General Public License v3 or later (LGPLv3+)
> License :: OSI Approved :: GNU General Public License v2 (GPLv2)
> License :: OSI Approved :: GNU General Public License v3 (GPLv3)
> License :: OSI Approved :: GNU General Public License v2 or later (GPLv2+)
> License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
> License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)
>
>
> Thank you,
> -Toshio

From kencochrane at gmail.com  Fri Apr  6 01:18:47 2012
From: kencochrane at gmail.com (ken cochrane)
Date: Thu, 5 Apr 2012 19:18:47 -0400
Subject: [Catalog-sig] PyPI mirrors are out of date
In-Reply-To: <CACBKDniohLzbMEPrW1Yq_6Qpk_q29yKJ6Q_mkb1VfxJsTw2ThQ@mail.gmail.com>
References: <CACBKDnj5N1ra=1BwJfFKxj5P59KWAWp5oXRBHq+rQ38EdAvLgg@mail.gmail.com>
	<CAAGHeXEqpxpL6cMREPKQxTMbmzBFKo-wNMoxhkHg7QrDxKs_nA@mail.gmail.com>
	<CACBKDniohLzbMEPrW1Yq_6Qpk_q29yKJ6Q_mkb1VfxJsTw2ThQ@mail.gmail.com>
Message-ID: <CACBKDngO_FDoiBgOySDvma9XeGjKDZpziDe=VJ8OC6bXnZsLrg@mail.gmail.com>

OK, I decided to build a rough prototype during my lunch break, it
isn't anything special, very very basic and rough.

http://pypimirrors-kencochrane.dotcloud.com

available on github here: https://github.com/kencochrane/pypi-mirrors

Needs a lot of TLC, to make it looks as nice as the cpan one, but you
get the picture.

Ken


On Thu, Apr 5, 2012 at 1:56 PM, ken cochrane <kencochrane at gmail.com> wrote:
> Yeah that is pretty cool, and it wouldn't be that hard to build, you
> would just need a list of all mirrors and then ping each pypi mirrors
> last-modified date url every so often and update the page. making it
> look nice might be the most complex part.
>
> Ken
>
>
>
> On Thu, Apr 5, 2012 at 1:32 PM, Sebastien Douche <sdouche at gmail.com> wrote:
>> On Thu, Apr 5, 2012 at 16:38, ken cochrane <kencochrane at gmail.com> wrote:
>>> Half of the PyPI mirrors are out of date, I have submitted a ticket here.
>>>
>>> https://sourceforge.net/tracker/?func=detail&atid=513503&aid=3515213&group_id=66150
>>>
>>> Is there anything I can do to help get this resolved?
>>
>> It could be great to have a similar page to
>> http://mirrors.cpan.org/
>>
>>
>> --
>> Sebastien Douche <sdouche at gmail.com>
>> Twitter: @sdouche / G+: +sdouche
>> _______________________________________________
>> Catalog-SIG mailing list
>> Catalog-SIG at python.org
>> http://mail.python.org/mailman/listinfo/catalog-sig

From martin at v.loewis.de  Sat Apr  7 10:04:24 2012
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 07 Apr 2012 10:04:24 +0200
Subject: [Catalog-sig] Restart discussion on GNU Public License with
 version classifiers
In-Reply-To: <CABVPEKoFY10sgzCY3eaSCvb=erjdEcNnKxYsLcoxZP88MzUE6A@mail.gmail.com>
References: <20120315203835.GB11151@unaka.lan>	<CAJ5sox7SyGCEF-69NNQ_McSouEjYs4XGZQzUt6HMs+Na2w0pcw@mail.gmail.com>	<20120329150159.GF11151@unaka.lan>
	<CABVPEKoFY10sgzCY3eaSCvb=erjdEcNnKxYsLcoxZP88MzUE6A@mail.gmail.com>
Message-ID: <4F7FF508.8030507@v.loewis.de>

Am 05.04.2012 20:36, schrieb Toshio Kuratomi:
> Ping, Richard, Martin, somebody?
> 
> Thanks,
> Toshio
> 
> On Thu, Mar 29, 2012 at 8:01 AM, Toshio Kuratomi <a.badger at gmail.com> wrote:
>>
>> Two weeks with no objections raised.  Could we have the following added?
>>
>> License :: OSI Approved :: GNU Lesser General Public License v2 (LGPLv2)
>> License :: OSI Approved :: GNU Lesser General Public License v3 (LGPLv3)
>> License :: OSI Approved :: GNU Lesser General Public License v2 or later (LGPLv2+)
>> License :: OSI Approved :: GNU Lesser General Public License v3 or later (LGPLv3+)
>> License :: OSI Approved :: GNU General Public License v2 (GPLv2)
>> License :: OSI Approved :: GNU General Public License v3 (GPLv3)
>> License :: OSI Approved :: GNU General Public License v2 or later (GPLv2+)
>> License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
>> License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)

Done. Sorry it took so long.

Regards,
Martin

From tarek at ziade.org  Sat Apr  7 13:10:41 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 07 Apr 2012 13:10:41 +0200
Subject: [Catalog-sig] PyPI mirrors are out of date
In-Reply-To: <CACBKDngO_FDoiBgOySDvma9XeGjKDZpziDe=VJ8OC6bXnZsLrg@mail.gmail.com>
References: <CACBKDnj5N1ra=1BwJfFKxj5P59KWAWp5oXRBHq+rQ38EdAvLgg@mail.gmail.com>
	<CAAGHeXEqpxpL6cMREPKQxTMbmzBFKo-wNMoxhkHg7QrDxKs_nA@mail.gmail.com>
	<CACBKDniohLzbMEPrW1Yq_6Qpk_q29yKJ6Q_mkb1VfxJsTw2ThQ@mail.gmail.com>
	<CACBKDngO_FDoiBgOySDvma9XeGjKDZpziDe=VJ8OC6bXnZsLrg@mail.gmail.com>
Message-ID: <4F8020B1.80306@ziade.org>

On a side note, I think we should contact b. and e. maintainers and ask 
them to get up to speed, or eventually remove them

Cheers


On 4/6/12 1:18 AM, ken cochrane wrote:
> OK, I decided to build a rough prototype during my lunch break, it
> isn't anything special, very very basic and rough.
>
> http://pypimirrors-kencochrane.dotcloud.com
>
> available on github here: https://github.com/kencochrane/pypi-mirrors
>
> Needs a lot of TLC, to make it looks as nice as the cpan one, but you
> get the picture.
>
> Ken
>
>
> On Thu, Apr 5, 2012 at 1:56 PM, ken cochrane<kencochrane at gmail.com>  wrote:
>> Yeah that is pretty cool, and it wouldn't be that hard to build, you
>> would just need a list of all mirrors and then ping each pypi mirrors
>> last-modified date url every so often and update the page. making it
>> look nice might be the most complex part.
>>
>> Ken
>>
>>
>>
>> On Thu, Apr 5, 2012 at 1:32 PM, Sebastien Douche<sdouche at gmail.com>  wrote:
>>> On Thu, Apr 5, 2012 at 16:38, ken cochrane<kencochrane at gmail.com>  wrote:
>>>> Half of the PyPI mirrors are out of date, I have submitted a ticket here.
>>>>
>>>> https://sourceforge.net/tracker/?func=detail&atid=513503&aid=3515213&group_id=66150
>>>>
>>>> Is there anything I can do to help get this resolved?
>>> It could be great to have a similar page to
>>> http://mirrors.cpan.org/
>>>
>>>
>>> --
>>> Sebastien Douche<sdouche at gmail.com>
>>> Twitter: @sdouche / G+: +sdouche
>>> _______________________________________________
>>> Catalog-SIG mailing list
>>> Catalog-SIG at python.org
>>> http://mail.python.org/mailman/listinfo/catalog-sig
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig


From tarek at ziade.org  Sat Apr  7 13:09:10 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 07 Apr 2012 13:09:10 +0200
Subject: [Catalog-sig] PyPI mirrors are out of date
In-Reply-To: <CACBKDngO_FDoiBgOySDvma9XeGjKDZpziDe=VJ8OC6bXnZsLrg@mail.gmail.com>
References: <CACBKDnj5N1ra=1BwJfFKxj5P59KWAWp5oXRBHq+rQ38EdAvLgg@mail.gmail.com>
	<CAAGHeXEqpxpL6cMREPKQxTMbmzBFKo-wNMoxhkHg7QrDxKs_nA@mail.gmail.com>
	<CACBKDniohLzbMEPrW1Yq_6Qpk_q29yKJ6Q_mkb1VfxJsTw2ThQ@mail.gmail.com>
	<CACBKDngO_FDoiBgOySDvma9XeGjKDZpziDe=VJ8OC6bXnZsLrg@mail.gmail.com>
Message-ID: <4F802056.5020808@ziade.org>

On 4/6/12 1:18 AM, ken cochrane wrote:
> OK, I decided to build a rough prototype during my lunch break, it
> isn't anything special, very very basic and rough.
>
> http://pypimirrors-kencochrane.dotcloud.com
>
> available on github here: https://github.com/kencochrane/pypi-mirrors
>
> Needs a lot of TLC, to make it looks as nice as the cpan one, but you
> get the picture.
>
> Ken

Very nice !  If you're willing to keep that page up and running, I'd 
love to see a link to that page at pypi.python.org.


Cheers
Tarek


From kencochrane at gmail.com  Sat Apr  7 15:45:25 2012
From: kencochrane at gmail.com (ken cochrane)
Date: Sat, 7 Apr 2012 09:45:25 -0400
Subject: [Catalog-sig] PyPI mirrors are out of date
In-Reply-To: <4F802056.5020808@ziade.org>
References: <CACBKDnj5N1ra=1BwJfFKxj5P59KWAWp5oXRBHq+rQ38EdAvLgg@mail.gmail.com>
	<CAAGHeXEqpxpL6cMREPKQxTMbmzBFKo-wNMoxhkHg7QrDxKs_nA@mail.gmail.com>
	<CACBKDniohLzbMEPrW1Yq_6Qpk_q29yKJ6Q_mkb1VfxJsTw2ThQ@mail.gmail.com>
	<CACBKDngO_FDoiBgOySDvma9XeGjKDZpziDe=VJ8OC6bXnZsLrg@mail.gmail.com>
	<4F802056.5020808@ziade.org>
Message-ID: <CACBKDnjpLbxyn=+m1XR77p9=QwB+u15t1zZ4jb6f05q6Trr=7g@mail.gmail.com>

Tarek,

I can keep it up as long as people feel it is useful, it is running
for free at dotCloud so it isn't costing me any money :) . If we want
to make it more legit we should give it a better url though. Any
suggestions?

I also agree about contacting the b and e mirrors. Do we know who
maintains them?

Ken


On Sat, Apr 7, 2012 at 7:09 AM, Tarek Ziad? <tarek at ziade.org> wrote:
> On 4/6/12 1:18 AM, ken cochrane wrote:
>>
>> OK, I decided to build a rough prototype during my lunch break, it
>> isn't anything special, very very basic and rough.
>>
>> http://pypimirrors-kencochrane.dotcloud.com
>>
>> available on github here: https://github.com/kencochrane/pypi-mirrors
>>
>> Needs a lot of TLC, to make it looks as nice as the cpan one, but you
>> get the picture.
>>
>> Ken
>
>
> Very nice ! ?If you're willing to keep that page up and running, I'd love to
> see a link to that page at pypi.python.org.
>
>
> Cheers
> Tarek
>
>
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig

From martin at v.loewis.de  Sat Apr  7 16:48:14 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Sat, 07 Apr 2012 16:48:14 +0200
Subject: [Catalog-sig] PyPI mirrors are out of date
In-Reply-To: <4F802056.5020808@ziade.org>
References: <CACBKDnj5N1ra=1BwJfFKxj5P59KWAWp5oXRBHq+rQ38EdAvLgg@mail.gmail.com>
	<CAAGHeXEqpxpL6cMREPKQxTMbmzBFKo-wNMoxhkHg7QrDxKs_nA@mail.gmail.com>
	<CACBKDniohLzbMEPrW1Yq_6Qpk_q29yKJ6Q_mkb1VfxJsTw2ThQ@mail.gmail.com>
	<CACBKDngO_FDoiBgOySDvma9XeGjKDZpziDe=VJ8OC6bXnZsLrg@mail.gmail.com>
	<4F802056.5020808@ziade.org>
Message-ID: <20120407164814.Horde.eKPiGKGZi1VPgFOunWkQw6A@webmail.df.eu>

> Very nice !  If you're willing to keep that page up and running, I'd  
> love to see a link to that page at pypi.python.org.

If this is considered useful, I think it should be *in* PyPI.

Regards,
Martin



From lists at zopyx.com  Sat Apr  7 17:21:07 2012
From: lists at zopyx.com (Andreas Jung)
Date: Sat, 07 Apr 2012 17:21:07 +0200
Subject: [Catalog-sig] PyPI mirrors are out of date
In-Reply-To: <20120407164814.Horde.eKPiGKGZi1VPgFOunWkQw6A@webmail.df.eu>
References: <CACBKDnj5N1ra=1BwJfFKxj5P59KWAWp5oXRBHq+rQ38EdAvLgg@mail.gmail.com>
	<CAAGHeXEqpxpL6cMREPKQxTMbmzBFKo-wNMoxhkHg7QrDxKs_nA@mail.gmail.com>
	<CACBKDniohLzbMEPrW1Yq_6Qpk_q29yKJ6Q_mkb1VfxJsTw2ThQ@mail.gmail.com>
	<CACBKDngO_FDoiBgOySDvma9XeGjKDZpziDe=VJ8OC6bXnZsLrg@mail.gmail.com>
	<4F802056.5020808@ziade.org>
	<20120407164814.Horde.eKPiGKGZi1VPgFOunWkQw6A@webmail.df.eu>
Message-ID: <4F805B63.4010203@zopyx.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



martin at v.loewis.de wrote:
>> Very nice !  If you're willing to keep that page up and running,
>> I'd love to see a link to that page at pypi.python.org.
> 
> If this is considered useful, I think it should be *in* PyPI.

+1

I would also like to receive notifications through mail
if my own mirror is out of sync. E.g. my pep381client was hanging
last week for a day or so without that I noticed it.

Andreas
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQGUBAEBAgAGBQJPgFtiAAoJEADcfz7u4AZj+zsLv1d6RXEshoLviVDNK05/9wyB
kP2SkgrixClzWl9sxoGhRiSwSpSVcMvApyECICXejnQRXSh81LVOHlIVjGfXWCQD
4P0lkcKunFM/fFCMkUjKH7oTBwmfxnbUsYSLgOdGD/ckOR1oJyBJyff275134TXS
MVh127VtEqyPa23jAVQ4blpBhEiDymUS5RoH/YtfJ7YGOxxmSFzcUts6Zoe0isWI
TQcvXs7D/2GdTH6Qx4iwE6BCr2/UQ7kJCYf6aLTkrgdf7Nt/Uq0W35LWelCmSBeS
IZMtks4jk5Z3p8n0EcNas17+Ez21gKRPXjZbBEa1LBbPlk5BSva97jfO75Vz9kkx
eSKS8PBFLiIFEmhcXwziP5ufhFhT9c0yEKACjzo+8cQhpFh1sI7gE8P7cr+L6l81
9ZmHcIJ1X5gS7wvzdev3Co23SiB+YTIeQSQAXBJklVwAtPRnq0/3pf/9gS2rkWsG
YTWmDZNN3x6W8EFVyfw1vt8XV7wxfSk=
=afAP
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lists.vcf
Type: text/x-vcard
Size: 310 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120407/cf9928bd/attachment.vcf>

From aclark at aclark.net  Sat Apr  7 18:32:44 2012
From: aclark at aclark.net (Alex Clark)
Date: Sat, 07 Apr 2012 12:32:44 -0400
Subject: [Catalog-sig] PyPI mirrors are out of date
In-Reply-To: <4F805B63.4010203@zopyx.com>
References: <CACBKDnj5N1ra=1BwJfFKxj5P59KWAWp5oXRBHq+rQ38EdAvLgg@mail.gmail.com>
	<CAAGHeXEqpxpL6cMREPKQxTMbmzBFKo-wNMoxhkHg7QrDxKs_nA@mail.gmail.com>
	<CACBKDniohLzbMEPrW1Yq_6Qpk_q29yKJ6Q_mkb1VfxJsTw2ThQ@mail.gmail.com>
	<CACBKDngO_FDoiBgOySDvma9XeGjKDZpziDe=VJ8OC6bXnZsLrg@mail.gmail.com>
	<4F802056.5020808@ziade.org>
	<20120407164814.Horde.eKPiGKGZi1VPgFOunWkQw6A@webmail.df.eu>
	<4F805B63.4010203@zopyx.com>
Message-ID: <jlpq7a$gpb$1@dough.gmane.org>

Hi

On 4/7/12 11:21 AM, Andreas Jung wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
>
> martin at v.loewis.de wrote:
>>> Very nice !  If you're willing to keep that page up and running,
>>> I'd love to see a link to that page at pypi.python.org.
>>
>> If this is considered useful, I think it should be *in* PyPI.
>
> +1
>
> I would also like to receive notifications through mail
> if my own mirror is out of sync. E.g. my pep381client was hanging
> last week for a day or so without that I noticed it.


And now that PyPI is on bitbucket, anyone can fork and send pull 
requests IIUC.


Speaking of bitbucket, should PyPI tickets go there now? I opened a PyPI 
ticket on sf.net last week to the resounding sounds of? crickets. :-)


Alex





>
> Andreas
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (Darwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iQGUBAEBAgAGBQJPgFtiAAoJEADcfz7u4AZj+zsLv1d6RXEshoLviVDNK05/9wyB
> kP2SkgrixClzWl9sxoGhRiSwSpSVcMvApyECICXejnQRXSh81LVOHlIVjGfXWCQD
> 4P0lkcKunFM/fFCMkUjKH7oTBwmfxnbUsYSLgOdGD/ckOR1oJyBJyff275134TXS
> MVh127VtEqyPa23jAVQ4blpBhEiDymUS5RoH/YtfJ7YGOxxmSFzcUts6Zoe0isWI
> TQcvXs7D/2GdTH6Qx4iwE6BCr2/UQ7kJCYf6aLTkrgdf7Nt/Uq0W35LWelCmSBeS
> IZMtks4jk5Z3p8n0EcNas17+Ez21gKRPXjZbBEa1LBbPlk5BSva97jfO75Vz9kkx
> eSKS8PBFLiIFEmhcXwziP5ufhFhT9c0yEKACjzo+8cQhpFh1sI7gE8P7cr+L6l81
> 9ZmHcIJ1X5gS7wvzdev3Co23SiB+YTIeQSQAXBJklVwAtPRnq0/3pf/9gS2rkWsG
> YTWmDZNN3x6W8EFVyfw1vt8XV7wxfSk=
> =afAP
> -----END PGP SIGNATURE-----
>
>
>
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig


-- 
Alex Clark ? http://pythonpackages.com


From kencochrane at gmail.com  Sat Apr  7 18:51:42 2012
From: kencochrane at gmail.com (ken cochrane)
Date: Sat, 7 Apr 2012 12:51:42 -0400
Subject: [Catalog-sig] PyPI mirrors are out of date
In-Reply-To: <4F805B63.4010203@zopyx.com>
References: <CACBKDnj5N1ra=1BwJfFKxj5P59KWAWp5oXRBHq+rQ38EdAvLgg@mail.gmail.com>
	<CAAGHeXEqpxpL6cMREPKQxTMbmzBFKo-wNMoxhkHg7QrDxKs_nA@mail.gmail.com>
	<CACBKDniohLzbMEPrW1Yq_6Qpk_q29yKJ6Q_mkb1VfxJsTw2ThQ@mail.gmail.com>
	<CACBKDngO_FDoiBgOySDvma9XeGjKDZpziDe=VJ8OC6bXnZsLrg@mail.gmail.com>
	<4F802056.5020808@ziade.org>
	<20120407164814.Horde.eKPiGKGZi1VPgFOunWkQw6A@webmail.df.eu>
	<4F805B63.4010203@zopyx.com>
Message-ID: <CACBKDnh_mLeKY0Vv2r_KKb3RKPVwY=-Kz+MuffEggMuVno=7Hw@mail.gmail.com>

Andreas,

I have added 'uploading to PyPI' and 'notifications' to my TODO list.

Question about the notifications.

1. Do you run one of the official PyPI mirrors?

2. How do you want to get notified? I'm thinking once a day, we look
to see which mirrors are older then 24 hours out of date, and if
stale, we send an email to the owner of that mirror.

3. How do we envision this working? Should we have one website that
monitors the mirrors and sends out 1 notification per day per mirror
to it's owner, or make it so that people can run their own monitoring
app and notify themselves for their own mirror?

4. Email sending: If we have a list of the maintainers for each
mirror, we can email them directly, or we can send one daily email to
this list, with all mirrors that are out of date and let the people on
this list manage contacting the people directly. Any preferences?

Ken




On Sat, Apr 7, 2012 at 11:21 AM, Andreas Jung <lists at zopyx.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
>
> martin at v.loewis.de wrote:
>>> Very nice ! ?If you're willing to keep that page up and running,
>>> I'd love to see a link to that page at pypi.python.org.
>>
>> If this is considered useful, I think it should be *in* PyPI.
>
> +1
>
> I would also like to receive notifications through mail
> if my own mirror is out of sync. E.g. my pep381client was hanging
> last week for a day or so without that I noticed it.
>
> Andreas
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (Darwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iQGUBAEBAgAGBQJPgFtiAAoJEADcfz7u4AZj+zsLv1d6RXEshoLviVDNK05/9wyB
> kP2SkgrixClzWl9sxoGhRiSwSpSVcMvApyECICXejnQRXSh81LVOHlIVjGfXWCQD
> 4P0lkcKunFM/fFCMkUjKH7oTBwmfxnbUsYSLgOdGD/ckOR1oJyBJyff275134TXS
> MVh127VtEqyPa23jAVQ4blpBhEiDymUS5RoH/YtfJ7YGOxxmSFzcUts6Zoe0isWI
> TQcvXs7D/2GdTH6Qx4iwE6BCr2/UQ7kJCYf6aLTkrgdf7Nt/Uq0W35LWelCmSBeS
> IZMtks4jk5Z3p8n0EcNas17+Ez21gKRPXjZbBEa1LBbPlk5BSva97jfO75Vz9kkx
> eSKS8PBFLiIFEmhcXwziP5ufhFhT9c0yEKACjzo+8cQhpFh1sI7gE8P7cr+L6l81
> 9ZmHcIJ1X5gS7wvzdev3Co23SiB+YTIeQSQAXBJklVwAtPRnq0/3pf/9gS2rkWsG
> YTWmDZNN3x6W8EFVyfw1vt8XV7wxfSk=
> =afAP
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
>

From donald.stufft at gmail.com  Sat Apr  7 19:22:30 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Sat, 7 Apr 2012 13:22:30 -0400
Subject: [Catalog-sig] PyPI mirrors are out of date
In-Reply-To: <CACBKDnh_mLeKY0Vv2r_KKb3RKPVwY=-Kz+MuffEggMuVno=7Hw@mail.gmail.com>
References: <CACBKDnj5N1ra=1BwJfFKxj5P59KWAWp5oXRBHq+rQ38EdAvLgg@mail.gmail.com>
	<CAAGHeXEqpxpL6cMREPKQxTMbmzBFKo-wNMoxhkHg7QrDxKs_nA@mail.gmail.com>
	<CACBKDniohLzbMEPrW1Yq_6Qpk_q29yKJ6Q_mkb1VfxJsTw2ThQ@mail.gmail.com>
	<CACBKDngO_FDoiBgOySDvma9XeGjKDZpziDe=VJ8OC6bXnZsLrg@mail.gmail.com>
	<4F802056.5020808@ziade.org>
	<20120407164814.Horde.eKPiGKGZi1VPgFOunWkQw6A@webmail.df.eu>
	<4F805B63.4010203@zopyx.com>
	<CACBKDnh_mLeKY0Vv2r_KKb3RKPVwY=-Kz+MuffEggMuVno=7Hw@mail.gmail.com>
Message-ID: <D199A71D49EA442B944AB25814DBF310@gmail.com>

I already have notifications on crate.io for the mirror getting out of Sync, but I would gladly accept more notifications 
from this resource as well.

Nice Work ;) 


On Saturday, April 7, 2012 at 12:51 PM, ken cochrane wrote:

> Andreas,
> 
> I have added 'uploading to PyPI' and 'notifications' to my TODO list.
> 
> Question about the notifications.
> 
> 1. Do you run one of the official PyPI mirrors?
> 
> 2. How do you want to get notified? I'm thinking once a day, we look
> to see which mirrors are older then 24 hours out of date, and if
> stale, we send an email to the owner of that mirror.
> 
> 3. How do we envision this working? Should we have one website that
> monitors the mirrors and sends out 1 notification per day per mirror
> to it's owner, or make it so that people can run their own monitoring
> app and notify themselves for their own mirror?
> 
> 4. Email sending: If we have a list of the maintainers for each
> mirror, we can email them directly, or we can send one daily email to
> this list, with all mirrors that are out of date and let the people on
> this list manage contacting the people directly. Any preferences?
> 
> Ken
> 
> 
> 
> 
> On Sat, Apr 7, 2012 at 11:21 AM, Andreas Jung <lists at zopyx.com (mailto:lists at zopyx.com)> wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> > 
> > 
> > 
> > martin at v.loewis.de (mailto:martin at v.loewis.de) wrote:
> > > > Very nice !  If you're willing to keep that page up and running,
> > > > I'd love to see a link to that page at pypi.python.org (http://pypi.python.org).
> > > > 
> > > 
> > > 
> > > If this is considered useful, I think it should be *in* PyPI.
> > 
> > +1
> > 
> > I would also like to receive notifications through mail
> > if my own mirror is out of sync. E.g. my pep381client was hanging
> > last week for a day or so without that I noticed it.
> > 
> > Andreas
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v1.4.11 (Darwin)
> > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
> > 
> > iQGUBAEBAgAGBQJPgFtiAAoJEADcfz7u4AZj+zsLv1d6RXEshoLviVDNK05/9wyB
> > kP2SkgrixClzWl9sxoGhRiSwSpSVcMvApyECICXejnQRXSh81LVOHlIVjGfXWCQD
> > 4P0lkcKunFM/fFCMkUjKH7oTBwmfxnbUsYSLgOdGD/ckOR1oJyBJyff275134TXS
> > MVh127VtEqyPa23jAVQ4blpBhEiDymUS5RoH/YtfJ7YGOxxmSFzcUts6Zoe0isWI
> > TQcvXs7D/2GdTH6Qx4iwE6BCr2/UQ7kJCYf6aLTkrgdf7Nt/Uq0W35LWelCmSBeS
> > IZMtks4jk5Z3p8n0EcNas17+Ez21gKRPXjZbBEa1LBbPlk5BSva97jfO75Vz9kkx
> > eSKS8PBFLiIFEmhcXwziP5ufhFhT9c0yEKACjzo+8cQhpFh1sI7gE8P7cr+L6l81
> > 9ZmHcIJ1X5gS7wvzdev3Co23SiB+YTIeQSQAXBJklVwAtPRnq0/3pf/9gS2rkWsG
> > YTWmDZNN3x6W8EFVyfw1vt8XV7wxfSk=
> > =afAP
> > -----END PGP SIGNATURE-----
> > 
> > _______________________________________________
> > Catalog-SIG mailing list
> > Catalog-SIG at python.org (mailto:Catalog-SIG at python.org)
> > http://mail.python.org/mailman/listinfo/catalog-sig
> > 
> 
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org (mailto:Catalog-SIG at python.org)
> http://mail.python.org/mailman/listinfo/catalog-sig
> 
> 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120407/5e75a68d/attachment.html>

From martin at v.loewis.de  Sat Apr  7 19:47:30 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Sat, 07 Apr 2012 19:47:30 +0200
Subject: [Catalog-sig] PyPI mirrors are out of date
In-Reply-To: <jlpq7a$gpb$1@dough.gmane.org>
References: <CACBKDnj5N1ra=1BwJfFKxj5P59KWAWp5oXRBHq+rQ38EdAvLgg@mail.gmail.com>
	<CAAGHeXEqpxpL6cMREPKQxTMbmzBFKo-wNMoxhkHg7QrDxKs_nA@mail.gmail.com>
	<CACBKDniohLzbMEPrW1Yq_6Qpk_q29yKJ6Q_mkb1VfxJsTw2ThQ@mail.gmail.com>
	<CACBKDngO_FDoiBgOySDvma9XeGjKDZpziDe=VJ8OC6bXnZsLrg@mail.gmail.com>
	<4F802056.5020808@ziade.org>
	<20120407164814.Horde.eKPiGKGZi1VPgFOunWkQw6A@webmail.df.eu>
	<4F805B63.4010203@zopyx.com> <jlpq7a$gpb$1@dough.gmane.org>
Message-ID: <20120407194730.Horde.WKEVUdjz9kRPgH2yeVZFXQA@webmail.df.eu>

> Speaking of bitbucket, should PyPI tickets go there now? I opened a  
> PyPI ticket on sf.net last week to the resounding sounds of?  
> crickets. :-)
>

Depends on the nature of the report. If this is a pull request, it can
go to bitbucket. PyPI support requests should continue to go to SourceForge.
If people do not "get" this, I'll just have to disable the bug tracker
on bitbucket.

Regards,
Martin



From martin at v.loewis.de  Sun Apr  8 19:47:27 2012
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 08 Apr 2012 19:47:27 +0200
Subject: [Catalog-sig] PyPI mirrors are out of date
In-Reply-To: <CACBKDnj5N1ra=1BwJfFKxj5P59KWAWp5oXRBHq+rQ38EdAvLgg@mail.gmail.com>
References: <CACBKDnj5N1ra=1BwJfFKxj5P59KWAWp5oXRBHq+rQ38EdAvLgg@mail.gmail.com>
Message-ID: <4F81CF2F.80207@v.loewis.de>

Am 05.04.2012 16:38, schrieb ken cochrane:
> Half of the PyPI mirrors are out of date, I have submitted a ticket here.
> 
> https://sourceforge.net/tracker/?func=detail&atid=513503&aid=3515213&group_id=66150
> 
> Is there anything I can do to help get this resolved?

I fixed e, which also had a stuck process. b is google appengine, and
it's in the process of updating (the way appengine works, this will take
a few days).

Regards,
Martin

From monitor at jacobian.org  Mon Apr  9 08:52:46 2012
From: monitor at jacobian.org (monitor at jacobian.org)
Date: Mon, 09 Apr 2012 01:52:46 -0500
Subject: [Catalog-sig] [monit] pypi.python.org - Connection failed
Message-ID: <1333954368.1@jacobian.org>

Connection failed Service pypi.python.org 

	Date:        Mon, 09 Apr 2012 01:52:46 -0500
	Action:      alert
	Host:        jacobian.org
	Description: failed protocol test [HTTP] at INET[pypi.python.org:80] via TCP

Your faithful employee,
monit


From sigmundv at gmail.com  Mon Apr  9 09:35:42 2012
From: sigmundv at gmail.com (sigmundv at gmail.com)
Date: Mon, 9 Apr 2012 07:35:42 +0000
Subject: [Catalog-sig] PyZenity
Message-ID: <CAN60Jh4WDQUk2SCLZmb6c347yhDLUeY=aH5vfhsHnFML4dnbUw@mail.gmail.com>

Hi all,

I just want to bring attention to therecord for PyZenity:
http://pypi.python.org/pypi/PyZenity/0.1.4. The links are broken and should
be corrected.


Best regards,

Sigmund Vestergaard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120409/0fa5f7c2/attachment.html>

From lists at zopyx.com  Mon Apr  9 12:01:24 2012
From: lists at zopyx.com (Andreas Jung)
Date: Mon, 09 Apr 2012 12:01:24 +0200
Subject: [Catalog-sig] PyZenity
In-Reply-To: <CAN60Jh4WDQUk2SCLZmb6c347yhDLUeY=aH5vfhsHnFML4dnbUw@mail.gmail.com>
References: <CAN60Jh4WDQUk2SCLZmb6c347yhDLUeY=aH5vfhsHnFML4dnbUw@mail.gmail.com>
Message-ID: <4F82B374.1090809@zopyx.com>

Ask the package maintainer.

-aj

sigmundv at gmail.com wrote:
> Hi all,
>
> I just want to bring attention to therecord for
> PyZenity: http://pypi.python.org/pypi/PyZenity/0.1.4. The links are
> broken and should be corrected.
>
>
> Best regards,
>
> Sigmund Vestergaard
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lists.vcf
Type: text/x-vcard
Size: 324 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120409/b5b11f7c/attachment.vcf>

From sigmundv at gmail.com  Tue Apr 10 09:15:40 2012
From: sigmundv at gmail.com (sigmundv at gmail.com)
Date: Tue, 10 Apr 2012 07:15:40 +0000
Subject: [Catalog-sig] PyZenity
In-Reply-To: <4F82B374.1090809@zopyx.com>
References: <CAN60Jh4WDQUk2SCLZmb6c347yhDLUeY=aH5vfhsHnFML4dnbUw@mail.gmail.com>
	<4F82B374.1090809@zopyx.com>
Message-ID: <CAN60Jh4fd+CnuidDcCsvh-WO72FpekBckEHgBQZOw2KJEkuU0w@mail.gmail.com>

On Mon, Apr 9, 2012 at 10:01, Andreas Jung <lists at zopyx.com> wrote:

> Ask the package maintainer.
>
> -aj


Thank you for the reply. I have now asked the maintainer about this.

Sigmund
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120410/0133237b/attachment.html>

From me at m.merickel.org  Tue Apr 10 15:46:13 2012
From: me at m.merickel.org (Michael Merickel)
Date: Tue, 10 Apr 2012 08:46:13 -0500
Subject: [Catalog-sig] new Pyramid web framework classifier
Message-ID: <CAKdhhwH+4sWBOYxmzz++9Y+N_hXg5Wc3kiXpVB874OJTi0T4FA@mail.gmail.com>

Can we get a new classifier for the Pyramid web framework? Using the
old Pylons classifier is a little deceptive, I think.

I think it should be:

Framework :: Pyramid

Thanks,
Michael

From kencochrane at gmail.com  Sun Apr 15 15:07:50 2012
From: kencochrane at gmail.com (ken cochrane)
Date: Sun, 15 Apr 2012 09:07:50 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
Message-ID: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>

It looks like it took a little while, but all of the PyPI mirrors are
now up to date. Thank you to everyone that helped get those back up to
date. If you want to check out the current status I created this
little website to make it easier to see where things stand mirror
wise. I'm going to be adding a few more things, but it is fully useful
right now. If you have any suggestions, please let me know.

http://www.pypi-mirrors.org


If you run an unofficial PyPI mirror and want it added to the list,
just let me know.

Thanks,
Ken Cochrane
@KenCochrane

From ubershmekel at gmail.com  Sun Apr 15 16:35:43 2012
From: ubershmekel at gmail.com (Yuval Greenfield)
Date: Sun, 15 Apr 2012 17:35:43 +0300
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
Message-ID: <CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>

What's the ranking between Excellent, Great and Awesome?

On Sun, Apr 15, 2012 at 4:07 PM, ken cochrane <kencochrane at gmail.com> wrote:

> It looks like it took a little while, but all of the PyPI mirrors are
> now up to date. Thank you to everyone that helped get those back up to
> date. If you want to check out the current status I created this
> little website to make it easier to see where things stand mirror
> wise. I'm going to be adding a few more things, but it is fully useful
> right now. If you have any suggestions, please let me know.
>
> http://www.pypi-mirrors.org
>
>
> If you run an unofficial PyPI mirror and want it added to the list,
> just let me know.
>
> Thanks,
> Ken Cochrane
> @KenCochrane
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120415/fa05df8b/attachment.html>

From kencochrane at gmail.com  Sun Apr 15 16:44:43 2012
From: kencochrane at gmail.com (Ken Cochrane)
Date: Sun, 15 Apr 2012 10:44:43 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
Message-ID: <0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>

Yeah I need to clean that up and make it more standard but this is how it works now.

Age < 5 min = excellent
Age > 5 min and age < 15 min = awesome
Age > 15 min and age < 1 hour = great.
Age > 1 hour and age <  6 hour = good
Age > 6 hours and age < 12 hour = OK
Age > 12 hours and age < 1 day = getting stale
Age > 1 day = out of date

Feel free to make suggestions on how to improve.

Ken

Sent from my iPhone

On Apr 15, 2012, at 10:35 AM, Yuval Greenfield <ubershmekel at gmail.com> wrote:

> What's the ranking between Excellent, Great and Awesome?
> 
> On Sun, Apr 15, 2012 at 4:07 PM, ken cochrane <kencochrane at gmail.com> wrote:
> It looks like it took a little while, but all of the PyPI mirrors are
> now up to date. Thank you to everyone that helped get those back up to
> date. If you want to check out the current status I created this
> little website to make it easier to see where things stand mirror
> wise. I'm going to be adding a few more things, but it is fully useful
> right now. If you have any suggestions, please let me know.
> 
> http://www.pypi-mirrors.org
> 
> 
> If you run an unofficial PyPI mirror and want it added to the list,
> just let me know.
> 
> Thanks,
> Ken Cochrane
> @KenCochrane
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120415/6be15f5e/attachment.html>

From hanno at hannosch.eu  Sun Apr 15 17:11:31 2012
From: hanno at hannosch.eu (Hanno Schlichting)
Date: Sun, 15 Apr 2012 17:11:31 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
Message-ID: <CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>

On Sun, Apr 15, 2012 at 4:44 PM, Ken Cochrane <kencochrane at gmail.com> wrote:
> Yeah I need to clean that up and make it more standard but this is how it
> works now.
>
> Age < 5 min = excellent
> Age > 5 min and age < 15 min = awesome
> Age > 15 min and age < 1 hour = great.
> Age > 1 hour and age < ?6 hour = good
> Age > 6 hours and age < 12 hour = OK
> Age > 12 hours and age < 1 day = getting stale
> Age > 1 day = out of date
>
> Feel free to make suggestions on how to improve.

My suggestion, keep it simple:

Age < 15 minutes - green (fresh)
Age < 1 day - yellow (oldish)
Age > 1 day - red (old)

Apache and CPAN mirrors are much more forgiving.

http://www.apache.org/mirrors/#age-histogram

< 30 hours - green
< 54 hours - yellow
> 54 hours - red

http://mirrors.cpan.org/#age-histogram

< 2 days green
< 4days - yellow
> 4 days - red

Hanno

From kencochrane at gmail.com  Mon Apr 16 04:09:30 2012
From: kencochrane at gmail.com (ken cochrane)
Date: Sun, 15 Apr 2012 22:09:30 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
Message-ID: <CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>

I have taken your advice and I have updated it so that it is a little
clearer. I changed the times a little but I stuck to the 3 statuses
that you proposed.

I also added a bunch of other features to the site, so feel free to
check it out and let me know if you can think of anything else.

http://www.pypi-mirrors.org


On Sun, Apr 15, 2012 at 11:11 AM, Hanno Schlichting <hanno at hannosch.eu> wrote:
> On Sun, Apr 15, 2012 at 4:44 PM, Ken Cochrane <kencochrane at gmail.com> wrote:
>> Yeah I need to clean that up and make it more standard but this is how it
>> works now.
>>
>> Age < 5 min = excellent
>> Age > 5 min and age < 15 min = awesome
>> Age > 15 min and age < 1 hour = great.
>> Age > 1 hour and age < ?6 hour = good
>> Age > 6 hours and age < 12 hour = OK
>> Age > 12 hours and age < 1 day = getting stale
>> Age > 1 day = out of date
>>
>> Feel free to make suggestions on how to improve.
>
> My suggestion, keep it simple:
>
> Age < 15 minutes - green (fresh)
> Age < 1 day - yellow (oldish)
> Age > 1 day - red (old)
>
> Apache and CPAN mirrors are much more forgiving.
>
> http://www.apache.org/mirrors/#age-histogram
>
> < 30 hours - green
> < 54 hours - yellow
>> 54 hours - red
>
> http://mirrors.cpan.org/#age-histogram
>
> < 2 days green
> < 4days - yellow
>> 4 days - red
>
> Hanno

From merwok at netwok.org  Mon Apr 16 05:20:43 2012
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Sun, 15 Apr 2012 23:20:43 -0400
Subject: [Catalog-sig] new Pyramid web framework classifier
In-Reply-To: <CAKdhhwH+4sWBOYxmzz++9Y+N_hXg5Wc3kiXpVB874OJTi0T4FA@mail.gmail.com>
References: <CAKdhhwH+4sWBOYxmzz++9Y+N_hXg5Wc3kiXpVB874OJTi0T4FA@mail.gmail.com>
Message-ID: <4F8B900B.9080802@netwok.org>

Hi,

Le 10/04/2012 09:46, Michael Merickel a ?crit :
> Can we get a new classifier for the Pyramid web framework? Using the
> old Pylons classifier is a little deceptive, I think.
> 
> I think it should be:
> 
> Framework :: Pyramid

?Deceptive? sounds a bit strong to me, but +1 on the proposal.

Cheers

From lists at zopyx.com  Mon Apr 16 05:33:53 2012
From: lists at zopyx.com (Andreas Jung)
Date: Mon, 16 Apr 2012 05:33:53 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
Message-ID: <4F8B9321.3000702@zopyx.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

This here is weird

c.pypi.python.org	Russian Federation

That's my server it it is running in Germany :)

- -aj

ken cochrane wrote:
> I have taken your advice and I have updated it so that it is a
> little clearer. I changed the times a little but I stuck to the 3
> statuses that you proposed.
> 
> I also added a bunch of other features to the site, so feel free to 
> check it out and let me know if you can think of anything else.
> 
> http://www.pypi-mirrors.org
> 
> 
> On Sun, Apr 15, 2012 at 11:11 AM, Hanno Schlichting
> <hanno at hannosch.eu> wrote:
>> On Sun, Apr 15, 2012 at 4:44 PM, Ken Cochrane
>> <kencochrane at gmail.com> wrote:
>>> Yeah I need to clean that up and make it more standard but this
>>> is how it works now.
>>> 
>>> Age < 5 min = excellent Age > 5 min and age < 15 min = awesome 
>>> Age > 15 min and age < 1 hour = great. Age > 1 hour and age <  6
>>> hour = good Age > 6 hours and age < 12 hour = OK Age > 12 hours
>>> and age < 1 day = getting stale Age > 1 day = out of date
>>> 
>>> Feel free to make suggestions on how to improve.
>> My suggestion, keep it simple:
>> 
>> Age < 15 minutes - green (fresh) Age < 1 day - yellow (oldish) Age
>> > 1 day - red (old)
>> 
>> Apache and CPAN mirrors are much more forgiving.
>> 
>> http://www.apache.org/mirrors/#age-histogram
>> 
>> < 30 hours - green < 54 hours - yellow
>>> 54 hours - red
>> http://mirrors.cpan.org/#age-histogram
>> 
>> < 2 days green < 4days - yellow
>>> 4 days - red
>> Hanno
> _______________________________________________ Catalog-SIG mailing
> list Catalog-SIG at python.org 
> http://mail.python.org/mailman/listinfo/catalog-sig

- -- 
ZOPYX Limited           | zopyx group
Charlottenstr. 37/1     | The full-service network for Zope & Plone
D-72070 T?bingen        | Produce & Publish
www.zopyx.com           | www.produce-and-publish.com
- ------------------------------------------------------------------------
E-Publishing, Python, Zope & Plone development, Consulting


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQGUBAEBAgAGBQJPi5MhAAoJEADcfz7u4AZjNNkLwI4YhWH6JND0j4OKdQJEqk63
caeRq+h8q+Pt4eAE+MiXkLxyx3VRK0tSEesGLFTC39aGKuSMKfpbKzYtc5dT8qIX
+WAhzrHw1MMzO7s6ShATv2VDYqcSCrsJx2KcHexxdpawpQTh9G53dnHRnyT6/Cq/
CaQHXnIyTgCbh3xK8ruZWJ098lvr6Nqj2mS4JFILb0qf6N+4nNR+6FeEITqFf7HS
Hemwx6i5m2ZIx1gvPIsoqFnNNvLBSCMdluFPVESkIPf6Ocm19ykHYKTOY+6I7iCV
zUNkKaucA6sTOW3OMQ8MR4/Km2tBy+PPjRx/0wY99SaPDAenvTGovq62koaYTBs7
NE4Cd5P1ydUkNVXFIyVc/ZiAWAJCGg5FL7nxKZRbQ2fb9L7YGYYVIhy3AaOziL6T
qBFT35uz3ZWh0linYUPNpHplJ7V6hdEazA5AgVaVaeNyor86JIYtuxZOC8UBHQQl
hk686sVopjJUCzu3UZOT8eoZlIOqpIA=
=O4Ej
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lists.vcf
Type: text/x-vcard
Size: 310 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120416/85fd3878/attachment.vcf>

From tjreedy at udel.edu  Mon Apr 16 07:54:33 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 16 Apr 2012 01:54:33 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
Message-ID: <jmgc6q$2cq$1@dough.gmane.org>

On 4/15/2012 10:09 PM, ken cochrane wrote:
> I have taken your advice and I have updated it so that it is a little
> clearer. I changed the times a little but I stuck to the 3 statuses
> that you proposed.
>
> I also added a bunch of other features to the site, so feel free to
> check it out and let me know if you can think of anything else.
>
> http://www.pypi-mirrors.org

Looks pretty nice, except I suggest 'aging' rather than 'oldish', which 
is not a real word.

>
> On Sun, Apr 15, 2012 at 11:11 AM, Hanno Schlichting<hanno at hannosch.eu>  wrote:
>> On Sun, Apr 15, 2012 at 4:44 PM, Ken Cochrane<kencochrane at gmail.com>  wrote:
>>> Yeah I need to clean that up and make it more standard but this is how it
>>> works now.
>>>
>>> Age<  5 min = excellent
>>> Age>  5 min and age<  15 min = awesome
>>> Age>  15 min and age<  1 hour = great.
>>> Age>  1 hour and age<    6 hour = good
>>> Age>  6 hours and age<  12 hour = OK
>>> Age>  12 hours and age<  1 day = getting stale
>>> Age>  1 day = out of date
>>>
>>> Feel free to make suggestions on how to improve.
>>
>> My suggestion, keep it simple:
>>
>> Age<  15 minutes - green (fresh)
>> Age<  1 day - yellow (oldish)
>> Age>  1 day - red (old)
>>
>> Apache and CPAN mirrors are much more forgiving.
>>
>> http://www.apache.org/mirrors/#age-histogram
>>
>> <  30 hours - green
>> <  54 hours - yellow
>>> 54 hours - red
>>
>> http://mirrors.cpan.org/#age-histogram
>>
>> <  2 days green
>> <  4days - yellow
>>> 4 days - red
>>
>> Hanno


-- 
Terry Jan Reedy


From ubershmekel at gmail.com  Mon Apr 16 09:00:04 2012
From: ubershmekel at gmail.com (Yuval Greenfield)
Date: Mon, 16 Apr 2012 10:00:04 +0300
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <jmgc6q$2cq$1@dough.gmane.org>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<jmgc6q$2cq$1@dough.gmane.org>
Message-ID: <CANSw7KyvDywiR-w3Es5qmKFA58tXn0r4QmM8x5Uua5LWzBLAVQ@mail.gmail.com>

Looks very nice. I especially like the graphs. If we're into the bike
shedding stage then here are a few suggestions:

   - Make columns JS sortable
   - Ping is relative - that column should state "Response time from
   Florida" or something to that effect as I got very different results from
   those listed.
   - "Oldish" is ok though I think "Ok", "Medium", "Aging", "Freshish" or
   "So so" would be fine as well.


Yuval

On Mon, Apr 16, 2012 at 8:54 AM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 4/15/2012 10:09 PM, ken cochrane wrote:
>
>> I have taken your advice and I have updated it so that it is a little
>> clearer. I changed the times a little but I stuck to the 3 statuses
>> that you proposed.
>>
>> I also added a bunch of other features to the site, so feel free to
>> check it out and let me know if you can think of anything else.
>>
>> http://www.pypi-mirrors.org
>>
>
> Looks pretty nice, except I suggest 'aging' rather than 'oldish', which is
> not a real word.
>
>
>
>> On Sun, Apr 15, 2012 at 11:11 AM, Hanno Schlichting<hanno at hannosch.eu>
>>  wrote:
>>
>>> On Sun, Apr 15, 2012 at 4:44 PM, Ken Cochrane<kencochrane at gmail.com**>
>>>  wrote:
>>>
>>>> Yeah I need to clean that up and make it more standard but this is how
>>>> it
>>>> works now.
>>>>
>>>> Age<  5 min = excellent
>>>> Age>  5 min and age<  15 min = awesome
>>>> Age>  15 min and age<  1 hour = great.
>>>> Age>  1 hour and age<    6 hour = good
>>>> Age>  6 hours and age<  12 hour = OK
>>>> Age>  12 hours and age<  1 day = getting stale
>>>> Age>  1 day = out of date
>>>>
>>>> Feel free to make suggestions on how to improve.
>>>>
>>>
>>> My suggestion, keep it simple:
>>>
>>> Age<  15 minutes - green (fresh)
>>> Age<  1 day - yellow (oldish)
>>> Age>  1 day - red (old)
>>>
>>> Apache and CPAN mirrors are much more forgiving.
>>>
>>> http://www.apache.org/mirrors/**#age-histogram<http://www.apache.org/mirrors/#age-histogram>
>>>
>>> <  30 hours - green
>>> <  54 hours - yellow
>>>
>>>> 54 hours - red
>>>>
>>>
>>> http://mirrors.cpan.org/#age-**histogram<http://mirrors.cpan.org/#age-histogram>
>>>
>>> <  2 days green
>>> <  4days - yellow
>>>
>>>> 4 days - red
>>>>
>>>
>>> Hanno
>>>
>>
>
> --
> Terry Jan Reedy
>
>
> ______________________________**_________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/**mailman/listinfo/catalog-sig<http://mail.python.org/mailman/listinfo/catalog-sig>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120416/02275f19/attachment.html>

From djay at pretaweb.com  Mon Apr 16 09:10:05 2012
From: djay at pretaweb.com (Dylan Jay)
Date: Mon, 16 Apr 2012 17:10:05 +1000
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
Message-ID: <1E18D937-2E71-4394-9D3B-BC01A9EE65DF@pretaweb.com>

Hi,

Some DNS load balancing services have api's. For example  
edgedirector.com, but there others (and perhaps cheaper).

Why not go to the next level and get the script on this page to update  
a DNS load balancer with all green mirrors?
Then switching mirrors would be automatic.


---
Dylan Jay
Technical Solutions Manager
PretaWeb: Multisite Performance Support
P: +612 80819071 | M: +61421477460 | twitter.com/djay75 | linkedin.com/ 
in/djay75

On 15/04/2012, at 11:07 PM, ken cochrane wrote:

> It looks like it took a little while, but all of the PyPI mirrors are
> now up to date. Thank you to everyone that helped get those back up to
> date. If you want to check out the current status I created this
> little website to make it easier to see where things stand mirror
> wise. I'm going to be adding a few more things, but it is fully useful
> right now. If you have any suggestions, please let me know.
>
> http://www.pypi-mirrors.org
>
>
> If you run an unofficial PyPI mirror and want it added to the list,
> just let me know.
>
> Thanks,
> Ken Cochrane
> @KenCochrane
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig


From tarek at ziade.org  Mon Apr 16 09:23:23 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 16 Apr 2012 09:23:23 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
Message-ID: <4F8BC8EB.3050303@ziade.org>

On 4/15/12 3:07 PM, ken cochrane wrote:
> It looks like it took a little while, but all of the PyPI mirrors are
> now up to date. Thank you to everyone that helped get those back up to
> date. If you want to check out the current status I created this
> little website to make it easier to see where things stand mirror
> wise. I'm going to be adding a few more things, but it is fully useful
> right now. If you have any suggestions, please let me know.
>
> http://www.pypi-mirrors.org
>
>
> If you run an unofficial PyPI mirror and want it added to the list,
> just let me know.
extra !

a nice feature would be to send a mail automatically in this ML when a 
mirror becomes "old"

Cheers & thanks for this work
Tarek

>
> Thanks,
> Ken Cochrane
> @KenCochrane
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig


From noah at coderanger.net  Mon Apr 16 09:42:48 2012
From: noah at coderanger.net (Noah Kantrowitz)
Date: Mon, 16 Apr 2012 00:42:48 -0700
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <1E18D937-2E71-4394-9D3B-BC01A9EE65DF@pretaweb.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<1E18D937-2E71-4394-9D3B-BC01A9EE65DF@pretaweb.com>
Message-ID: <C868083F-D17E-4514-B840-54DE18E0A906@coderanger.net>


On Apr 16, 2012, at 12:10 AM, Dylan Jay wrote:

> Hi,
> 
> Some DNS load balancing services have api's. For example edgedirector.com, but there others (and perhaps cheaper).
> 
> Why not go to the next level and get the script on this page to update a DNS load balancer with all green mirrors?
> Then switching mirrors would be automatic.

This wouldn't be a good idea since getting redirected to two different mirrors during one install attempt would likely confuse most existing scripts if versions were slightly out of sync.

As for uptime scanning, I can add sensors for this to the PSF's existing Pingdom account and you could use their API in the same way as http://ispypiup.com/. That would give some basic site-indpendence since Pingdom has check servers all over the world (we currently require 6 sites to be in agreement before a downtime alert is issued). If this interests you, let me know :-)

--Noah

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 203 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120416/46876d74/attachment.pgp>

From djay at pretaweb.com  Mon Apr 16 10:05:31 2012
From: djay at pretaweb.com (Dylan Jay)
Date: Mon, 16 Apr 2012 18:05:31 +1000
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <C868083F-D17E-4514-B840-54DE18E0A906@coderanger.net>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<1E18D937-2E71-4394-9D3B-BC01A9EE65DF@pretaweb.com>
	<C868083F-D17E-4514-B840-54DE18E0A906@coderanger.net>
Message-ID: <E271E129-7642-4093-9863-E73442EDA3D6@pretaweb.com>

On 16/04/2012, at 5:42 PM, Noah Kantrowitz wrote:

>
> On Apr 16, 2012, at 12:10 AM, Dylan Jay wrote:
>
>> Hi,
>>
>> Some DNS load balancing services have api's. For example  
>> edgedirector.com, but there others (and perhaps cheaper).
>>
>> Why not go to the next level and get the script on this page to  
>> update a DNS load balancer with all green mirrors?
>> Then switching mirrors would be automatic.
>
> This wouldn't be a good idea since getting redirected to two  
> different mirrors during one install attempt would likely confuse  
> most existing scripts if versions were slightly out of sync.

I don't think it would direct you to two different mirrors during one  
install attempt. Wouldn't a single python session cache DNS resolution?


>
> As for uptime scanning, I can add sensors for this to the PSF's  
> existing Pingdom account and you could use their API in the same way  
> as http://ispypiup.com/. That would give some basic site-indpendence  
> since Pingdom has check servers all over the world (we currently  
> require 6 sites to be in agreement before a downtime alert is  
> issued). If this interests you, let me know :-)
>
> --Noah
>


From noah at coderanger.net  Mon Apr 16 10:12:07 2012
From: noah at coderanger.net (Noah Kantrowitz)
Date: Mon, 16 Apr 2012 01:12:07 -0700
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <E271E129-7642-4093-9863-E73442EDA3D6@pretaweb.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<1E18D937-2E71-4394-9D3B-BC01A9EE65DF@pretaweb.com>
	<C868083F-D17E-4514-B840-54DE18E0A906@coderanger.net>
	<E271E129-7642-4093-9863-E73442EDA3D6@pretaweb.com>
Message-ID: <D9BB6E84-7BB7-4450-8650-ECFE6A4C9FB9@coderanger.net>


On Apr 16, 2012, at 1:05 AM, Dylan Jay wrote:

> On 16/04/2012, at 5:42 PM, Noah Kantrowitz wrote:
> 
>> 
>> On Apr 16, 2012, at 12:10 AM, Dylan Jay wrote:
>> 
>>> Hi,
>>> 
>>> Some DNS load balancing services have api's. For example edgedirector.com, but there others (and perhaps cheaper).
>>> 
>>> Why not go to the next level and get the script on this page to update a DNS load balancer with all green mirrors?
>>> Then switching mirrors would be automatic.
>> 
>> This wouldn't be a good idea since getting redirected to two different mirrors during one install attempt would likely confuse most existing scripts if versions were slightly out of sync.
> 
> I don't think it would direct you to two different mirrors during one install attempt. Wouldn't a single python session cache DNS resolution?

I don't see how you could assume that given that it would follow the system resolver's behavior in most cases, and that isn't the kind of thing to count on. Regardless the uptime of PyPI will shortly become a non-issue :-)

--Noah
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 203 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120416/0d2e8f0c/attachment.pgp>

From sdouche at gmail.com  Mon Apr 16 11:15:03 2012
From: sdouche at gmail.com (Sebastien Douche)
Date: Mon, 16 Apr 2012 11:15:03 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
Message-ID: <CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>

On Mon, Apr 16, 2012 at 04:09, ken cochrane <kencochrane at gmail.com> wrote:
> I also added a bunch of other features to the site, so feel free to
> check it out and let me know if you can think of anything else.
>
> http://www.pypi-mirrors.org

Nice! Thanks Ken. Missing only an information imho: the number of
packages on each mirror (to known if the mirror is really up to date).


-- 
Sebastien Douche <sdouche at gmail.com>
Twitter: @sdouche / G+: +sdouche

From zopyxfilter at gmail.com  Sun Apr 15 17:31:44 2012
From: zopyxfilter at gmail.com (zopyxfilter at gmail.com)
Date: Sun, 15 Apr 2012 17:31:44 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
Message-ID: <4F8AE9E0.1010703@gmail.com>

Hanno Schlichting wrote:
> On Sun, Apr 15, 2012 at 4:44 PM, Ken Cochrane <kencochrane at gmail.com> wrote:
>> Yeah I need to clean that up and make it more standard but this is how it
>> works now.
>>
>> Age < 5 min = excellent
>> Age > 5 min and age < 15 min = awesome
>> Age > 15 min and age < 1 hour = great.
>> Age > 1 hour and age <  6 hour = good
>> Age > 6 hours and age < 12 hour = OK
>> Age > 12 hours and age < 1 day = getting stale
>> Age > 1 day = out of date
>>
>> Feel free to make suggestions on how to improve.
> 
> My suggestion, keep it simple:
> 
> Age < 15 minutes - green (fresh)
> Age < 1 day - yellow (oldish)
> Age > 1 day - red (old)

+1

-aj

From martin at v.loewis.de  Mon Apr 16 15:18:17 2012
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 16 Apr 2012 15:18:17 +0200
Subject: [Catalog-sig] new Pyramid web framework classifier
In-Reply-To: <CAKdhhwH+4sWBOYxmzz++9Y+N_hXg5Wc3kiXpVB874OJTi0T4FA@mail.gmail.com>
References: <CAKdhhwH+4sWBOYxmzz++9Y+N_hXg5Wc3kiXpVB874OJTi0T4FA@mail.gmail.com>
Message-ID: <4F8C1C19.9070401@v.loewis.de>

Am 10.04.2012 15:46, schrieb Michael Merickel:
> Can we get a new classifier for the Pyramid web framework? Using the
> old Pylons classifier is a little deceptive, I think.
> 
> I think it should be:
> 
> Framework :: Pyramid

What specific packages already on PyPI would use this classifier?

Regards,
Martin

From lists at zopyx.com  Mon Apr 16 15:30:03 2012
From: lists at zopyx.com (Andreas Jung)
Date: Mon, 16 Apr 2012 15:30:03 +0200
Subject: [Catalog-sig] new Pyramid web framework classifier
In-Reply-To: <4F8C1C19.9070401@v.loewis.de>
References: <CAKdhhwH+4sWBOYxmzz++9Y+N_hXg5Wc3kiXpVB874OJTi0T4FA@mail.gmail.com>
	<4F8C1C19.9070401@v.loewis.de>
Message-ID: <4F8C1EDB.5010508@zopyx.com>

Applications running on top of Pyramid like

http://pypi.python.org/pypi/zopyx.smartprintng.server/1.1.1

?

-aj

Martin v. L?wis wrote:
> Am 10.04.2012 15:46, schrieb Michael Merickel:
>> Can we get a new classifier for the Pyramid web framework? Using the
>> old Pylons classifier is a little deceptive, I think.
>>
>> I think it should be:
>>
>> Framework :: Pyramid
>
> What specific packages already on PyPI would use this classifier?
>
> Regards,
> Martin
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lists.vcf
Type: text/x-vcard
Size: 324 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120416/4c8d2c93/attachment.vcf>

From martin at v.loewis.de  Mon Apr 16 15:30:28 2012
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 16 Apr 2012 15:30:28 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <1E18D937-2E71-4394-9D3B-BC01A9EE65DF@pretaweb.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<1E18D937-2E71-4394-9D3B-BC01A9EE65DF@pretaweb.com>
Message-ID: <4F8C1EF4.5030207@v.loewis.de>

> Why not go to the next level and get the script on this page to update a
> DNS load balancer with all green mirrors?

I still think that mirror selection should be on the client side. First,
if you have an infrastructure that updates the mirror list, it is again
a single point of failure. Plus, clients would want to select a mirror
based on network performance, which a DNS load balancer cannot achieve.

Regards,
Martin

From lists at zopyx.com  Mon Apr 16 15:38:58 2012
From: lists at zopyx.com (Andreas Jung)
Date: Mon, 16 Apr 2012 15:38:58 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <4F8C1EF4.5030207@v.loewis.de>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<1E18D937-2E71-4394-9D3B-BC01A9EE65DF@pretaweb.com>
	<4F8C1EF4.5030207@v.loewis.de>
Message-ID: <4F8C20F2.4090604@zopyx.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Martin v. L?wis wrote:
>> Why not go to the next level and get the script on this page to
>> update a DNS load balancer with all green mirrors?
> 
> I still think that mirror selection should be on the client side.
> First, if you have an infrastructure that updates the mirror list, it
> is again a single point of failure. Plus, clients would want to
> select a mirror based on network performance, which a DNS load
> balancer cannot achieve.

+1

- -aj
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQGUBAEBAgAGBQJPjCDxAAoJEADcfz7u4AZjvVMLvj/ynm/sZjXVFwm8A+mFmyfv
w/mNRf9DA8kznajtcOrhEHNHIaBmaBehLVtFwC/g9IYeZjJLlkVfJWNowZp/KxfE
oAjUgw5JxlJA+8pOEICMB/q33tHf8unl1832SEoTpNYeJFqR9XlvyPtUATHVqmX5
VNMymFSPUawgPxTyCrxE567+IxQC2e0xzmPZI60ets+1tBe+7PyNmF9rCDx274wr
+brDsYbpcxDld0G7Ezifd59xYdBAPMJhayBGPLACVqUixm64QQaPV1tbWvIJVIHc
UlaPC9eIpdtp7lAR4OifHgBnMnEFtCKUKHWRS3tZ62Ypu+sGAT1UC2ejz4mn+UXB
BY+gpiaeJufnbNxLucSvm30WiWQ3By77ml5S7A+ulHBo/mppF4vYXtm3Hy11CgSU
FiOhgS+kPeDrisWOqynus4s8opD4frHo4Hoat+bKK09Bmqr95UwIYGRY8m0Ks+xH
5WHAgZpp6EhRQ0DDgIVk+jdy0kmCRZo=
=ngzn
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lists.vcf
Type: text/x-vcard
Size: 310 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120416/328396b4/attachment.vcf>

From donald.stufft at gmail.com  Mon Apr 16 15:44:05 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Mon, 16 Apr 2012 09:44:05 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <4F8C1EF4.5030207@v.loewis.de>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<1E18D937-2E71-4394-9D3B-BC01A9EE65DF@pretaweb.com>
	<4F8C1EF4.5030207@v.loewis.de>
Message-ID: <59F9A109AE65438FA604B56C972241CE@gmail.com>

fwiw I go back and forth on how I feel about this. For sure the client wants to select the mirror  
based on network performance to them. But I feel like if a mirror get's to be "stuck" and not
updating that PyPI should at some point remove it from the pool (and notify the owner that it has done so)
until it catches back up.


On Monday, April 16, 2012 at 9:30 AM, "Martin v. L?wis" wrote:

> > Why not go to the next level and get the script on this page to update a
> > DNS load balancer with all green mirrors?
> >  
>  
>  
> I still think that mirror selection should be on the client side. First,
> if you have an infrastructure that updates the mirror list, it is again
> a single point of failure. Plus, clients would want to select a mirror
> based on network performance, which a DNS load balancer cannot achieve.
>  
> Regards,
> Martin
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org (mailto:Catalog-SIG at python.org)
> http://mail.python.org/mailman/listinfo/catalog-sig
>  
>  


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120416/bfd8ef3e/attachment.html>

From martin at v.loewis.de  Mon Apr 16 15:47:59 2012
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 16 Apr 2012 15:47:59 +0200
Subject: [Catalog-sig] new Pyramid web framework classifier
In-Reply-To: <4F8C1EDB.5010508@zopyx.com>
References: <CAKdhhwH+4sWBOYxmzz++9Y+N_hXg5Wc3kiXpVB874OJTi0T4FA@mail.gmail.com>
	<4F8C1C19.9070401@v.loewis.de> <4F8C1EDB.5010508@zopyx.com>
Message-ID: <4F8C230F.9060007@v.loewis.de>

Am 16.04.2012 15:30, schrieb Andreas Jung:
> Applications running on top of Pyramid like
> 
> http://pypi.python.org/pypi/zopyx.smartprintng.server/1.1.1
> 
> ?

Can you kindly provide a second example?

I (honestly) have no clue what Pyramid is or who might be
using it, and I need "a few" packages to justify the introduction
of a new classifier.

Regards,
Martin

From merwok at netwok.org  Mon Apr 16 15:53:11 2012
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Mon, 16 Apr 2012 09:53:11 -0400
Subject: [Catalog-sig] new Pyramid web framework classifier
In-Reply-To: <4F8C230F.9060007@v.loewis.de>
References: <CAKdhhwH+4sWBOYxmzz++9Y+N_hXg5Wc3kiXpVB874OJTi0T4FA@mail.gmail.com>
	<4F8C1C19.9070401@v.loewis.de> <4F8C1EDB.5010508@zopyx.com>
	<4F8C230F.9060007@v.loewis.de>
Message-ID: <4F8C2447.3090202@netwok.org>

Hello Martin,

> I (honestly) have no clue what Pyramid is or who might be
> using it, and I need "a few" packages to justify the introduction
> of a new classifier.

Pyramid is the Web framework resulting from the merge of Pylons and 
repoze.bfg.  Pylons is not developed anymore and users switch to 
Pyramid.  A new classifier would let people tag their extensions and 
applications; see examples at 
http://pypi.python.org/pypi?%3Aaction=search&term=pyramid

HTH

Regards

From martin at v.loewis.de  Mon Apr 16 16:25:25 2012
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 16 Apr 2012 16:25:25 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <59F9A109AE65438FA604B56C972241CE@gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>	<1E18D937-2E71-4394-9D3B-BC01A9EE65DF@pretaweb.com>	<4F8C1EF4.5030207@v.loewis.de>
	<59F9A109AE65438FA604B56C972241CE@gmail.com>
Message-ID: <4F8C2BD5.5020507@v.loewis.de>

Am 16.04.2012 15:44, schrieb Donald Stufft:
> fwiw I go back and forth on how I feel about this. For sure the client
> wants to select the mirror
> based on network performance to them. But I feel like if a mirror get's
> to be "stuck" and not
> updating that PyPI should at some point remove it from the pool (and
> notify the owner that it has done so)
> until it catches back up.

This is actually how it works. There may be differences though in what
the threshold for removal is. I'd personally set it to
a) trying to reach the operator at least three times, *and*
b) the mirror being outdated by more than six month (sic).

>From my own experience, I find it very plausible to not be able to
work on a problem for several months, so an outage by that time
doesn't bother me - as long as clients can reliably detect the outage.
If the mirror was corrupt in some other way (e.g. claiming to be
up-to-date even though it is not) is a different story; that could
cause immediate removal.

Regards,
Martin

From martin at v.loewis.de  Mon Apr 16 17:03:53 2012
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 16 Apr 2012 17:03:53 +0200
Subject: [Catalog-sig] new Pyramid web framework classifier
In-Reply-To: <4F8C2447.3090202@netwok.org>
References: <CAKdhhwH+4sWBOYxmzz++9Y+N_hXg5Wc3kiXpVB874OJTi0T4FA@mail.gmail.com>	<4F8C1C19.9070401@v.loewis.de>
	<4F8C1EDB.5010508@zopyx.com>	<4F8C230F.9060007@v.loewis.de>
	<4F8C2447.3090202@netwok.org>
Message-ID: <4F8C34D9.3060709@v.loewis.de>

Am 16.04.2012 15:53, schrieb ?ric Araujo:
> Hello Martin,
> 
>> I (honestly) have no clue what Pyramid is or who might be
>> using it, and I need "a few" packages to justify the introduction
>> of a new classifier.
> 
> Pyramid is the Web framework resulting from the merge of Pylons and
> repoze.bfg.  Pylons is not developed anymore and users switch to
> Pyramid.  A new classifier would let people tag their extensions and
> applications; see examples at
> http://pypi.python.org/pypi?%3Aaction=search&term=pyramid
> 
> HTH

It does indeed. I've added the classifier.

Regards,
Martin

From chris at python.org  Mon Apr 16 16:56:29 2012
From: chris at python.org (Chris Withers)
Date: Mon, 16 Apr 2012 15:56:29 +0100
Subject: [Catalog-sig] new Pyramid web framework classifier
In-Reply-To: <4F8C230F.9060007@v.loewis.de>
References: <CAKdhhwH+4sWBOYxmzz++9Y+N_hXg5Wc3kiXpVB874OJTi0T4FA@mail.gmail.com>
	<4F8C1C19.9070401@v.loewis.de> <4F8C1EDB.5010508@zopyx.com>
	<4F8C230F.9060007@v.loewis.de>
Message-ID: <4F8C331D.1080002@python.org>

On 16/04/2012 14:47, "Martin v. L?wis" wrote:
> I (honestly) have no clue what Pyramid is or who might be
> using it, and I need "a few" packages to justify the introduction
> of a new classifier.

It's a pretty popular web framework now ;-)

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
             - http://www.simplistix.co.uk

From djay at pretaweb.com  Mon Apr 16 19:04:53 2012
From: djay at pretaweb.com (Dylan Jay)
Date: Tue, 17 Apr 2012 03:04:53 +1000
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <4F8C1EF4.5030207@v.loewis.de>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<1E18D937-2E71-4394-9D3B-BC01A9EE65DF@pretaweb.com>
	<4F8C1EF4.5030207@v.loewis.de>
Message-ID: <19FD1D6D-EAB1-4606-B53B-C6CBDD7DDC5D@pretaweb.com>


On 16/04/2012, at 11:30 PM, Martin v. L?wis wrote:

>> Why not go to the next level and get the script on this page to  
>> update a
>> DNS load balancer with all green mirrors?
>
> I still think that mirror selection should be on the client side.  
> First,
> if you have an infrastructure that updates the mirror list, it is  
> again
> a single point of failure. Plus, clients would want to select a mirror
> based on network performance, which a DNS load balancer cannot  
> achieve.

One of the services offered by edgedirector.com etc is "geo-targetted  
global load balancing" which means they will serve up the IP closest  
to your location.

In any case offering a load balanced domain like "any.pypi.org"  
doesn't preclude individual mirror domains. The user could choose.


>
> Regards,
> Martin


From kencochrane at gmail.com  Mon Apr 16 21:39:23 2012
From: kencochrane at gmail.com (ken cochrane)
Date: Mon, 16 Apr 2012 15:39:23 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <4F8B9321.3000702@zopyx.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<4F8B9321.3000702@zopyx.com>
Message-ID: <CACBKDni-QVMHKQBpw73c4E3Pp6aEwR9rfW21AtrtKrDwW5H+hg@mail.gmail.com>

Sorry about that, I'm getting the location from http://ipinfodb.com
feel free to update their location for that IP with the correct one.



On Sun, Apr 15, 2012 at 11:33 PM, Andreas Jung <lists at zopyx.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> This here is weird
>
> c.pypi.python.org ? ? ? Russian Federation
>
> That's my server it it is running in Germany :)
>
> - -aj
>
> ken cochrane wrote:
>> I have taken your advice and I have updated it so that it is a
>> little clearer. I changed the times a little but I stuck to the 3
>> statuses that you proposed.
>>
>> I also added a bunch of other features to the site, so feel free to
>> check it out and let me know if you can think of anything else.
>>
>> http://www.pypi-mirrors.org
>>
>>
>> On Sun, Apr 15, 2012 at 11:11 AM, Hanno Schlichting
>> <hanno at hannosch.eu> wrote:
>>> On Sun, Apr 15, 2012 at 4:44 PM, Ken Cochrane
>>> <kencochrane at gmail.com> wrote:
>>>> Yeah I need to clean that up and make it more standard but this
>>>> is how it works now.
>>>>
>>>> Age < 5 min = excellent Age > 5 min and age < 15 min = awesome
>>>> Age > 15 min and age < 1 hour = great. Age > 1 hour and age < ?6
>>>> hour = good Age > 6 hours and age < 12 hour = OK Age > 12 hours
>>>> and age < 1 day = getting stale Age > 1 day = out of date
>>>>
>>>> Feel free to make suggestions on how to improve.
>>> My suggestion, keep it simple:
>>>
>>> Age < 15 minutes - green (fresh) Age < 1 day - yellow (oldish) Age
>>> > 1 day - red (old)
>>>
>>> Apache and CPAN mirrors are much more forgiving.
>>>
>>> http://www.apache.org/mirrors/#age-histogram
>>>
>>> < 30 hours - green < 54 hours - yellow
>>>> 54 hours - red
>>> http://mirrors.cpan.org/#age-histogram
>>>
>>> < 2 days green < 4days - yellow
>>>> 4 days - red
>>> Hanno
>> _______________________________________________ Catalog-SIG mailing
>> list Catalog-SIG at python.org
>> http://mail.python.org/mailman/listinfo/catalog-sig
>
> - --
> ZOPYX Limited ? ? ? ? ? | zopyx group
> Charlottenstr. 37/1 ? ? | The full-service network for Zope & Plone
> D-72070 T?bingen ? ? ? ?| Produce & Publish
> www.zopyx.com ? ? ? ? ? | www.produce-and-publish.com
> - ------------------------------------------------------------------------
> E-Publishing, Python, Zope & Plone development, Consulting
>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (Darwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iQGUBAEBAgAGBQJPi5MhAAoJEADcfz7u4AZjNNkLwI4YhWH6JND0j4OKdQJEqk63
> caeRq+h8q+Pt4eAE+MiXkLxyx3VRK0tSEesGLFTC39aGKuSMKfpbKzYtc5dT8qIX
> +WAhzrHw1MMzO7s6ShATv2VDYqcSCrsJx2KcHexxdpawpQTh9G53dnHRnyT6/Cq/
> CaQHXnIyTgCbh3xK8ruZWJ098lvr6Nqj2mS4JFILb0qf6N+4nNR+6FeEITqFf7HS
> Hemwx6i5m2ZIx1gvPIsoqFnNNvLBSCMdluFPVESkIPf6Ocm19ykHYKTOY+6I7iCV
> zUNkKaucA6sTOW3OMQ8MR4/Km2tBy+PPjRx/0wY99SaPDAenvTGovq62koaYTBs7
> NE4Cd5P1ydUkNVXFIyVc/ZiAWAJCGg5FL7nxKZRbQ2fb9L7YGYYVIhy3AaOziL6T
> qBFT35uz3ZWh0linYUPNpHplJ7V6hdEazA5AgVaVaeNyor86JIYtuxZOC8UBHQQl
> hk686sVopjJUCzu3UZOT8eoZlIOqpIA=
> =O4Ej
> -----END PGP SIGNATURE-----

From kencochrane at gmail.com  Mon Apr 16 21:41:01 2012
From: kencochrane at gmail.com (ken cochrane)
Date: Mon, 16 Apr 2012 15:41:01 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <jmgc6q$2cq$1@dough.gmane.org>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<jmgc6q$2cq$1@dough.gmane.org>
Message-ID: <CACBKDni24fM39d_4Y0_H==Mj3bYT7-DDHdrt7980B45OedW+ng@mail.gmail.com>

Thanks,

Change has been made, it will be available in next update.

Ken


On Mon, Apr 16, 2012 at 1:54 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 4/15/2012 10:09 PM, ken cochrane wrote:
>>
>> I have taken your advice and I have updated it so that it is a little
>> clearer. I changed the times a little but I stuck to the 3 statuses
>> that you proposed.
>>
>> I also added a bunch of other features to the site, so feel free to
>> check it out and let me know if you can think of anything else.
>>
>> http://www.pypi-mirrors.org
>
>
> Looks pretty nice, except I suggest 'aging' rather than 'oldish', which is
> not a real word.
>
>
>>
>> On Sun, Apr 15, 2012 at 11:11 AM, Hanno Schlichting<hanno at hannosch.eu>
>> ?wrote:
>>>
>>> On Sun, Apr 15, 2012 at 4:44 PM, Ken Cochrane<kencochrane at gmail.com>
>>> ?wrote:
>>>>
>>>> Yeah I need to clean that up and make it more standard but this is how
>>>> it
>>>> works now.
>>>>
>>>> Age< ?5 min = excellent
>>>> Age> ?5 min and age< ?15 min = awesome
>>>> Age> ?15 min and age< ?1 hour = great.
>>>> Age> ?1 hour and age< ? ?6 hour = good
>>>> Age> ?6 hours and age< ?12 hour = OK
>>>> Age> ?12 hours and age< ?1 day = getting stale
>>>> Age> ?1 day = out of date
>>>>
>>>> Feel free to make suggestions on how to improve.
>>>
>>>
>>> My suggestion, keep it simple:
>>>
>>> Age< ?15 minutes - green (fresh)
>>> Age< ?1 day - yellow (oldish)
>>> Age> ?1 day - red (old)
>>>
>>> Apache and CPAN mirrors are much more forgiving.
>>>
>>> http://www.apache.org/mirrors/#age-histogram
>>>
>>> < ?30 hours - green
>>> < ?54 hours - yellow
>>>>
>>>> 54 hours - red
>>>
>>>
>>> http://mirrors.cpan.org/#age-histogram
>>>
>>> < ?2 days green
>>> < ?4days - yellow
>>>>
>>>> 4 days - red
>>>
>>>
>>> Hanno
>
>
>
> --
> Terry Jan Reedy
>
>
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig

From kencochrane at gmail.com  Mon Apr 16 21:42:54 2012
From: kencochrane at gmail.com (ken cochrane)
Date: Mon, 16 Apr 2012 15:42:54 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <CANSw7KyvDywiR-w3Es5qmKFA58tXn0r4QmM8x5Uua5LWzBLAVQ@mail.gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<jmgc6q$2cq$1@dough.gmane.org>
	<CANSw7KyvDywiR-w3Es5qmKFA58tXn0r4QmM8x5Uua5LWzBLAVQ@mail.gmail.com>
Message-ID: <CACBKDngZmqGQQTENdJHWnt-0GtV1RBjZ6gFK0uZa7tDj53Wg3w@mail.gmail.com>

Thanks for the feedback, I'll think about the sortable tables, it
might be overkill for such a small dataset, but I'll keep it in mind.

I added a disclaimer that the response times are from Virginia, US.

Status has been changed to Fresh, Aging and Old.

Ken


On Mon, Apr 16, 2012 at 3:00 AM, Yuval Greenfield <ubershmekel at gmail.com> wrote:
> Looks very nice. I especially like the graphs. If we're into the bike
> shedding stage then here are a few suggestions:
>
> Make columns JS sortable
> Ping is relative - that column should state "Response time from Florida" or
> something to that effect as I got very different results from those listed.
> "Oldish" is ok though I think "Ok", "Medium", "Aging", "Freshish" or "So so"
> would be fine as well.
>
>
> Yuval
>
> On Mon, Apr 16, 2012 at 8:54 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>>
>> On 4/15/2012 10:09 PM, ken cochrane wrote:
>>>
>>> I have taken your advice and I have updated it so that it is a little
>>> clearer. I changed the times a little but I stuck to the 3 statuses
>>> that you proposed.
>>>
>>> I also added a bunch of other features to the site, so feel free to
>>> check it out and let me know if you can think of anything else.
>>>
>>> http://www.pypi-mirrors.org
>>
>>
>> Looks pretty nice, except I suggest 'aging' rather than 'oldish', which is
>> not a real word.
>>
>>
>>>
>>> On Sun, Apr 15, 2012 at 11:11 AM, Hanno Schlichting<hanno at hannosch.eu>
>>> ?wrote:
>>>>
>>>> On Sun, Apr 15, 2012 at 4:44 PM, Ken Cochrane<kencochrane at gmail.com>
>>>> ?wrote:
>>>>>
>>>>> Yeah I need to clean that up and make it more standard but this is how
>>>>> it
>>>>> works now.
>>>>>
>>>>> Age< ?5 min = excellent
>>>>> Age> ?5 min and age< ?15 min = awesome
>>>>> Age> ?15 min and age< ?1 hour = great.
>>>>> Age> ?1 hour and age< ? ?6 hour = good
>>>>> Age> ?6 hours and age< ?12 hour = OK
>>>>> Age> ?12 hours and age< ?1 day = getting stale
>>>>> Age> ?1 day = out of date
>>>>>
>>>>> Feel free to make suggestions on how to improve.
>>>>
>>>>
>>>> My suggestion, keep it simple:
>>>>
>>>> Age< ?15 minutes - green (fresh)
>>>> Age< ?1 day - yellow (oldish)
>>>> Age> ?1 day - red (old)
>>>>
>>>> Apache and CPAN mirrors are much more forgiving.
>>>>
>>>> http://www.apache.org/mirrors/#age-histogram
>>>>
>>>> < ?30 hours - green
>>>> < ?54 hours - yellow
>>>>>
>>>>> 54 hours - red
>>>>
>>>>
>>>> http://mirrors.cpan.org/#age-histogram
>>>>
>>>> < ?2 days green
>>>> < ?4days - yellow
>>>>>
>>>>> 4 days - red
>>>>
>>>>
>>>> Hanno
>>
>>
>>
>> --
>> Terry Jan Reedy
>>
>>
>> _______________________________________________
>> Catalog-SIG mailing list
>> Catalog-SIG at python.org
>> http://mail.python.org/mailman/listinfo/catalog-sig
>
>
>
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
>

From kencochrane at gmail.com  Mon Apr 16 21:44:55 2012
From: kencochrane at gmail.com (ken cochrane)
Date: Mon, 16 Apr 2012 15:44:55 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <C868083F-D17E-4514-B840-54DE18E0A906@coderanger.net>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<1E18D937-2E71-4394-9D3B-BC01A9EE65DF@pretaweb.com>
	<C868083F-D17E-4514-B840-54DE18E0A906@coderanger.net>
Message-ID: <CACBKDnjD9unHh=pd3rjZOTtG=sRC1K30xHqCX-uDT3V7Ldkcdw@mail.gmail.com>

> As for uptime scanning, I can add sensors for this to the PSF's existing Pingdom account and you could use their API in the same way as http://ispypiup.com/. That would give some basic site-indpendence since Pingdom has check servers all over the world (we currently require 6 sites to be in agreement before a downtime alert is issued). If this interests you, let me know :-)
>

That might be interesting, I'll need to figure out how to fit it with
the current setup, but I'm open to suggestions.

Ken

From kencochrane at gmail.com  Mon Apr 16 21:46:01 2012
From: kencochrane at gmail.com (ken cochrane)
Date: Mon, 16 Apr 2012 15:46:01 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>
Message-ID: <CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>

Is it possible to get the number of packages on each mirror? If so,
can you let me know and I'll look into adding it.

On Mon, Apr 16, 2012 at 5:15 AM, Sebastien Douche <sdouche at gmail.com> wrote:

> Nice! Thanks Ken. Missing only an information imho: the number of
> packages on each mirror (to known if the mirror is really up to date).
>

From kencochrane at gmail.com  Mon Apr 16 21:49:01 2012
From: kencochrane at gmail.com (ken cochrane)
Date: Mon, 16 Apr 2012 15:49:01 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <19FD1D6D-EAB1-4606-B53B-C6CBDD7DDC5D@pretaweb.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<1E18D937-2E71-4394-9D3B-BC01A9EE65DF@pretaweb.com>
	<4F8C1EF4.5030207@v.loewis.de>
	<19FD1D6D-EAB1-4606-B53B-C6CBDD7DDC5D@pretaweb.com>
Message-ID: <CACBKDnidJbwN_HEpoL3nHcdnAFi6Xn0mefJO5ekMvxSg5n-Brw@mail.gmail.com>

Dyn offers those services as well, and I believe they are the current
DNS provider for python.org, If they are going to add those sort of
features, (which I have suggested in the past), then it might be best
to stick the current provider.

http://dyn.com/dns/dynect-managed-dns/


Ken


On Mon, Apr 16, 2012 at 1:04 PM, Dylan Jay <djay at pretaweb.com> wrote:
>
> On 16/04/2012, at 11:30 PM, Martin v. L?wis wrote:
>
>>> Why not go to the next level and get the script on this page to update a
>>> DNS load balancer with all green mirrors?
>>
>>
>> I still think that mirror selection should be on the client side. First,
>> if you have an infrastructure that updates the mirror list, it is again
>> a single point of failure. Plus, clients would want to select a mirror
>> based on network performance, which a DNS load balancer cannot achieve.
>
>
> One of the services offered by edgedirector.com etc is "geo-targetted global
> load balancing" which means they will serve up the IP closest to your
> location.
>
> In any case offering a load balanced domain like "any.pypi.org" doesn't
> preclude individual mirror domains. The user could choose.
>
>
>>
>> Regards,
>> Martin
>
>

From kencochrane at gmail.com  Mon Apr 16 21:49:47 2012
From: kencochrane at gmail.com (ken cochrane)
Date: Mon, 16 Apr 2012 15:49:47 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <4F8BC8EB.3050303@ziade.org>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<4F8BC8EB.3050303@ziade.org>
Message-ID: <CACBKDngWg86NsZkSjxmsBBbH2uiFZ3hA+xpWv2qJfn3gk8gB8A@mail.gmail.com>

On the list, thanks..



On Mon, Apr 16, 2012 at 3:23 AM, Tarek Ziad? <tarek at ziade.org> wrote:
>
> a nice feature would be to send a mail automatically in this ML when a
> mirror becomes "old"

From donald.stufft at gmail.com  Mon Apr 16 21:52:19 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Mon, 16 Apr 2012 15:52:19 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>
	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>
Message-ID: <529FB8BD64A54ED6854C34A29A15821F@gmail.com>

On Monday, April 16, 2012 at 3:46 PM, ken cochrane wrote:
> Is it possible to get the number of packages on each mirror? If so,
> can you let me know and I'll look into adding it.
> 
> 

Not without parsing the simple api. If you just want to know the total number of
names, you can parse the index page for number of links. If you want the total number
of files then you'll need to descend into all 20kish pages as well and pull links out of there.
> 
> On Mon, Apr 16, 2012 at 5:15 AM, Sebastien Douche <sdouche at gmail.com (mailto:sdouche at gmail.com)> wrote:
> 
> > Nice! Thanks Ken. Missing only an information imho: the number of
> > packages on each mirror (to known if the mirror is really up to date).
> > 
> 
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org (mailto:Catalog-SIG at python.org)
> http://mail.python.org/mailman/listinfo/catalog-sig
> 
> 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120416/5905a15d/attachment-0001.html>

From lists at zopyx.com  Mon Apr 16 21:53:34 2012
From: lists at zopyx.com (Andreas Jung)
Date: Mon, 16 Apr 2012 21:53:34 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <CACBKDni-QVMHKQBpw73c4E3Pp6aEwR9rfW21AtrtKrDwW5H+hg@mail.gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<4F8B9321.3000702@zopyx.com>
	<CACBKDni-QVMHKQBpw73c4E3Pp6aEwR9rfW21AtrtKrDwW5H+hg@mail.gmail.com>
Message-ID: <4F8C78BE.7020407@zopyx.com>

The service tells me 

DE; State/Province : NORDRHEIN-WESTFALEN

-aj

ken cochrane wrote:
> Sorry about that, I'm getting the location from http://ipinfodb.com
> feel free to update their location for that IP with the correct one.
>
>
>
> On Sun, Apr 15, 2012 at 11:33 PM, Andreas Jung <lists at zopyx.com> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> This here is weird
>>
>> c.pypi.python.org       Russian Federation
>>
>> That's my server it it is running in Germany :)
>>
>> - -aj
>>
>> ken cochrane wrote:
>>> I have taken your advice and I have updated it so that it is a
>>> little clearer. I changed the times a little but I stuck to the 3
>>> statuses that you proposed.
>>>
>>> I also added a bunch of other features to the site, so feel free to
>>> check it out and let me know if you can think of anything else.
>>>
>>> http://www.pypi-mirrors.org
>>>
>>>
>>> On Sun, Apr 15, 2012 at 11:11 AM, Hanno Schlichting
>>> <hanno at hannosch.eu> wrote:
>>>> On Sun, Apr 15, 2012 at 4:44 PM, Ken Cochrane
>>>> <kencochrane at gmail.com> wrote:
>>>>> Yeah I need to clean that up and make it more standard but this
>>>>> is how it works now.
>>>>>
>>>>> Age < 5 min = excellent Age > 5 min and age < 15 min = awesome
>>>>> Age > 15 min and age < 1 hour = great. Age > 1 hour and age <  6
>>>>> hour = good Age > 6 hours and age < 12 hour = OK Age > 12 hours
>>>>> and age < 1 day = getting stale Age > 1 day = out of date
>>>>>
>>>>> Feel free to make suggestions on how to improve.
>>>> My suggestion, keep it simple:
>>>>
>>>> Age < 15 minutes - green (fresh) Age < 1 day - yellow (oldish) Age
>>>>> 1 day - red (old)
>>>> Apache and CPAN mirrors are much more forgiving.
>>>>
>>>> http://www.apache.org/mirrors/#age-histogram
>>>>
>>>> < 30 hours - green < 54 hours - yellow
>>>>> 54 hours - red
>>>> http://mirrors.cpan.org/#age-histogram
>>>>
>>>> < 2 days green < 4days - yellow
>>>>> 4 days - red
>>>> Hanno
>>> _______________________________________________ Catalog-SIG mailing
>>> list Catalog-SIG at python.org
>>> http://mail.python.org/mailman/listinfo/catalog-sig
>> - --
>> ZOPYX Limited           | zopyx group
>> Charlottenstr. 37/1     | The full-service network for Zope & Plone
>> D-72070 T?bingen        | Produce & Publish
>> www.zopyx.com           | www.produce-and-publish.com
>> - ------------------------------------------------------------------------
>> E-Publishing, Python, Zope & Plone development, Consulting
>>
>>
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.11 (Darwin)
>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>>
>> iQGUBAEBAgAGBQJPi5MhAAoJEADcfz7u4AZjNNkLwI4YhWH6JND0j4OKdQJEqk63
>> caeRq+h8q+Pt4eAE+MiXkLxyx3VRK0tSEesGLFTC39aGKuSMKfpbKzYtc5dT8qIX
>> +WAhzrHw1MMzO7s6ShATv2VDYqcSCrsJx2KcHexxdpawpQTh9G53dnHRnyT6/Cq/
>> CaQHXnIyTgCbh3xK8ruZWJ098lvr6Nqj2mS4JFILb0qf6N+4nNR+6FeEITqFf7HS
>> Hemwx6i5m2ZIx1gvPIsoqFnNNvLBSCMdluFPVESkIPf6Ocm19ykHYKTOY+6I7iCV
>> zUNkKaucA6sTOW3OMQ8MR4/Km2tBy+PPjRx/0wY99SaPDAenvTGovq62koaYTBs7
>> NE4Cd5P1ydUkNVXFIyVc/ZiAWAJCGg5FL7nxKZRbQ2fb9L7YGYYVIhy3AaOziL6T
>> qBFT35uz3ZWh0linYUPNpHplJ7V6hdEazA5AgVaVaeNyor86JIYtuxZOC8UBHQQl
>> hk686sVopjJUCzu3UZOT8eoZlIOqpIA=
>> =O4Ej
>> -----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lists.vcf
Type: text/x-vcard
Size: 324 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120416/a7439496/attachment.vcf>

From kencochrane at gmail.com  Mon Apr 16 21:58:33 2012
From: kencochrane at gmail.com (ken cochrane)
Date: Mon, 16 Apr 2012 15:58:33 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <529FB8BD64A54ED6854C34A29A15821F@gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>
	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>
	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>
Message-ID: <CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>

Ouch, does that take a while to gather the number of files?
Is there any open source code that already does this, that I can look at?
I'm thinking packages might be find for now.

Thanks,
Ken


On Mon, Apr 16, 2012 at 3:52 PM, Donald Stufft <donald.stufft at gmail.com> wrote:
> On Monday, April 16, 2012 at 3:46 PM, ken cochrane wrote:
>
> Is it possible to get the number of packages on each mirror? If so,
> can you let me know and I'll look into adding it.
>
> Not without parsing the simple api. If you just want to know the total
> number of
> names, you can parse the index page for number of links. If you want the
> total number
> of files then you'll need to descend into all 20kish pages as well and pull
> links out of there.
>
>
> On Mon, Apr 16, 2012 at 5:15 AM, Sebastien Douche <sdouche at gmail.com> wrote:
>
> Nice! Thanks Ken. Missing only an information imho: the number of
> packages on each mirror (to known if the mirror is really up to date).
>
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
>
>

From kencochrane at gmail.com  Mon Apr 16 22:43:14 2012
From: kencochrane at gmail.com (ken cochrane)
Date: Mon, 16 Apr 2012 16:43:14 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>
	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>
	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>
	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>
Message-ID: <CACBKDnhqihZdGmqGBKQrnFt-SbMPE1jEpChGRNJEwfLS8qzL7g@mail.gmail.com>

OK, Thanks to Donald's help, I know show the number of packages for each mirror.

Ken


On Mon, Apr 16, 2012 at 3:58 PM, ken cochrane <kencochrane at gmail.com> wrote:
> Ouch, does that take a while to gather the number of files?
> Is there any open source code that already does this, that I can look at?
> I'm thinking packages might be find for now.
>
> Thanks,
> Ken
>
>
> On Mon, Apr 16, 2012 at 3:52 PM, Donald Stufft <donald.stufft at gmail.com> wrote:
>> On Monday, April 16, 2012 at 3:46 PM, ken cochrane wrote:
>>
>> Is it possible to get the number of packages on each mirror? If so,
>> can you let me know and I'll look into adding it.
>>
>> Not without parsing the simple api. If you just want to know the total
>> number of
>> names, you can parse the index page for number of links. If you want the
>> total number
>> of files then you'll need to descend into all 20kish pages as well and pull
>> links out of there.
>>
>>
>> On Mon, Apr 16, 2012 at 5:15 AM, Sebastien Douche <sdouche at gmail.com> wrote:
>>
>> Nice! Thanks Ken. Missing only an information imho: the number of
>> packages on each mirror (to known if the mirror is really up to date).
>>
>> _______________________________________________
>> Catalog-SIG mailing list
>> Catalog-SIG at python.org
>> http://mail.python.org/mailman/listinfo/catalog-sig
>>
>>

From tarek at ziade.org  Mon Apr 16 22:48:36 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 16 Apr 2012 22:48:36 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>
	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>
	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>
	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>
Message-ID: <4F8C85A4.2010905@ziade.org>

On 4/16/12 9:58 PM, ken cochrane wrote:
> Ouch, does that take a while to gather the number of files?
> Is there any open source code that already does this, that I can look at?
> I'm thinking packages might be find for now.
the other option would be to change the mirroring protocol to have that 
total in a static page, like the timestamp

>
> Thanks,
> Ken
>
>
> On Mon, Apr 16, 2012 at 3:52 PM, Donald Stufft<donald.stufft at gmail.com>  wrote:
>> On Monday, April 16, 2012 at 3:46 PM, ken cochrane wrote:
>>
>> Is it possible to get the number of packages on each mirror? If so,
>> can you let me know and I'll look into adding it.
>>
>> Not without parsing the simple api. If you just want to know the total
>> number of
>> names, you can parse the index page for number of links. If you want the
>> total number
>> of files then you'll need to descend into all 20kish pages as well and pull
>> links out of there.
>>
>>
>> On Mon, Apr 16, 2012 at 5:15 AM, Sebastien Douche<sdouche at gmail.com>  wrote:
>>
>> Nice! Thanks Ken. Missing only an information imho: the number of
>> packages on each mirror (to known if the mirror is really up to date).
>>
>> _______________________________________________
>> Catalog-SIG mailing list
>> Catalog-SIG at python.org
>> http://mail.python.org/mailman/listinfo/catalog-sig
>>
>>
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig


From kencochrane at gmail.com  Mon Apr 16 22:57:58 2012
From: kencochrane at gmail.com (ken cochrane)
Date: Mon, 16 Apr 2012 16:57:58 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <4F8C85A4.2010905@ziade.org>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>
	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>
	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>
	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>
	<4F8C85A4.2010905@ziade.org>
Message-ID: <CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>

That would be nice, you could include other stats information as well
(not sure what yet).

Ken


On Mon, Apr 16, 2012 at 4:48 PM, Tarek Ziad? <tarek at ziade.org> wrote:
> On 4/16/12 9:58 PM, ken cochrane wrote:
>>
>> Ouch, does that take a while to gather the number of files?
>> Is there any open source code that already does this, that I can look at?
>> I'm thinking packages might be find for now.
>
> the other option would be to change the mirroring protocol to have that
> total in a static page, like the timestamp
>
>
>>
>> Thanks,
>> Ken
>>
>>
>> On Mon, Apr 16, 2012 at 3:52 PM, Donald Stufft<donald.stufft at gmail.com>
>> ?wrote:
>>>
>>> On Monday, April 16, 2012 at 3:46 PM, ken cochrane wrote:
>>>
>>> Is it possible to get the number of packages on each mirror? If so,
>>> can you let me know and I'll look into adding it.
>>>
>>> Not without parsing the simple api. If you just want to know the total
>>> number of
>>> names, you can parse the index page for number of links. If you want the
>>> total number
>>> of files then you'll need to descend into all 20kish pages as well and
>>> pull
>>> links out of there.
>>>
>>>
>>> On Mon, Apr 16, 2012 at 5:15 AM, Sebastien Douche<sdouche at gmail.com>
>>> ?wrote:
>>>
>>> Nice! Thanks Ken. Missing only an information imho: the number of
>>> packages on each mirror (to known if the mirror is really up to date).
>>>
>>> _______________________________________________
>>> Catalog-SIG mailing list
>>> Catalog-SIG at python.org
>>> http://mail.python.org/mailman/listinfo/catalog-sig
>>>
>>>
>> _______________________________________________
>> Catalog-SIG mailing list
>> Catalog-SIG at python.org
>> http://mail.python.org/mailman/listinfo/catalog-sig
>
>
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig

From tarek at ziade.org  Mon Apr 16 23:05:53 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 16 Apr 2012 23:05:53 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>
	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>
	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>
	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>
	<4F8C85A4.2010905@ziade.org>
	<CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>
Message-ID: <4F8C89B1.20000@ziade.org>

On 4/16/12 10:57 PM, ken cochrane wrote:
> That would be nice, you could include other stats information as well
> (not sure what yet).
sounds like a nice "checksum" addition to PEP 381- that'd avoid 
client-side parsing just to get the number of mirrored packages.

>
> Ken
>
>
> On Mon, Apr 16, 2012 at 4:48 PM, Tarek Ziad?<tarek at ziade.org>  wrote:
>> On 4/16/12 9:58 PM, ken cochrane wrote:
>>> Ouch, does that take a while to gather the number of files?
>>> Is there any open source code that already does this, that I can look at?
>>> I'm thinking packages might be find for now.
>> the other option would be to change the mirroring protocol to have that
>> total in a static page, like the timestamp
>>
>>
>>> Thanks,
>>> Ken
>>>
>>>
>>> On Mon, Apr 16, 2012 at 3:52 PM, Donald Stufft<donald.stufft at gmail.com>
>>>   wrote:
>>>> On Monday, April 16, 2012 at 3:46 PM, ken cochrane wrote:
>>>>
>>>> Is it possible to get the number of packages on each mirror? If so,
>>>> can you let me know and I'll look into adding it.
>>>>
>>>> Not without parsing the simple api. If you just want to know the total
>>>> number of
>>>> names, you can parse the index page for number of links. If you want the
>>>> total number
>>>> of files then you'll need to descend into all 20kish pages as well and
>>>> pull
>>>> links out of there.
>>>>
>>>>
>>>> On Mon, Apr 16, 2012 at 5:15 AM, Sebastien Douche<sdouche at gmail.com>
>>>>   wrote:
>>>>
>>>> Nice! Thanks Ken. Missing only an information imho: the number of
>>>> packages on each mirror (to known if the mirror is really up to date).
>>>>
>>>> _______________________________________________
>>>> Catalog-SIG mailing list
>>>> Catalog-SIG at python.org
>>>> http://mail.python.org/mailman/listinfo/catalog-sig
>>>>
>>>>
>>> _______________________________________________
>>> Catalog-SIG mailing list
>>> Catalog-SIG at python.org
>>> http://mail.python.org/mailman/listinfo/catalog-sig
>>
>> _______________________________________________
>> Catalog-SIG mailing list
>> Catalog-SIG at python.org
>> http://mail.python.org/mailman/listinfo/catalog-sig


From martin at v.loewis.de  Mon Apr 16 23:23:07 2012
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 16 Apr 2012 23:23:07 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <4F8C89B1.20000@ziade.org>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>	<4F8C85A4.2010905@ziade.org>	<CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>
	<4F8C89B1.20000@ziade.org>
Message-ID: <4F8C8DBB.5020409@v.loewis.de>

Am 16.04.2012 23:05, schrieb Tarek Ziad?:
> On 4/16/12 10:57 PM, ken cochrane wrote:
>> That would be nice, you could include other stats information as well
>> (not sure what yet).
> sounds like a nice "checksum" addition to PEP 381- that'd avoid
> client-side parsing just to get the number of mirrored packages.

I'm not sure what purpose this would serve. I already don't see the
point in computing the number of mirrored packages or the number of
mirrored files (other than to rejoice the PyPI maintainer: wow,
we have 20545 packages in PyPI! It was only 10,000 just a short time
ago).

I think the original requester of this feature suggested that as a
sanity check, apparently not trusting the last-modified information.
If it's really this kind of distrust, any generated stats information
should be just as distrusted. For example, it may be that mirrors
failed to correctly copy one of the files, or show other
inconsistencies. No mirror-generated statistics would indicate this
problem.

So before extending the protocol, there should be some serious
motivation for it.

Regards,
Martin

From tarek at ziade.org  Mon Apr 16 23:45:27 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 16 Apr 2012 23:45:27 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <4F8C8DBB.5020409@v.loewis.de>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>	<4F8C85A4.2010905@ziade.org>	<CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>
	<4F8C89B1.20000@ziade.org> <4F8C8DBB.5020409@v.loewis.de>
Message-ID: <4F8C92F7.1020404@ziade.org>

On 4/16/12 11:23 PM, "Martin v. L?wis" wrote:
> Am 16.04.2012 23:05, schrieb Tarek Ziad?:
>> On 4/16/12 10:57 PM, ken cochrane wrote:
>>> That would be nice, you could include other stats information as well
>>> (not sure what yet).
>> sounds like a nice "checksum" addition to PEP 381- that'd avoid
>> client-side parsing just to get the number of mirrored packages.
> I'm not sure what purpose this would serve. I already don't see the
> point in computing the number of mirrored packages or the number of
> mirrored files (other than to rejoice the PyPI maintainer: wow,
> we have 20545 packages in PyPI! It was only 10,000 just a short time
> ago).
>
> I think the original requester of this feature suggested that as a
> sanity check, apparently not trusting the last-modified information.
Yeah that's what I understood too. So I used the term 'checksum'
> If it's really this kind of distrust, any generated stats information
> should be just as distrusted. For example, it may be that mirrors
> failed to correctly copy one of the files, or show other
> inconsistencies. No mirror-generated statistics would indicate this
> problem.
the number of packages is a good indication of a successful mirroring - 
I think that was the point.

I can definitely see a mirroring implementation where the last-modified 
field is updated at the end
while some packages are not copied over at the end for whatever network 
issue.

Maybe a better checksum would be a global hash calculated differently ?

>
> So before extending the protocol, there should be some serious
> motivation for it.
>
> Regards,
> Martin


From martin at v.loewis.de  Mon Apr 16 23:57:51 2012
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 16 Apr 2012 23:57:51 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <4F8C92F7.1020404@ziade.org>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>	<4F8C85A4.2010905@ziade.org>	<CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>
	<4F8C89B1.20000@ziade.org> <4F8C8DBB.5020409@v.loewis.de>
	<4F8C92F7.1020404@ziade.org>
Message-ID: <4F8C95DF.4020703@v.loewis.de>

> Maybe a better checksum would be a global hash calculated differently ?

Define a protocol, and I present you with an implementation that
conforms to the protocol, and still has inconsistent data, and not
in a malicious manner, but due to bugs/race conditions/unexpected
events. It's pointless. Ultimately, clients will need to verify the
data that they receive (if they suspect issues), and fall back gracefully.

> I can definitely see a mirroring implementation where the
> last-modified field is updated at the end while some packages are not
> copied over at the end for whatever network issue.

That mirroring implementation would violate the principle that
last-modified should only be updated when the mirroring run was
completed successfully.

Regards,
Martin

From tarek at ziade.org  Tue Apr 17 00:09:36 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 17 Apr 2012 00:09:36 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <4F8C95DF.4020703@v.loewis.de>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>	<4F8C85A4.2010905@ziade.org>	<CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>
	<4F8C89B1.20000@ziade.org> <4F8C8DBB.5020409@v.loewis.de>
	<4F8C92F7.1020404@ziade.org> <4F8C95DF.4020703@v.loewis.de>
Message-ID: <4F8C98A0.5030909@ziade.org>

On 4/16/12 11:57 PM, "Martin v. L?wis" wrote:
>> Maybe a better checksum would be a global hash calculated differently ?
> Define a protocol, and I present you with an implementation that
> conforms to the protocol, and still has inconsistent data, and not
> in a malicious manner, but due to bugs/race conditions/unexpected
> events. It's pointless.
if you calculate a checksum with all mirrored files - you can guarantee 
that the bits are the same
on both side, no ? then you have the md5 checksum per file for the 
download so the client can
be sure he got a non corrupted file.

> Ultimately, clients will need to verify the
> data that they receive (if they suspect issues), and fall back gracefully.
how can they know if version 1.3 of package foo never made it to the 
mirror they use ?

They can't. They have to trust the last modified date and make the 
assumption that the mirror
is fresh enough, for foo 1.3 to be present in both the master and the 
mirror.

>
>> I can definitely see a mirroring implementation where the
>> last-modified field is updated at the end while some packages are not
>> copied over at the end for whatever network issue.
> That mirroring implementation would violate the principle that
> last-modified should only be updated when the mirroring run was
> completed successfully.
like a file that claims in its metadata it's 512 kb long and only has 2 
kb in reality because something went wrong ?

I think the idea of the checksum is to double-check that kind of claim.  
But maybe that's overkill ?
maybe the mirroring code should check file by file that everything was 
copied correctly ?

Cheers
Tarek

>
> Regards,
> Martin


From donald.stufft at gmail.com  Tue Apr 17 00:10:32 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Mon, 16 Apr 2012 18:10:32 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <4F8C95DF.4020703@v.loewis.de>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>
	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>
	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>
	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>
	<4F8C85A4.2010905@ziade.org>
	<CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>
	<4F8C89B1.20000@ziade.org> <4F8C8DBB.5020409@v.loewis.de>
	<4F8C92F7.1020404@ziade.org> <4F8C95DF.4020703@v.loewis.de>
Message-ID: <21C506F457BF4A04A40DD594731022B4@gmail.com>

On Monday, April 16, 2012 at 5:57 PM, "Martin v. L?wis" wrote:
> > Maybe a better checksum would be a global hash calculated differently ?
>  
>  
> Define a protocol, and I present you with an implementation that
> conforms to the protocol, and still has inconsistent data, and not
> in a malicious manner, but due to bugs/race conditions/unexpected
> events. It's pointless. Ultimately, clients will need to verify the
> data that they receive (if they suspect issues), and fall back gracefully.
>  
> > I can definitely see a mirroring implementation where the
> > last-modified field is updated at the end while some packages are not
> > copied over at the end for whatever network issue.
> >  
>  
>  
> That mirroring implementation would violate the principle that
> last-modified should only be updated when the mirroring run was
> completed successfully.
>  
>  

Is this documented anywhere? I don't see it in PEP381. I agree that
last-modified should only be updated when mirroring was completed
successfully but I don't see that specified as part of the mirroring protocol
which could lead to inconsistent definitions of what last-modified is.  
>  
> Regards,
> Martin
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org (mailto:Catalog-SIG at python.org)
> http://mail.python.org/mailman/listinfo/catalog-sig
>  
>  


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120416/51a43423/attachment-0001.html>

From donald.stufft at gmail.com  Tue Apr 17 00:14:11 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Mon, 16 Apr 2012 18:14:11 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <4F8C98A0.5030909@ziade.org>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>
	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>
	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>
	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>
	<4F8C85A4.2010905@ziade.org>
	<CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>
	<4F8C89B1.20000@ziade.org> <4F8C8DBB.5020409@v.loewis.de>
	<4F8C92F7.1020404@ziade.org> <4F8C95DF.4020703@v.loewis.de>
	<4F8C98A0.5030909@ziade.org>
Message-ID: <B81C0EEAA7BC46AEA5668E8A56D7CB98@gmail.com>



On Monday, April 16, 2012 at 6:09 PM, Tarek Ziad? wrote:

> On 4/16/12 11:57 PM, "Martin v. L?wis" wrote:
> > > Maybe a better checksum would be a global hash calculated differently ?
> >  
> > Define a protocol, and I present you with an implementation that
> > conforms to the protocol, and still has inconsistent data, and not
> > in a malicious manner, but due to bugs/race conditions/unexpected
> > events. It's pointless.
> >  
>  
> if you calculate a checksum with all mirrored files - you can guarantee  
> that the bits are the same
> on both side, no ? then you have the md5 checksum per file for the  
> download so the client can
> be sure he got a non corrupted file.
>  
> > Ultimately, clients will need to verify the
> > data that they receive (if they suspect issues), and fall back gracefully.
> >  
>  
> how can they know if version 1.3 of package foo never made it to the  
> mirror they use ?
>  
> They can't. They have to trust the last modified date and make the  
> assumption that the mirror
> is fresh enough, for foo 1.3 to be present in both the master and the  
> mirror.
>  
> >  
> > > I can definitely see a mirroring implementation where the
> > > last-modified field is updated at the end while some packages are not
> > > copied over at the end for whatever network issue.
> > >  
> >  
> > That mirroring implementation would violate the principle that
> > last-modified should only be updated when the mirroring run was
> > completed successfully.
> >  
>  
> like a file that claims in its metadata it's 512 kb long and only has 2  
> kb in reality because something went wrong ?
>  
> I think the idea of the checksum is to double-check that kind of claim.  
> But maybe that's overkill ?
> maybe the mirroring code should check file by file that everything was  
> copied correctly ?
>  
>  

fwiw Crate verifies the md5 hashes that the simple api gives for each package. I think not
doing that should be considered wrong. (it's considered important for clients to check the
checksum of packages they download, but mirrors that are going to be redistributing their
files to clients this isn't important? Seems to be a disconnect between the thoughts).
>  
> Cheers
> Tarek
>  
> >  
> > Regards,
> > Martin
> >  
>  
>  
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org (mailto:Catalog-SIG at python.org)
> http://mail.python.org/mailman/listinfo/catalog-sig
>  
>  


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120416/10a7982f/attachment.html>

From martin at v.loewis.de  Tue Apr 17 00:19:07 2012
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 17 Apr 2012 00:19:07 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <4F8C98A0.5030909@ziade.org>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>	<4F8C85A4.2010905@ziade.org>	<CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>
	<4F8C89B1.20000@ziade.org> <4F8C8DBB.5020409@v.loewis.de>
	<4F8C92F7.1020404@ziade.org> <4F8C95DF.4020703@v.loewis.de>
	<4F8C98A0.5030909@ziade.org>
Message-ID: <4F8C9ADB.4040603@v.loewis.de>

Am 17.04.2012 00:09, schrieb Tarek Ziad?:
> On 4/16/12 11:57 PM, "Martin v. L?wis" wrote:
>>> Maybe a better checksum would be a global hash calculated differently ?
>> Define a protocol, and I present you with an implementation that
>> conforms to the protocol, and still has inconsistent data, and not
>> in a malicious manner, but due to bugs/race conditions/unexpected
>> events. It's pointless.
> if you calculate a checksum with all mirrored files - you can guarantee
> that the bits are the same
> on both side, no ?

How exactly would you calculate that checksum? Would you really require
concatenation of all files? That could take a few hours per change. It
would also raise the question in what order the files ought to be
concatenated.

> how can they know if version 1.3 of package foo never made it to the
> mirror they use ?
> 
> They can't. They have to trust the last modified date and make the
> assumption that the mirror
> is fresh enough, for foo 1.3 to be present in both the master and the
> mirror.

How could they do so using your protocol?

> I think the idea of the checksum is to double-check that kind of claim. 
> But maybe that's overkill ?

I think it's both overkill, and it doesn't help.

> maybe the mirroring code should check file by file that everything was
> copied correctly ?

If you also assume malicious mirrors, then you definitely need to check
every file, as specified in

http://www.python.org/dev/peps/pep-0381/#mirror-authenticity

However, if a mirror claims it is up-to-date, and that verification
fails, my recommendation would be to give up in the tool and have
the user submit a bug report, in order to eliminate the mirror from
the mirror list.

Regards,
Martin

From martin at v.loewis.de  Tue Apr 17 00:20:51 2012
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 17 Apr 2012 00:20:51 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <21C506F457BF4A04A40DD594731022B4@gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>
	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>
	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>
	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>
	<4F8C85A4.2010905@ziade.org>
	<CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>
	<4F8C89B1.20000@ziade.org> <4F8C8DBB.5020409@v.loewis.de>
	<4F8C92F7.1020404@ziade.org> <4F8C95DF.4020703@v.loewis.de>
	<21C506F457BF4A04A40DD594731022B4@gmail.com>
Message-ID: <4F8C9B43.6070700@v.loewis.de>

>> That mirroring implementation would violate the principle that
>> last-modified should only be updated when the mirroring run was
>> completed successfully.
> Is this documented anywhere? I don't see it in PEP381.

It's certainly meant by "the last synchronisation date the mirror
maintains". Should I qualify that with "successful synchronization"
to make it more clear?

Regards,
Martin

From martin at v.loewis.de  Tue Apr 17 00:22:54 2012
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 17 Apr 2012 00:22:54 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <B81C0EEAA7BC46AEA5668E8A56D7CB98@gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>	<4F8C85A4.2010905@ziade.org>	<CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>	<4F8C89B1.20000@ziade.org>
	<4F8C8DBB.5020409@v.loewis.de>	<4F8C92F7.1020404@ziade.org>
	<4F8C95DF.4020703@v.loewis.de>	<4F8C98A0.5030909@ziade.org>
	<B81C0EEAA7BC46AEA5668E8A56D7CB98@gmail.com>
Message-ID: <4F8C9BBE.8040808@v.loewis.de>

> fwiw Crate verifies the md5 hashes that the simple api gives for
> each package. I think not doing that should be considered wrong.
> (it's considered important for clients to check the checksum of
> packages they download, but mirrors that are going to be 
> redistributing their files to clients this isn't important? Seems to
> be a disconnect between the thoughts).

I think this is a quality-of-implementation issue. They could verify
the md5; they could also verify the serversig. Some do, some don't.
Or they could claim they do but actually don't. Ultimately, clients
either need to trust the mirror, or do the verification themselves.

Regards,
Martin

From donald.stufft at gmail.com  Tue Apr 17 00:24:47 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Mon, 16 Apr 2012 18:24:47 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <4F8C9B43.6070700@v.loewis.de>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>
	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>
	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>
	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>
	<4F8C85A4.2010905@ziade.org>
	<CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>
	<4F8C89B1.20000@ziade.org> <4F8C8DBB.5020409@v.loewis.de>
	<4F8C92F7.1020404@ziade.org> <4F8C95DF.4020703@v.loewis.de>
	<21C506F457BF4A04A40DD594731022B4@gmail.com>
	<4F8C9B43.6070700@v.loewis.de>
Message-ID: <EBC2B0B20999433198893A0496B2C7F8@gmail.com>



On Monday, April 16, 2012 at 6:20 PM, "Martin v. L?wis" wrote:

> > > That mirroring implementation would violate the principle that
> > > last-modified should only be updated when the mirroring run was
> > > completed successfully.
> > >  
> >  
> > Is this documented anywhere? I don't see it in PEP381.
> >  
>  
>  
> It's certainly meant by "the last synchronisation date the mirror
> maintains". Should I qualify that with "successful synchronization"
> to make it more clear?
>  
>  

That is more clear. Don't mean to be pedantic about it, just that when I
personally was implementing pep381 the fact that should be stored
_after_ the synchronization didn't occur to me (I ended up doing it that way
for other reasons anyways) and I thought that I might not be the only one.  
>  
> Regards,
> Martin
>  
>  


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120416/0fc80a21/attachment-0001.html>

From donald.stufft at gmail.com  Tue Apr 17 00:31:54 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Mon, 16 Apr 2012 18:31:54 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <4F8C9BBE.8040808@v.loewis.de>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>
	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>
	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>
	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>
	<4F8C85A4.2010905@ziade.org>
	<CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>
	<4F8C89B1.20000@ziade.org> <4F8C8DBB.5020409@v.loewis.de>
	<4F8C92F7.1020404@ziade.org> <4F8C95DF.4020703@v.loewis.de>
	<4F8C98A0.5030909@ziade.org>
	<B81C0EEAA7BC46AEA5668E8A56D7CB98@gmail.com>
	<4F8C9BBE.8040808@v.loewis.de>
Message-ID: <00FBE05634684DA298AB73F776572845@gmail.com>

On Monday, April 16, 2012 at 6:22 PM, "Martin v. L?wis" wrote:
> > fwiw Crate verifies the md5 hashes that the simple api gives for
> > each package. I think not doing that should be considered wrong.
> > (it's considered important for clients to check the checksum of
> > packages they download, but mirrors that are going to be  
> > redistributing their files to clients this isn't important? Seems to
> > be a disconnect between the thoughts).
> >  
>  
>  
> I think this is a quality-of-implementation issue. They could verify
> the md5; they could also verify the serversig. Some do, some don't.
> Or they could claim they do but actually don't. Ultimately, clients
> either need to trust the mirror, or do the verification themselves.
>  
>  

Yea ultimately the clients will need to do the verification themselves, I
only meant to have somewhat more useful mirrors (corrupted data,  bugs
in scripts, etc) it would make sense for mirrors to verify that the data they
are getting is the data they are expecting so they don't serve bad data. While
a verifying client won't be affected by that bad data, it does lower the overall
availability for that file.
>  
> Regards,
> Martin
>  
>  


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120416/0034f1ef/attachment.html>

From tarek at ziade.org  Tue Apr 17 00:48:05 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 17 Apr 2012 00:48:05 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <4F8C9ADB.4040603@v.loewis.de>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>	<4F8C85A4.2010905@ziade.org>	<CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>
	<4F8C89B1.20000@ziade.org> <4F8C8DBB.5020409@v.loewis.de>
	<4F8C92F7.1020404@ziade.org> <4F8C95DF.4020703@v.loewis.de>
	<4F8C98A0.5030909@ziade.org> <4F8C9ADB.4040603@v.loewis.de>
Message-ID: <4F8CA1A5.2050909@ziade.org>

On 4/17/12 12:19 AM, "Martin v. L?wis" wrote:
> Am 17.04.2012 00:09, schrieb Tarek Ziad?:
>> On 4/16/12 11:57 PM, "Martin v. L?wis" wrote:
>>>> Maybe a better checksum would be a global hash calculated differently ?
>>> Define a protocol, and I present you with an implementation that
>>> conforms to the protocol, and still has inconsistent data, and not
>>> in a malicious manner, but due to bugs/race conditions/unexpected
>>> events. It's pointless.
>> if you calculate a checksum with all mirrored files - you can guarantee
>> that the bits are the same
>> on both side, no ?
> How exactly would you calculate that checksum?
by calculating the grand hash of each file hash.
>   Would you really require
> concatenation of all files?
I did not say that. You are claiming it in a rhetorical question.

> That could take a few hours per change.
why that ? you don't calculate the checksum of a file your already have 
twice.

Even if you do, it's very fast to call md5.

try it:

$ find mirror | xargs md5

this takes a few seconds at most on the whole mirror

> It
> would also raise the question in what order the files ought to be
> concatenated.
Anything reproductible, a sorted list.  In bash I *suspect* the 
calculation of the grand hash of the mirror is a one-liner that takes 
less than a minute.

I am going to stop here anyways because I don't see the point of 
discussing implementation details at this stage, since we were
barely starting to talk about the idea of a checksum - and that seems to 
be going nowhere.

Cheers
Tarek




From martin at v.loewis.de  Tue Apr 17 11:57:11 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Tue, 17 Apr 2012 11:57:11 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <4F8CA1A5.2050909@ziade.org>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>
	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>
	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>
	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>
	<4F8C85A4.2010905@ziade.org>
	<CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>
	<4F8C89B1.20000@ziade.org> <4F8C8DBB.5020409@v.loewis.de>
	<4F8C92F7.1020404@ziade.org> <4F8C95DF.4020703@v.loewis.de>
	<4F8C98A0.5030909@ziade.org> <4F8C9ADB.4040603@v.loewis.de>
	<4F8CA1A5.2050909@ziade.org>
Message-ID: <20120417115711.Horde.azZHL6GZi1VPjT53LEyEPsA@webmail.df.eu>


>>> if you calculate a checksum with all mirrored files - you can guarantee
>>> that the bits are the same
>>> on both side, no ?
>> How exactly would you calculate that checksum?
> by calculating the grand hash of each file hash.

In this case, the checksum would not be a reliable indication that the
files are actually up-to-date. For example, a mirror may keep updating
files into the wrong location (not the location that is then used to
serve the files), so that the files being served are from a stale copy.
This is not theoretical - it actually happened in my mirror setup at one
time.

>> That could take a few hours per change.
> why that ? you don't calculate the checksum of a file your already  
> have twice.
>
> Even if you do, it's very fast to call md5.
>
> try it:
>
> $ find mirror | xargs md5
>
> this takes a few seconds at most on the whole mirror

I tried it, and on my mirror, it took 27 minutes and 7 seconds.
So not exactly hours, but not "a few seconds" either.

Regards,
Martin



From tarek at ziade.org  Tue Apr 17 12:50:59 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 17 Apr 2012 12:50:59 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <20120417115711.Horde.azZHL6GZi1VPjT53LEyEPsA@webmail.df.eu>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>
	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>
	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>
	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>
	<4F8C85A4.2010905@ziade.org>
	<CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>
	<4F8C89B1.20000@ziade.org> <4F8C8DBB.5020409@v.loewis.de>
	<4F8C92F7.1020404@ziade.org> <4F8C95DF.4020703@v.loewis.de>
	<4F8C98A0.5030909@ziade.org> <4F8C9ADB.4040603@v.loewis.de>
	<4F8CA1A5.2050909@ziade.org>
	<20120417115711.Horde.azZHL6GZi1VPjT53LEyEPsA@webmail.df.eu>
Message-ID: <4F8D4B13.90502@ziade.org>

On 4/17/12 11:57 AM, martin at v.loewis.de wrote:
> > by calculating the grand hash of each file hash.
>
> In this case, the checksum would not be a reliable indication that the
> files are actually up-to-date. For example, a mirror may keep updating
> files into the wrong location (not the location that is then used to
> serve the files), so that the files being served are from a stale copy.
> This is not theoretical - it actually happened in my mirror setup at one
> time.
>
So you were updating a directory but serving another directory ?

But then updating the right last-modified page people were seeing ?

In that case, updating the checksum would have revealed you were on the 
wrong set of files.

Unless you script was updating everything on a stale copy that was not 
published ?


>>> That could take a few hours per change.
>> why that ? you don't calculate the checksum of a file your already 
>> have twice.
>>
>> Even if you do, it's very fast to call md5.
>>
>> try it:
>>
>> $ find mirror | xargs md5
>>
>> this takes a few seconds at most on the whole mirror
>
> I tried it, and on my mirror, it took 27 minutes and 7 seconds.
> So not exactly hours, but not "a few seconds" either.
oops sorry I ran it on the wrong directory, it's true that it takes more 
time !

So on my centos 5 VM - which is quite slow and doing many other stuff 
like running Jenkins jobs, running the "md5deep" program like this : 
http://tarek.pastebin.mozilla.org/1574557

It took 15minutes and 1 second.   It can be optimized of course, since 
most directories are done quickly and everything is in /source. That 
time can be divided by 2 at least with the proper load balancing between 
a few md5 runners.

But that just to be run *once*. You would not compute it on every mirror 
update but keep all md5 values somewhere.

So, recalculating the grand hash on every mirror update should takes a 
few seconds because it would just consist of calculating the hash for 
the new files, then
calculating the grand hash -- a loop that updates a md5 hash with 20k 
hashes takes less than a second if I don't count the file reading.

(see http://tarek.pastebin.mozilla.org/1574574)

I am not sure why we're having this discussion since it's implementation 
details, but it's fun :)

If there's interest I can write a multiprocess-based script that keeps a 
md5 database up-to-date

Cheers
Tarek

>
> Regards,
> Martin
>
>


From donald.stufft at gmail.com  Tue Apr 17 12:54:55 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Tue, 17 Apr 2012 06:54:55 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <4F8D4B13.90502@ziade.org>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>
	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>
	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>
	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>
	<4F8C85A4.2010905@ziade.org>
	<CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>
	<4F8C89B1.20000@ziade.org> <4F8C8DBB.5020409@v.loewis.de>
	<4F8C92F7.1020404@ziade.org> <4F8C95DF.4020703@v.loewis.de>
	<4F8C98A0.5030909@ziade.org> <4F8C9ADB.4040603@v.loewis.de>
	<4F8CA1A5.2050909@ziade.org>
	<20120417115711.Horde.azZHL6GZi1VPjT53LEyEPsA@webmail.df.eu>
	<4F8D4B13.90502@ziade.org>
Message-ID: <6375BB58D5494D61BB0F2599C75B9F93@gmail.com>

On Tuesday, April 17, 2012 at 6:50 AM, Tarek Ziad? wrote:
> On 4/17/12 11:57 AM, martin at v.loewis.de (mailto:martin at v.loewis.de) wrote:
> > > by calculating the grand hash of each file hash.
> >  
> >  
> > In this case, the checksum would not be a reliable indication that the
> > files are actually up-to-date. For example, a mirror may keep updating
> > files into the wrong location (not the location that is then used to
> > serve the files), so that the files being served are from a stale copy.
> > This is not theoretical - it actually happened in my mirror setup at one
> > time.
> >  
>  
> So you were updating a directory but serving another directory ?
>  
> But then updating the right last-modified page people were seeing ?
>  
> In that case, updating the checksum would have revealed you were on the  
> wrong set of files.
>  
> Unless you script was updating everything on a stale copy that was not  
> published ?
>  
>  
> > > > That could take a few hours per change.
> > > why that ? you don't calculate the checksum of a file your already  
> > > have twice.
> > >  
> > > Even if you do, it's very fast to call md5.
> > >  
> > > try it:
> > >  
> > > $ find mirror | xargs md5
> > >  
> > > this takes a few seconds at most on the whole mirror
> >  
> > I tried it, and on my mirror, it took 27 minutes and 7 seconds.
> > So not exactly hours, but not "a few seconds" either.
> >  
>  
> oops sorry I ran it on the wrong directory, it's true that it takes more  
> time !
>  
> So on my centos 5 VM - which is quite slow and doing many other stuff  
> like running Jenkins jobs, running the "md5deep" program like this :  
> http://tarek.pastebin.mozilla.org/1574557
>  
> It took 15minutes and 1 second. It can be optimized of course, since  
> most directories are done quickly and everything is in /source. That  
> time can be divided by 2 at least with the proper load balancing between  
> a few md5 runners.
>  
> But that just to be run *once*. You would not compute it on every mirror  
> update but keep all md5 values somewhere.
>  
> So, recalculating the grand hash on every mirror update should takes a  
> few seconds because it would just consist of calculating the hash for  
> the new files, then
> calculating the grand hash -- a loop that updates a md5 hash with 20k  
> hashes takes less than a second if I don't count the file reading.
>  
> (see http://tarek.pastebin.mozilla.org/1574574)
>  
> I am not sure why we're having this discussion since it's implementation  
> details, but it's fun :)
>  
> If there's interest I can write a multiprocess-based script that keeps a  
> md5 database up-to-date
>  
>  

I'd be interested ;) Although i'd prefer sha256 personally.  
>  
> Cheers
> Tarek
>  
> >  
> > Regards,
> > Martin
> >  
>  
>  
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org (mailto:Catalog-SIG at python.org)
> http://mail.python.org/mailman/listinfo/catalog-sig
>  
>  


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120417/769ebd6d/attachment.html>

From martin at v.loewis.de  Tue Apr 17 15:45:38 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Tue, 17 Apr 2012 15:45:38 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <4F8D4B13.90502@ziade.org>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>
	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>
	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>
	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>
	<4F8C85A4.2010905@ziade.org>
	<CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>
	<4F8C89B1.20000@ziade.org> <4F8C8DBB.5020409@v.loewis.de>
	<4F8C92F7.1020404@ziade.org> <4F8C95DF.4020703@v.loewis.de>
	<4F8C98A0.5030909@ziade.org> <4F8C9ADB.4040603@v.loewis.de>
	<4F8CA1A5.2050909@ziade.org>
	<20120417115711.Horde.azZHL6GZi1VPjT53LEyEPsA@webmail.df.eu>
	<4F8D4B13.90502@ziade.org>
Message-ID: <20120417154538.Horde.71mAClNNcXdPjXQCDvelsiA@webmail.df.eu>

> So you were updating a directory but serving another directory ?
>
> But then updating the right last-modified page people were seeing ?

It probably would have updated an unpublished last-modified as well.

For a similar issue, in appengine, I had duplicate File objects in the
database, and always served the one that GQL happened to return first.
In that case, both last-modified and the checksum might update correctly,
but still, the wrong file might get served.

> I am not sure why we're having this discussion since it's  
> implementation details, but it's fun :)

I'm still trying to prove my claim that it's not feasible to increase
the trustworthiness of a mirror by computing some kind of checksum.
If the mirror has some systematic or random error, it may well be that
the checksum is as-expected, yet the mirror is inconsistent.

> If there's interest I can write a multiprocess-based script that  
> keeps a md5 database up-to-date

That's besides the point. The question is whether doing so would practically
help to improve the consistency, and I believe the answer is: no. It may help
to increase people's trust (which is a subjective manner), which may be
worthwhile itself, but may also backfire if they download inconsistent files
despite the mirror giving "proof" that it is consistent.

Regards,
Martin



From donald.stufft at gmail.com  Tue Apr 17 19:57:11 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Tue, 17 Apr 2012 13:57:11 -0400
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <20120417154538.Horde.71mAClNNcXdPjXQCDvelsiA@webmail.df.eu>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<CANSw7KyhzBnj-y8QcEVmWJqB-XwkSdedkt=9m_3Wzun_047rCA@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>
	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>
	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>
	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>
	<4F8C85A4.2010905@ziade.org>
	<CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>
	<4F8C89B1.20000@ziade.org> <4F8C8DBB.5020409@v.loewis.de>
	<4F8C92F7.1020404@ziade.org> <4F8C95DF.4020703@v.loewis.de>
	<4F8C98A0.5030909@ziade.org> <4F8C9ADB.4040603@v.loewis.de>
	<4F8CA1A5.2050909@ziade.org>
	<20120417115711.Horde.azZHL6GZi1VPjT53LEyEPsA@webmail.df.eu>
	<4F8D4B13.90502@ziade.org>
	<20120417154538.Horde.71mAClNNcXdPjXQCDvelsiA@webmail.df.eu>
Message-ID: <FFDB90A73AC34414B6B4478E406EEB44@gmail.com>

On Tuesday, April 17, 2012 at 9:45 AM, martin at v.loewis.de wrote:
> > So you were updating a directory but serving another directory ?
> > 
> > But then updating the right last-modified page people were seeing ?
> 
> It probably would have updated an unpublished last-modified as well.
> 
> For a similar issue, in appengine, I had duplicate File objects in the
> database, and always served the one that GQL happened to return first.
> In that case, both last-modified and the checksum might update correctly,
> but still, the wrong file might get served.
> 
> > I am not sure why we're having this discussion since it's 
> > implementation details, but it's fun :)
> > 
> 
> 
> I'm still trying to prove my claim that it's not feasible to increase
> the trustworthiness of a mirror by computing some kind of checksum.
> If the mirror has some systematic or random error, it may well be that
> the checksum is as-expected, yet the mirror is inconsistent.
> 
> 

As I thought more about this, I think it is actually feasible. It's not feasible to
increase it to 100%, but in this case, assuming a bug over malevolence, the
bug would have to affect both the updating of the last-modified and the checksum
generation.

It's essentially the same thing as asking another person to check over some work,
that person could miss something, or also be wrong, but it decreases the likelihood
by adding another piece of verification. 
> 
> > If there's interest I can write a multiprocess-based script that 
> > keeps a md5 database up-to-date
> > 
> 
> 
> That's besides the point. The question is whether doing so would practically
> help to improve the consistency, and I believe the answer is: no. It may help
> to increase people's trust (which is a subjective manner), which may be
> worthwhile itself, but may also backfire if they download inconsistent files
> despite the mirror giving "proof" that it is consistent.
> 
> Regards,
> Martin
> 
> 
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org (mailto:Catalog-SIG at python.org)
> http://mail.python.org/mailman/listinfo/catalog-sig
> 
> 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120417/2acf320e/attachment-0001.html>

From tarek at ziade.org  Tue Apr 17 23:59:55 2012
From: tarek at ziade.org (=?UTF-8?B?VGFyZWsgWmlhZMOp?=)
Date: Tue, 17 Apr 2012 23:59:55 +0200
Subject: [Catalog-sig] PyPI mirrors are all up to date
In-Reply-To: <6375BB58D5494D61BB0F2599C75B9F93@gmail.com>
References: <CACBKDng6radNLst8H=bnfhg2bGD3SThq6DSbW2MUZbZh2RUNjg@mail.gmail.com>
	<0BC4A4DF-49BF-47BC-B598-354583951D3B@gmail.com>
	<CAJ5sox6N=HeqFQWKB6EJaWo7svfpjwiG48i3yVRAy3zEnU6sTg@mail.gmail.com>
	<CACBKDng5xuvi85RJVphcmxBpmr+UN1c2feGVDfcircHhSyhn0Q@mail.gmail.com>
	<CAAGHeXHYAzzw6r9MQ1KnF8=z7UTQeNS5omFncNA_5cy6_fUYZw@mail.gmail.com>
	<CACBKDnhDqE+KVdJ6AHQhgMi9nX1fKeHwrb_y_PUxxaUXQurO8Q@mail.gmail.com>
	<529FB8BD64A54ED6854C34A29A15821F@gmail.com>
	<CACBKDngNiFQy_bxs79Q+9Zx=SBCYaLbafG60q2zAYM0PjymyVA@mail.gmail.com>
	<4F8C85A4.2010905@ziade.org>
	<CACBKDngPuvii6ytC3eY=0KwNxwsgXEenNju0qFsXkFzscTCFpA@mail.gmail.com>
	<4F8C89B1.20000@ziade.org> <4F8C8DBB.5020409@v.loewis.de>
	<4F8C92F7.1020404@ziade.org> <4F8C95DF.4020703@v.loewis.de>
	<4F8C98A0.5030909@ziade.org> <4F8C9ADB.4040603@v.loewis.de>
	<4F8CA1A5.2050909@ziade.org>
	<20120417115711.Horde.azZHL6GZi1VPjT53LEyEPsA@webmail.df.eu>
	<4F8D4B13.90502@ziade.org>
	<6375BB58D5494D61BB0F2599C75B9F93@gmail.com>
Message-ID: <4F8DE7DB.4070809@ziade.org>

On 4/17/12 12:54 PM, Donald Stufft wrote:
>
>>
>> If there's interest I can write a multiprocess-based script that keeps a
>> md5 database up-to-date
> I'd be interested ;) Although i'd prefer sha256 personally.
>>
well, this is not really for security and I don't think a collision can 
happen that often with md5 :D

here's a raw script: http://tarek.pastebin.mozilla.org/1575563

The grand digest is done like a derived secret: I loop on the hash and do

    grand hash = hash(n & n+1) for n in hashes

I've run it against the 111,196 files I currently have in my mirror

- First *full* run from scratch - 15m32s  (not sure why I don't have 
better here, maybe Python's md5 is slower than md5deep)

- Second *full* run, md5 database filled - 2m33s - it scans the mirror, 
and adds missing md5s + build the grand digest.

- Just the digest, against a synced MD5 DB - 1m1s  (I just commented the 
first part that builds/updates the md5 db)

In a real mirror, once the first full run is done, the md5 db would be 
updated continuously everytime a new file is added
in the mirror, so the only extra load is recalculating the digest again.

So it would take around a minute each time, not a few seconds as I said 
previously. But that seems ok if a mirror is updated
for example every 5 minutes, 4 minutes can be spent to sync the files, 
and 1 minute to do the checksum I guess




>> Cheers
>> Tarek
>>
>>>
>>> Regards,
>>> Martin
>>
>> _______________________________________________
>> Catalog-SIG mailing list
>> Catalog-SIG at python.org <mailto:Catalog-SIG at python.org>
>> http://mail.python.org/mailman/listinfo/catalog-sig
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20120417/ff253592/attachment.html>

From btimby at gmail.com  Thu Apr 26 16:59:36 2012
From: btimby at gmail.com (Ben Timby)
Date: Thu, 26 Apr 2012 10:59:36 -0400
Subject: [Catalog-sig] Taking Over Maintainership.
Message-ID: <CAB9wFTwifzrM1BS-UHQC-VOJqmKO+Hvy3x+6VJZrChaf6UuvCw@mail.gmail.com>

I recently took over maintainership of the Python queues project at
google code hosting.

http://pypi.python.org/pypi/queues/0.6.2

I would like to be able to update the package in PyPI as well, how can
this be accomplished? I am unable to contact the original author.

Thanks.

From martin at v.loewis.de  Thu Apr 26 20:50:21 2012
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 26 Apr 2012 20:50:21 +0200
Subject: [Catalog-sig] Taking Over Maintainership.
In-Reply-To: <CAB9wFTwifzrM1BS-UHQC-VOJqmKO+Hvy3x+6VJZrChaf6UuvCw@mail.gmail.com>
References: <CAB9wFTwifzrM1BS-UHQC-VOJqmKO+Hvy3x+6VJZrChaf6UuvCw@mail.gmail.com>
Message-ID: <4F9998ED.5070609@v.loewis.de>

On 26.04.2012 16:59, Ben Timby wrote:
> I recently took over maintainership of the Python queues project at
> google code hosting.
>
> http://pypi.python.org/pypi/queues/0.6.2
>
> I would like to be able to update the package in PyPI as well, how can
> this be accomplished? I am unable to contact the original author.

Submit a support request to the PyPI bug tracker (at SF). We will try
to contact the original author ourselves, and if that fails, ask for
some proof that Google indeed given you maintainership to the code.

Regards,
Martin


From djfische at gmail.com  Mon Apr 30 17:52:22 2012
From: djfische at gmail.com (David Fischer)
Date: Mon, 30 Apr 2012 08:52:22 -0700
Subject: [Catalog-sig] PGP key for pip package
Message-ID: <CALKRDOmrnjiTVyXJ=sHLqUW3iQxMUXN3Z8r_wT=4B=SrpviX5g@mail.gmail.com>

Hi,

I've been playing around with PGP signing as it relates to Python
packages. There are relatively few signed packages in Cheeseshop but
one of them is the pip package[1]. Does anyone know where I could get
the public key used to verify this package (key ID 0171DF30)? Perhaps
I am missing something obvious.

-David


[1]  http://pypi.python.org/pypi/pip/1.1