[Catalog-sig] PyPI replication project

[Tarek Ziade]

2008/10/10 Andreas Jung <lists at zopyx.com>

> Am 09.10.2008 18:59 Uhr, Martin v. Löwis schrieb:
>
>> Mirrors help every other packaging system.  So it stands to reason
>>> that it would help pypi too.  I think since many zope people have been
>>> using mirrors instead of using pypi directly... pypi has been more
>>> available.  It's running lots better for other reasons too... but less
>>> load is probably also nice for pypi :)
>>>
>>
>> I'm fine with people operating their own mirrors. I just don't think
>> it can be made *invisible* to users that they use a mirror. In the
>> mirroring systems for Linux distributions, for example, people have
>> to explicitly select which mirror they want to use (and accept that
>> the mirror may lag behind by a day or so). It's also clear that it is
>> a "mere" mirror.
>>
>
> Implict or explict mirror selection is not primary point in phase 1
> of the project. The point is that we must have access to the distribution
> packages and eggs at any time - independent of the available of PyPI (either
> related to issues with the PyPI server or caused by internet outages or
> routing problems).
>
> An implicit selection of a mirror in case of an detected outtage would be
> nice but this is possibly not the most important issue right now.
> We can always reconfigure out buildout configurations easily to a new
> server or define a series of mirroring servers.
>
>
>> What Andreas was asking how a distributed PyPI installation could work,
>> by which I assume he was asking for one that a) is invisible (of called
>> misleadingly "transparent") to users, and b) allows updates to replica.
>>
>> I'm skeptical that such an system would work all that well, and can
>> be created in a reasonable amount of time
>>
>
> As stated earlier: we can already define multiple servers as part
> of a buildout configuration. A better mirror selection algorithm would be
> nice to have for the future but right now we don't actually need it and can
> live with the current state.


I think the key is to choose upon several indexes, not to use extra
find-links.

(I have submitted a patch for setuptools to handle several indexes for that:
I think the key
is to choose upon several indexes, not to used)

Last, I think we should create a mirror registration system on PyPI, to ping
mirrors when
a new package is uploaded, so the sync is simpler.



>
> Andreas
>
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
>
>


-- 
Tarek Ziadé - Directeur Technique
INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632
Bureaux de la Colline - 1 rue Royale - Bâtiment D - 9ème étage
92210 Saint Cloud - France
Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04
http://www.ingeniweb.com - une société du groupe Alter Way
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20081010/3accadfb/attachment.htm>