[pydotorg-www] [Infrastructure] State of the infrastructure project

[Benjamin Smith]

Adding my $.02 on a couple of points.

On Wed, Sep 14, 2011 at 11:53 AM, "Martin v. Löwis" <martin at v.loewis.de>wrote:

> 1. Prospect of moving PyPI to Postgres 9 for better replication in a
>> (closer to) HA scenario. I will post this to the catalog-sig group
>> for further discussion after I get done reading the existing Postgres
>> config and re-familiarizing myself with the PyPI codebase.
>>
>
> Not sure how software package management and updates will be done.
> I personally prefer (and it was policy so far) to use Debian packages,
> in order to get Debian upgrades. That would rule out using Postgres 9
> at this point.
>
>
What about:

http://www.openscg.org/se/postgresql/packages.jsp

They're certified, proper binary packages.

I wouldn't want to rely on debian for this. That's like relying on CentOS
for python > 2.4 :(.

I understand policy, but sometimes you have to make exceptions for the sake
of moving forward and stability. This key piece of the infrastructure would
be a prime candidate for an exception.


> Also, I'm skeptical that Posgres replication could actually improve
> availability. When PyPI becomes unavailable, it is typically not
> because of Postgres, and I would expect that the added complexity
> of supporting failover would actually reduce availability, and
> introduce the risk of data loss.


To the contrary, I think that replication could potentially increase
availability. There may not have been a problem with postgres yet, but it is
inevitable. Being prepared would ensure that we're not as
surprised/shocked/frustrated. The complexity is minimal with postgres 9.
It's a few config entries and an rsync, I've done it recently. Management
time is negligible in my experience thus far. See
http://wiki.postgresql.org/wiki/Binary_Replication_Tutorial#Binary_Replication_in_6_Steps

Would you rather restore from a backup (timely, painful), or run a script to
fail over (fast, easy)?

As far as data loss is concerned, Postgres 9.1 addresses that
with synchronous replication
http://wiki.postgresql.org/wiki/What's_new_in_PostgreSQL_9.1#Synchronous_replication_and_other_replication_features


>
>  2. What remote sites are pulling backups of the PyPI package data
>> from the Rsync server on ximinez?  I think at least tummy.com is from
>> discussions with Jannis, but a full list would be great.
>>
>
> I think that, for the rsync backup, it's only tummy.com. Of course,
> there are tons of PyPI mirrors, but they are not using rsync, and
> not doing backup.
>
>
>  3. Does anyone know the reason for the Webalizer config being under
>> /usr/local/etc and using a custom cron job instead of the
>> debian-standard locations?
>>
>
> Thomas set this up. I believe the motivation is that only this
> setup will support multiple vhosts. It actually supports auto-
> detection of vhosts, where new log files will trigger the creation
> of new webalyzer configs.
>
>
>  4. Is Postfix on ximinez doing anything other than outbound
>> delivery?
>>
>
> No.
>
>
>  5. Is lpd just a legacy artifact or is there actually a printer (or
>> something that looks like a printer) attached?
>>
>
> I think it got installed as a side effect of installing something
> else that had printing as a Debian dependency.
>
>
>  If anyone has any questions about the current state of the
>> infrastructure project or wants to make sure something specific gets
>> on the docket to be looked at, please don't hesitate to contact me.
>>
>
> I'd still urge that IPv6 connectivity is made a pre-requisite for
> switching providers.
>
> Regards,
> Martin
>
> ______________________________**__________________
> Infrastructure mailing list
> Infrastructure at python.org
> http://mail.python.org/**mailman/listinfo/**infrastructure<http://mail.python.org/mailman/listinfo/infrastructure>
> Unsubscribe: http://mail.python.org/**mailman/options/**infrastructure/**
> benjaminwarfield%40gmail.com<http://mail.python.org/mailman/options/infrastructure/benjaminwarfield%40gmail.com>
>



-- 
Benjamin Smith
http://just-another.net
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pydotorg-www/attachments/20110914/66955340/attachment-0001.html>