[Python-Dev] code.python.org - random 403 errors

[Sridhar Ratnakumar]

On 2010-04-22, at 3:01 PM, Martin v. Löwis wrote:

>> In general python.org seems flaky today. Sometimes I could not reach the site
>> at all, now a ping to svn.python.org gives 30% packet loss.
> 
> Yes. See
> 
> http://www.python.org/munin/localdomain/localhost.localdomain-if_eth0.html
> 
> Somebody was consuming all bandwidth, although we couldn't quite figure
> out who (whenever I was looking, the traffic looked genuine). It turned
> out that Sridhar himself contributed a good chunk of this traffic.

Background in private emails to Martin (below):


On 2010-04-22, at 5:42 PM, Sridhar Ratnakumar wrote:

> 
> On 2010-04-22, at 2:13 PM, Martin v. Löwis wrote:
> 
>>> [snip ... ip address]
>>> 
>>> I must note that we do a nightly build of ActivePython using Python
>>> mercurial repositories for trunk, release26-maint and release31-maint
>>> (3 URLs) on 5 of our build machines. So one nightly build will make
>>> 3x5=15 "hg clone" requests per day to code.python.org. This week, I
>>> have been debugging the nightly build code (run by hudson); so that
>>> number is likely higher this week.
>> 
>> [snip ... possibly security-confidential part]
>> 
>> Can you please find a way not to clone the complete repository every
>> time? IIUC, it should be possible to update a clone, rather than
>> refetching it from scratch.
> 
> Ok, I setup a cron job to maintain an internal mirror of the above mentioned repositories in code.python.org. We'll do a "hg pull -u" (equivalent to "svn up") every hour; no clones. Hopefully, that should reduce the amount of requests from our side. Let me know if in future this issue repeats.