"The Python Robot" is going mad ?

[Robert]

at http://www.robotstxt.org/wc/active/html/python.html I learned the robot
"The Python Robot" is maintained by www.python.org.

Thus I don't want to completely block it, if it is a regular bot. But the
volume increased over last months: 25x more than googlebot. that mad me
stunn.

Here is the top of my robot stats list for this month (22. of this month):

The Python Robot 2948 241.97 MB 22.03.2004 - 23:43
Googlebot (Google) 491+101 11.06 MB 22.03.2004 - 14:37
Scooter (AltaVista) 228+228 4.35 MB 22.03.2004 - 21:59
WISENutbot (Looksmart) 426+8 4.42 MB 22.03.2004 - 21:51


Or is "The Python Robot"  a fake robot and i'd have to check the IP? Cannot
imagine.

the page above says:

Name  The Python Robot
Cover Page  http://www.python.org/
Details Page
Operational Status  retired

"retired" at 241MB in 22 days ?  I know some pensioners become very busy at
60+ ...

Robert


"Skip Montanaro" <skip at pobox.com> schrieb im Newsbeitrag
news:mailman.291.1080059607.742.python-list at python.org...
>
>     Robert> I have python related stuff on some of my web pages.  This
month
>     Robert> "The Python Robot" is going for over 400 MB / 4000 accesses
>     Robert> downloads in my stats ! increasing frequency the last months!?
>     Robert> thats 25x more than Google Bot & Inktomi Slurp - the second
>     Robert> loquacious.
>
>     Robert> These numbers a going to cause substantial server cost. Why
does
>     Robert> "The Python Robot" need to scan my pages 20 times a day? How
can
>     Robert> I cool down this special bot a little bit ( robots.txt ? )
>
> What's "The Python Robot"?  Have you contacted the author/maintainer?
Have
> you tried blocking the IP address it's coming from?  If it's ill-behaved
> there's a good chance it will ignore robots.txt.
>
> Skip
>
>