"The Python Robot" is going mad ?

[Skip Montanaro]

    Robert> I have python related stuff on some of my web pages.  This month
    Robert> "The Python Robot" is going for over 400 MB / 4000 accesses
    Robert> downloads in my stats ! increasing frequency the last months!?
    Robert> thats 25x more than Google Bot & Inktomi Slurp - the second
    Robert> loquacious.

    Robert> These numbers a going to cause substantial server cost. Why does
    Robert> "The Python Robot" need to scan my pages 20 times a day? How can
    Robert> I cool down this special bot a little bit ( robots.txt ? )

What's "The Python Robot"?  Have you contacted the author/maintainer?  Have
you tried blocking the IP address it's coming from?  If it's ill-behaved
there's a good chance it will ignore robots.txt.

Skip