[Python-Dev] Webstats for www.python.org et al.

[Jack Diederich]

On Wed, Mar 01, 2006 Brett Cannon wrote:
> On 2/28/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> > Thomas Wouters wrote:
> >
> > > I added webstats for all subsites of python.org:
> > >
> > > http://www.python.org/webstats/
> >
> > what's that "Java/1.4.2_03" user agent doing?  (it's responsible for
> > 10% of all hits in january/february, and 20% of the hits today...)
> 
> Most likely a crawler.
> 
Youch, if I'm reading it right it consumed fully half of the bandwidth
for today on python.org.  And what 1.6 million pages did it spider on
the site last month?  Something smells broken.

-Jack