[python-advocacy] Python makes the "most wanted list"

[Michael Pittaro]

Roy Smith wrote:
> They say any publicity is good publicity, but this may not be.   
> Python seems to be getting called out by the W3C as a leading abuser  
> of their HTTP server.
>
> http://www.w3.org/blog/systeam/2008/02/08/w3c_s_excessive_dtd_traffic
>
>
>   
Wow!  Thats a lot of traffic.

It's hard to blame Python for this, since theres nothing in Python 
itself that pounds on the w3 servers.  My interpretation is that urllib2 
is being used in a lot of programs, many of which just use urlopen().

We could probably update the urllib documentation to strongly encourage 
setting the User-agent header. The examples already show how to do it, 
but if urllib2.urlopen()  is the popular function, adding useragent as 
an argument to might be worth considering.

My (optimistic) guess is that this problem might follow the 80/20 rule - 
there are a small number  of popular programs or libraries using urllib2 
that are the major offenders.

I'm not sure how to track them down, other than being aware of the 
problem and paying more attention.

mike
 

-- 
Mike Pittaro
Co-Founder                      Snaplogic, Inc.
mikeyp at snaplogic.org            http://www.snaplogic.org