[pydotorg-www] Steering search engines and users away from 3.0/3.1 docs?

[tritium-list at sdamon.com]

IIUC robots.txt will allow you to ‘deny /; allow /index.html’

 

From: pydotorg-www [mailto:pydotorg-www-bounces+tritium-list=sdamon.com at python.org] On Behalf Of Skip Montanaro
Sent: Tuesday, November 1, 2016 8:40 AM
To: pydotorg-www at python.org
Subject: Re: [pydotorg-www] Steering search engines and users away from 3.0/3.1 docs?

 

Tweaking robots.txt seems like the simplest route. It would be nice if /3.[01]/index.html where visible through search engines but not anything underneath. I don't recall if robots.txt is a sharp enough tool to make that distinction.

 

Following my original thought a bit further, I wondered how far down in the search results the mainline /3/ docs would be, so I modified my search to "python timeit site:docs.python.org <http://docs.python.org> ". The answer for me, at least, seems to be, "almost not on the front page." In fact, the 2.5 timeit reference is right after the 3.0 reference. 3.1 is a bit further down, then the v3 version of the quick stdlib tour is the last hit on page one.

 

If someone's going to mess with robots.txt, maybe cast the same spell for 2.[0-5] (and 6?) as you do for 3.0 and 3.1.

 

Skip

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pydotorg-www/attachments/20161101/1b85680d/attachment.html>