[pydotorg-www] robots.txt make web archive inaccessible

M.-A. Lemburg mal at egenix.com
Wed Jun 25 10:13:50 CEST 2014


On 25.06.2014 06:16, anatoly techtonik wrote:
> http://web.archive.org/web/*/http://docs.python.org/release/2.2.3/whatsnew/node5.html

Can you propose an alternative robots.txt file ?

"""
# Prevent development and old documentation from showing up in search results.
User-agent: *
# Disallow: /dev
Disallow: /release
"""

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 25 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-06-17: Released eGenix PyRun 2.0.0 ...       http://egenix.com/go58
2014-07-02: Python Meeting Duesseldorf ...                  7 days to go
2014-07-21: EuroPython 2014, Berlin, Germany ...           26 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


More information about the pydotorg-www mailing list