[Mailman-Users] How to block spiders?

Paul F. Williams paul.williams at uwex.edu
Wed Jan 21 17:25:04 CET 2004


We are running mailman-2.1.1-4 on Redhat 9.

How can I block spiders from indexing
the public archives on our site?

Normally with an apache web server you can
just place a robots.txt file in the root of
the documents directory.  Not sure how one might be
able to do this with mailman.

When you take a look at the archives for a list, one sees

   http://server-name/pipermail/list-name/

Since there isn't a pipermail directory, one can't just place
a robots.txt file in that directory.  Perhaps one could put
a robots.txt file in each list directory, but that would take
a lot of work with many lists and you would need to update
it all the time as new lists get added.

Suggestions, clarifications, or explanations are more than welcome.

Thanks,

paulw





More information about the Mailman-Users mailing list