[Moin-user] Searchbots create hundreds of empty folders

[Gnarlodious]

Stranger and stranger. It turns out moinmoin page is creating all
those empty folders, not a searchbot at all! I can load any page
(apparently), but when I load my home page it does this. I can delete
all the empties with a shell script. Then when I reload the home page
it waits about 10 seconds, presumable while hundreds of empty folders
are being created. Then it loads the page and renders normally.

What in the world is causing this? Page caching? A wayward Macro?

-- Gnarlie


On 3/23/07, Gnarlodious <lists.gnarlodious at gmail.com> wrote:
> I have been having a heck of a time with searchbots creating hundreds
> of empty folders in my "Pages" folder. These are named pages of
> obsolete URLs that were indexed months ago and deleted or moved.
>
> I reduced the problem by blocking many searchbots with rewrite like this:
>
> RewriteCond %{REMOTE_HOST} 60.191.80.29 [OR]
> RewriteCond %{REMOTE_HOST} 74.6.73.98 [OR]
> RewriteCond %{REMOTE_HOST} 74.6.86.151 [OR]
> RewriteCond %{REMOTE_HOST} 74.6.87.123 [OR]
> RewriteCond %{REMOTE_HOST} 65.55.208\..* [OR]
> RewriteCond %{REMOTE_HOST} 65.214.44.173 [OR]
> RewriteCond %{REMOTE_HOST} 66.231.189.78 [OR]
> RewriteCond %{REMOTE_HOST} 66.249.65.81 [OR]
> RewriteCond %{REMOTE_HOST} 71.168.107.138 [OR]
> RewriteCond %{REMOTE_HOST} 72.30.177.87 [OR]
> RewriteCond %{REMOTE_HOST} 72.30.177.93 [OR]
> RewriteCond %{REMOTE_HOST} 72.30.177.98 [OR]
> RewriteCond %{REMOTE_HOST} 74.6.67.212 [OR]
> RewriteCond %{REMOTE_HOST} 74.6.72.56 [OR]
> RewriteCond %{REMOTE_HOST} 74.6.70.58 [OR]
> RewriteCond %{REMOTE_HOST} 74.6.73.98 [OR]
> RewriteCond %{REMOTE_HOST} 74.6.86.151 [OR]
> RewriteCond %{REMOTE_HOST} 74.6.87.123 [OR]
> RewriteCond %{REMOTE_HOST} 81.52.143.15 [OR]
> RewriteCond %{REMOTE_HOST} 81.52.143.16 [OR]
> RewriteCond %{REMOTE_HOST} 82.165.167.199 [OR]
> RewriteCond %{REMOTE_HOST} 122.152.128.48 [OR]
> RewriteCond %{REMOTE_HOST} 128.194.135.94 [OR]
> RewriteCond %{REMOTE_HOST} 139.18.13.204 [OR]
> RewriteCond %{REMOTE_HOST} 193.47.80.49 [OR]
> RewriteCond %{REMOTE_HOST} 194.50.163.175 [OR]
> RewriteCond %{REMOTE_HOST} 202.160.179.131 [OR]
> RewriteCond %{REMOTE_HOST} 202.160.179.168 [OR]
> RewriteCond %{REMOTE_HOST} 202.160.180.184 [OR]
> RewriteCond %{REMOTE_HOST} 209.22.78.65 [OR]
> RewriteCond %{REMOTE_HOST} 220.181.19.173 [OR]
> RewriteCond %{REMOTE_HOST} 220.181.19.189 [OR]
> RewriteCond %{REMOTE_HOST} 220.181.34.182
> RewriteRule .* - [F,L]
>
>
> Unfortunately this does not solve the problem, because while Apache
> does not serve up a page the empty folders are still created. I don't
> understand why, because Apache should intercept the URL before the
> script runs.
>
> This is Moin Version 1.5.5a, would a newer version put an end to the
> empty folders, or is there a configuration way to stop it?
>
> -- Gnarlie
> http://Gnarlodious.com/
>