[Python-Dev] What google will let you do

[Skip Montanaro]

    >> Correct.  I think they want to avoid automated systems that tweak a
    >> webpage, see how placement is affected, tweak it again, etc.

    Guido> That would take forever (you'd have to wait for a second visit
    Guido> from their crawler).

Not necessarily, and of course, it depends on your definition of "forever".
I suspect they probably have their crawler coded to visit files more often
that change frequently (like freshmeat.net or slashdot.org).  It's not
practical to input change frequency info by humans, so the crawler probably
realizes that a page changed since the last time it was crawled and decides
on its own to shorten up the interval before the next check.  (They would
probably have it lengthen the interval if the file didn't change as well.)
They can thus adjust their visit frequency to keep track of pages that
change rapidly, but not waste time crawling static pages.  The click
frequency may also figure into this.  A page that is visited more often
probably warrants closer attention by the googlebot.

I suspect a person wanting to manipulate his site's google ranking could
twiddle his page and get feedback on the results within a few days.  Over
the course of a month or two, he could probably make substantial progress on
raising his Google quotient.

    Guido> I think they want to avoid added-value websites that don't
    Guido> mention the Google brand -- it would be like redistributing
    Guido> Python but claiming you wrote it yourself.

That too.  They explicitly deny meta-search access and even say "don't even
ask - your request will be denied", or words to that effect.

So, we've concluded nothing, but have given our frontal cortex its
speculative workout for the day... ;-)

Skip