security quirk

Arne Vajhøj arne at vajhoej.dk
Wed Jan 30 20:40:23 EST 2013


On 1/29/2013 11:55 PM, RichD wrote:
> I read Wall Street Journal, and occasionally check
> articles on their Web site.  It's mostly free, with some items
> available to subscribers only.  It seems random, which ones
> they block, about 20%.
>
> Anywho, sometimes I use their search utility, the usual author
> or title search, and it blocks, then I look it up on Google, and
> link from there, and it loads!  ok, Web gurus, what's going on?

WSJ want their articles to be findable from Google.

So they open up for Google indexing them.

If they require any type of registration to see an article,
then Google will remove the link.

So therefore WSJ (and many other web sites!) gives more access
if you come from Google than if not.

Arne





More information about the Python-list mailing list