Protect yourself against Google Proxy hack

[Bagi Zoltán]

Originally Posted by temi

I'm I correct?
Thanks
[/color] [/color]

Absolutly correct. I was the one who additionally inserted the Yahoo slurp validation.

Please test your site after installing.

[gkd_uk]

Thanks Bagi

I will copy the code and name the file reversedns.php and upload

and add <?php include("reversedns.php"); ?>

Thanks

[gkd_uk]

Originally Posted by Bagi Zoltán

Thanks Imran. At my seo dir the yahoo slurp validation isn't in the code since the script has some error with that. I used this reversedns.php file

PHP Code:

<?php
// Get the user agent.
$ua = $_SERVER['HTTP_USER_AGENT'];
// Check the user agent to see if it's identifying itself as a search engine bot.
if(stristr($ua, 'msnbot') || stristr($ua, 'googlebot')){
// The user agent is purporting to be MSN's bot or Google's bot.
// If the user agent string is spoofed, we won't find googlebot.com in the host name.
// Get the IP address requesting the page.
$ip = $_SERVER['REMOTE_ADDR'];
// Reverse DNS lookup the IP address to get a hostname.
$hostname = gethostbyaddr($ip);
// Check for '.googlebot.com' and '/search.live.com' in hostname.
if(!preg_match("/\.googlebot\.com$/", $hostname) &&!preg_match("/search\.live\.com$/", $hostname)) {
// The host name does not belong to either live.com or googlebot.com.
// Remember the UA already said it is either MSNBot or Googlebot.
$block = TRUE;
header("HTTP/1.0 403 Forbidden");
exit;
} else {
// Now we have a hit that half-passes the check. One last go:
// Forward DNS lookup the hostname to get an IP address.
$real_ip = gethostbyname($hostname);
if($ip!= $real_ip){
$block = TRUE;
header("HTTP/1.0 403 Forbidden");
exit;
} else {
// Real bot.
$block = FALSE;
}
}
}
?>

which validates only the googlebot and the msnbot.

Imran, don't forget that not i'm the person who developed this solution, i only share this

If the Yahoo Slurp did not work for you, does this mean it is not guaranteed to work for others?

[Bagi Zoltán]

No, Imran that means that it won't validate Yahoo!'s spider, but validates the msnbot and the googlebot.

[temi]

If it does not validate Yahoo spider, Yahoo can still craw the site, it just will not be able to cache it?

[gkd_uk]

Thanks Bagi

[Bagi Zoltán]

Originally Posted by temi

If it does not validate Yahoo spider, Yahoo can still craw the site, it just will not be able to cache it?

It means that when the Yahoo! Slurp comes through a proxy link it will see it as content duplication. If you use the code specified at the first post that will validate Yahoo spider as well, but my older indexscript site doesn't like it for some reason, so i use there the "non-yahoo-validating" version. I tested today the first code, which validate the three major spider at my other site, and seemed to work well.

[temi]

thanks for that clarification Bagi

[gkd_uk]

I set this up yesterday with help from Bagi and it seems to work fine.

I have not yet setup Yahoo Slurp but will be doing this tonight

[Bagi Zoltán]

The WordPress plugin is on the way. If everything will be fine tomorrow it will be released