Is Google's ranking algorithm somehow more sacred than my HTML? Google mines that to create their page rankings. Sure there's robots.txt but likewise Google could opt out of serving information to Internet Explorer since that's the only browser which has Bing toolbar. The entire search engine industry is based on using whatever someone has failed to opt out of.