I think people are probably not that excited because there is nothing new or exciting robot exclusion will do that robots.txt does not do at the moment or is there?
I also know that some rouge robots do not follow robots.txt, will the robots exclusion protocol address that?
Thanks and good luck with you article, when its own, let me know and I will be happy to draw some attention to it via this forum and my blogs.
Quote:
Originally Posted by searchtools
I'm writing an article on the changes in the Robots Exclusion Protocol for infotoday.com, and would very much like to know what SEO people think about them. Is it going to make your lives a lot easier, to have a protocol to write to for all three big crawlers, and I'd guess most of the small ones too? Are there any downsides you can see? Is it just ho-hum business as usual? I can't find a lot of discussion, so am asking here. Please note whether you would or would not like to be quoted.
|