Yahoo indexing old URL's

[Jennifer Dallas]

Hello everyone,

I'm working for a ski tour operator and the site is well optimised with our keywords. All the meta tags and descriptions are in place and in general SEO has been going along quite nicely.

In Google our site has 274 pages indexed, however in Yahoo we have over 800 indexed. This is largely due to a form that has not been added with a noindex tag and is therefore repeated over 100 times but I have also noticed something else which I'd like any ideas on.

Recently we redirected a number of pages from old URL's to new better optimised URL's. The problem is that Yahoo is indexing the old URL and the new URL. I have spoken with my entire eCommerce team and no-one has seen this before. The only information I've found is that if a 302 redirect is used sometimes the search engines continue to index the old URL but I am assurred by my web developers they have used 301 redirects. The web developers also tell me this is just caused by a stale cache but we are not convinced.

If anyone has any suggestions what could be causing this problem I'd be really greatful. Obviously at the moment it is creating an issue with duplicate content which is affecting my page rank.

Many thanks
Jennifer

[temi]

Yahoo has the silly habit of having very old urls in its database. If the urls in question do not exist any more, best thing is to set up a 404 page that forward requests to such pages to the most appropriate page.

[Jennifer Dallas]

Will that actually stop Yahoo from indexing them quicker?

[Robdale]

have you submitted your updated sitemap to yahoo? If not then i suggest you to submit it.

[Jennifer Dallas]

yes I've done that

[BlueBoden]

The web developers also tell me this is just caused by a stale cache but we are not convinced.

Many thanks
Jennifer

I would check with your developers to make sure that they specified the new location. I.e.

Simply throwing out a 301 response is not enough, because you need to inform where the page was moved as well, this is done like below.

    header("HTTP/1.1 301 Moved Permanently");
    header('Location: ' . $Location['New']);

This would ensure that you wouldn't lose visitors from those outdated URLs. You can then remove the redirect, when the traffic on those URLs had decreased.

The way to check which URL was requested, is to use something like below.

$requested_path = $_SERVER['REQUEST_URI'];

This can be used in an if statement, to throw out the correct response code, and relevant redirect. I.e.

  if (preg_match("/^\/([A-Za-z]+)\/([0-9]+)\/$/D", $requested_path, $match)) {
    header("HTTP/1.1 301 Moved Permanently");
    header('Location: ' . $Domain['Name'] . $match[1] . '/' . $UrlConstruct . '_' . $match[2] . '.html');
}

The above is an example from one of my own websites, just to show that you can use regular expressions, to match old URL patterns. The pattern would match urls like "/Tutorials/xx/", and redirect it to the optimized version, where "xx" is the unique PostID. You can include else if statements to match other patterns.