2

We redirected all the old URL's to new ones properly using .htaccess. The problem is Google, somehow is still finding content in the old page (which it shouldn't) and stores it in the cache rather than the new URL.

For e.g.:

  • Old page: http://www.natures-energies.com/iching.htm
  • New page: http://www.natures-energies.com/index.php?option=com_content&view=article&id=760

If you type the old URL into the browser it redirects. If you fetch the old URL as Googlebot in the Webmaster Tools the header says 301/permanently redirected. If I try to crawl as any other bot it still says 301 redirected. Even if you click the old link in Google it redirects to the new URL.

Only in its cache it shows the old URL and moreover it shows the new content in it!

I am stumped on how Google manages to grab the new content and puts in the old URL instead of the new one!

One more interesting thing is that if I try a cache for the new page it shows the cache of the new content with old URL!

Any help would be appreciated. I am at end of my wits. I think I have tried almost everything. Is there anything that I'm missing to see?

You can use this search to find the old URL's. Maybe you'll some patterns that I missed. site:www.natures-energies.com inurl:htm -inurl:https|index.

Improve this question
3
  • The situation is something similar to this webmasters.stackexchange.com/questions/24565/… We use mod_rewrite and this is the common format of redirecting in htaccess RewriteRule iching.htm natures-energies.com/… [R=301,L]
    – user2178914
    Nov 10, 2013 at 6:22
  • I'd be more concerned about the length of time it is taking to load your pages (I'm sorry, I gave up waiting for them whilst looking into this issue for you).
    – zigojacko
    Nov 11, 2013 at 15:53
  • Ya the latency is really high..around 29 sec!!! we are in the midst of moving to a dedicated server. Hopefully that should solve the high latency
    – user2178914
    Nov 12, 2013 at 4:48

2 Answers 2

Reset to default
1

This most certainly comes from the way they save the data in their index. Google makes use of a database that is built on what they call Big Tables. The Cassandra database is an open source replicate of that database system if you want to learn more about it.

It is clear to me that the main key in their indexes is the URI of the websites (written in reverse to simplify the sort order). Google does not take a 301 literally. The fact is that when you create a 301, you may change your mind a few times (i.e. you really meant 302, or you re-rename the page and the URI changes again.) I think that one reason they update their index keys (the URI) at a slower rate than the content of the pages because it is a lot more problematic than to change the content. That URI being a key, it must appear billions of times in the database and changing has a huge impact.

Just in case I checked your Drupal site and you do not have a canonical URI defined. I suppose such could have an effect too. But I think it will be updated later. I do not know how long it takes though. Why did you decide to remove the URL aliases?

Improve this answer
6
  • Thanks Alexis, That explains a lot. Is there a way to speed up the process or something? Do you mean the use of rel=canonical tag? Yes we haven't used it anywhere since there aren't any duplicates of the same page. By URL aliases do you mean the switch from iching.htm to new page? We did it because the previous site was based on plain html and now we have moved to CMS, Joomla. We moved the site on Aug 29th and still the old url's there(an unusually long tome for google to catch up). In fact bing has only 40 old url's left in their index. Google also is dropping old url's but only 10/day.
    – user2178914
    Nov 10, 2013 at 9:08
  • There isn't any problem from our side in the redirection, right? I suppose we did everything right and it is upto google to catch up. here's bing index bing.com/…
    – user2178914
    Nov 10, 2013 at 9:36
  • Indeed your redirect is correct. I will just take time for Google to do the index (as in URI) update.
    – Alexis Wilke
    Nov 11, 2013 at 23:59
  • It looks like a Drupal install, not Joomla. Drupal has an option called "clean URL". That gives you a way to have regular paths instead of the index.php?q=path syntax which I think makes it cleaner.
    – Alexis Wilke
    Nov 12, 2013 at 0:03
  • We did this migration on August 29th. Still no luck. That's why a bit worried. Also it is Joomla. I regularly login to the admin panel of Joomla :) Any idea on how we can speed up this process?
    – user2178914
    Nov 12, 2013 at 4:44
1

Part of the problem may be that your new URL is much more complex than the old URL. Your new URL is served on a dynamic page with three URL parameters. The old URL appears to be just a static page.

Why do you have three URL parameters? This version of the new URL seems to work fine: http://www.natures-energies.com/index.php?id=760

Google might be more willing to believe the 301 redirect if the URL were simplified a bit.

Improve this answer
2
  • That is the default url that Joomla sets. We let this be because we had already redirected the old ones to this format. I thought if we further setup a new redirect which redirects to a more search engine friendly url google might have difficulty in following the redirects. It has to first look up for old url then get redirect to the new non SEF url and then to the SEF url. Since it was already having difficulty with crawling the new url's we decided let google sort this out first then we will move to SEF url's later. Any suggestion to speed up the process?
    – user2178914
    Nov 10, 2013 at 11:13
  • Most of all is everything ok? Or have we configured anything wrong? I checked every little thing that can be a potential issue. Everything seems to be alright. One Google Top Contributor told me the old url's are still returning a 200 and asked me to check the redirect implementation! Everything seems to be working as it should!
    – user2178914
    Nov 10, 2013 at 11:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.