We're running a Magento shop and we experienced a significant raise in 404 hits lately, almost 40,000 in one day. Most of the pages are old ones that once existed but don't anymore, and there are some really strange occurrences where as much as 200 pages are linked from one to another, for example


None of them really exist, all return 404 error, but somehow Google sees them and follows the links in them to another (again non-existant), which creates circles of 404 crawling. And these pages are not online for more than a month, yet they are constantly detected, last time yesterday. And when I take a look to see where they came from, it's only from within the website, from another 404 page, nothing external.

Does anyone have any advice how to deal with this? Is there a way to obtain some kind of crawl log where I could see where the hits exactly came from, because information provided from GWT is not suphicient enough. And I still don't know why google crawls non-existant pages.

2 Answers 2


Add a rel="nofollow" attribute to all links except the ones that really exist (such as back to homepage and "perhaps you were looking for some other article" links) on your 404 pages.

Have you also checked your sitemap.xml (or whatever you named the file) in Google Webmaster Tools? Perhaps the links are all there and Google tries to look for them.

Another way to deal with it is to get rid of the 404's altogether and set 301's or 302's respectively to the next existing or replaced-by article in your shop.

As you said, the pages do not exist anymore and SEO-wise it would be best to pass the old pages' ranking values to existing pages instead of triggering a hard 404.

  • Thank you for your answer. My home page is shown as 404, but sent with 404 code (magento feature). Sitemap is ok, changes daily, but the links I stated above are not in it, and can't figure out where do they come from.
    – Relja
    Feb 27, 2013 at 14:32
  • 1
    No Sir. Your homepage (I google'd it ;) ) is not a 404. It's being transferred with a header 200 correctly. If you go to your landingpage that is being 404'ed. But instead it should be 301'ed and redirected to an alternate page. You are just killing your rankings that way...
    – David K.
    Feb 28, 2013 at 10:12

There is a small trick to land all this traffic to your home page. You need to add a location of your home page for example index.html or nodeID on 404 error page. And all of the traffic will be directed to home page.

Later you can thoroughly check sitemap and add rel="nofollow".

Hope this will solve the above mentioned problem for you.

  • Please don't use self-promotion link.
    – Zistoloen
    Apr 29, 2013 at 9:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.