Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

We're running a Magento shop and we experienced a significant raise in 404 hits lately, almost 40,000 in one day. Most of the pages are old ones that once existed but don't anymore, and there are some really strange occurrences where as much as 200 pages are linked from one to another, for example

/kleidung-accessoires-schmuck/jungs.html?manufacturer=633-341
/kleidung-accessoires-schmuck/maedels.html?manufacturer=420-346-440
/kleidung-accessoires-schmuck/maedels.html?manufacturer=114-510-414

None of them really exist, all return 404 error, but somehow Google sees them and follows the links in them to another (again non-existant), which creates circles of 404 crawling. And these pages are not online for more than a month, yet they are constantly detected, last time yesterday. And when I take a look to see where they came from, it's only from within the website, from another 404 page, nothing external.

Does anyone have any advice how to deal with this? Is there a way to obtain some kind of crawl log where I could see where the hits exactly came from, because information provided from GWT is not suphicient enough. And I still don't know why google crawls non-existant pages.

share|improve this question
add comment

2 Answers 2

Add a rel="nofollow" attribute to all links except the ones that really exist (such as back to homepage and "perhaps you were looking for some other article" links) on your 404 pages.

Have you also checked your sitemap.xml (or whatever you named the file) in Google Webmaster Tools? Perhaps the links are all there and Google tries to look for them.

Another way to deal with it is to get rid of the 404's altogether and set 301's or 302's respectively to the next existing or replaced-by article in your shop.

As you said, the pages do not exist anymore and SEO-wise it would be best to pass the old pages' ranking values to existing pages instead of triggering a hard 404.

share|improve this answer
    
Thank you for your answer. My home page is shown as 404, but sent with 404 code (magento feature). Sitemap is ok, changes daily, but the links I stated above are not in it, and can't figure out where do they come from. –  Relja Feb 27 '13 at 14:32
1  
No Sir. Your homepage (I google'd it ;) ) is not a 404. It's being transferred with a header 200 correctly. If you go to your landingpage that is being 404'ed. But instead it should be 301'ed and redirected to an alternate page. You are just killing your rankings that way... –  DKOATED Feb 28 '13 at 10:12
add comment

There is a small trick to land all this traffic to your home page. You need to add a location of your home page for example index.html or nodeID on 404 error page. And all of the traffic will be directed to home page.

Later you can thoroughly check sitemap and add rel="nofollow".

Hope this will solve the above mentioned problem for you.

share|improve this answer
    
Please don't use self-promotion link. –  Zistoloen Apr 29 '13 at 9:44
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.