Large increase in 404 hits

Question

We're running a Magento shop and we experienced a significant raise in 404 hits lately, almost 40,000 in one day. Most of the pages are old ones that once existed but don't anymore, and there are some really strange occurrences where as much as 200 pages are linked from one to another, for example

/kleidung-accessoires-schmuck/jungs.html?manufacturer=633-341
/kleidung-accessoires-schmuck/maedels.html?manufacturer=420-346-440
/kleidung-accessoires-schmuck/maedels.html?manufacturer=114-510-414

None of them really exist, all return 404 error, but somehow Google sees them and follows the links in them to another (again non-existant), which creates circles of 404 crawling. And these pages are not online for more than a month, yet they are constantly detected, last time yesterday. And when I take a look to see where they came from, it's only from within the website, from another 404 page, nothing external.

Does anyone have any advice how to deal with this? Is there a way to obtain some kind of crawl log where I could see where the hits exactly came from, because information provided from GWT is not suphicient enough. And I still don't know why google crawls non-existant pages.

Answer 1

Add a rel="nofollow" attribute to all links except the ones that really exist (such as back to homepage and "perhaps you were looking for some other article" links) on your 404 pages.

Have you also checked your sitemap.xml (or whatever you named the file) in Google Webmaster Tools? Perhaps the links are all there and Google tries to look for them.

Another way to deal with it is to get rid of the 404's altogether and set 301's or 302's respectively to the next existing or replaced-by article in your shop.

As you said, the pages do not exist anymore and SEO-wise it would be best to pass the old pages' ranking values to existing pages instead of triggering a hard 404.

Answer 2

There is a small trick to land all this traffic to your home page. You need to add a location of your home page for example index.html or nodeID on 404 error page. And all of the traffic will be directed to home page.

Later you can thoroughly check sitemap and add rel="nofollow".

Hope this will solve the above mentioned problem for you.