We're running a Magento shop and we experienced a significant raise in 404 hits lately, almost 40,000 in one day. Most of the pages are old ones that once existed but don't anymore, and there are some really strange occurrences where as much as 200 pages are linked from one to another, for example
None of them really exist, all return 404 error, but somehow Google sees them and follows the links in them to another (again non-existant), which creates circles of 404 crawling. And these pages are not online for more than a month, yet they are constantly detected, last time yesterday. And when I take a look to see where they came from, it's only from within the website, from another 404 page, nothing external.
Does anyone have any advice how to deal with this? Is there a way to obtain some kind of crawl log where I could see where the hits exactly came from, because information provided from GWT is not suphicient enough. And I still don't know why google crawls non-existant pages.