1

I am receiving Crawl Errors in my Google Webmaster Tools reports with some unique URLs that I am not understanding.

For Example:

http://www.example.com/portfolio/?cat_url=custom-cms

This page opens my portfolio page, which the URL for is currently:

http://www.example.com/portfolio/

From where, and why are these URLs crawled by Google?

I am also seeing a similar list of URLs that I have not created but am finding them in my crawl errors.

Some pages that are listed here were old pages and they were totally removed from my database, but I don't know how Google found them.

Last week I requested Google to remove these URLs from their cache and marked these errors as fixed. But various links have come back to the list of errors.

Will this effect my SEO and Google ranking as well?

1

I am going through something similar though my situation is slightly different.

If all of your on-site links are correct and any sitemaps that you have are correct (and I would check them), then I would suspect that these are coming from links made by scraper spammer sites. Your one example is a bit odd though. The cat_url=custom-cms, unless something your site software exposed, is very strange. I would not begin to guess how this happened. Your old URLs are a strong indication of a scraper spammer site. In the past, these sites would use the Google API and build databases for reuse when creating garbage sites. These databases are shared, sold, and live for a very long time. Google now has restrictions that limits this activity. But spammers continue to use these databases that are years old.

It is likely that Google spidered one or more of these scraper spammer sites and is seeing that these are pages are not known and putting it into their fetch queue.

A bit of a warning: too many people get wrapped up with Crawl Errors like this is always something to fix. Sometimes it is, but most of the time it is not. If these are showing up as 404, you can ignore them. Do not mark them as fixed. You are just resetting the retry counter when you do. With 404 errors, Google will retry a certain number of times and then stop. It will then drop any entries from the index and fetch queues. The problem is, this can take a while so patience is needed. Meanwhile, 404 errors are not a problem unless a valid page shows up in the list.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.