4

I have a website where Google Webmaster Tools reports 15,000 links as 404 errors. However, all links return a 200 when I visit them. The problem is, that eventhough I can visit these pages and return a 200, all those 15,000 pages won't index in Google. They aren't appearing in search results.

These are constant errors Google Webmaster Tools keeps reporting and I'm not sure what the problem is. We've thought of a DNS issue, but it shouldn't be a DNS issue, because if it were, no page would be indexed (I have 10,000 perfectly indexed).

Regarding URL parameters, my pages do not share a similarity in URL parameters that can make it obvious to me what could be causing the error.

Improve this question
7
  • I am experiencing a similar problem.
    – c s h
    Commented May 10, 2013 at 22:52
  • Its unlikely this will help but could we see a url of a page that is not indexed?
    – Simon Hayter
    Commented May 10, 2013 at 23:10
  • Hi @c-s-h any luck on your end? And sorry we cannot post URLs :( - and we're still having a rough time with this.
    – Edgar Quintero
    Commented May 17, 2013 at 20:44
  • @EdgarQuintero Yes not much luck.
    – c s h
    Commented May 19, 2013 at 2:34
  • @EdgarQuintero Google reports are about 3-4 days old usually, look at the time at which the crawl happened. Also try to curl your urls to see if you see a 404 or 200.
    – Sairam
    Commented Sep 8, 2013 at 5:25

2 Answers 2

Reset to default
4

Google offers a fetch as Googlebot option as part of Google Webmaster Tools.

Use that tool to fetch your pages from Google's servers. That will show you if it is a problem with connectivity from Google or with the Googlebot user agent as opposed to downtime of your website when nobody can reach it periodically.

Improve this answer
0

I suggest you temporary put noindex for those 404. It would very much harmful to your site where 2/3 is not indexed.

Then slowly identify what's the real problem. Since you can't disclose your link then no way we can tell what are the real problem.

But I sure you can find plenty of sources on "why google didn't indexed my pages".

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.