2

I just saw in my WMT that I have a lot of indexed soft 404 pages. They all starts with mysite.com/index.php/somepage.. The problem is that I don't have not a single page starts with index.php.

Can someone explain me:

  1. Why is those soft 404 pages generated at all when I don't have index.php part on my site?

  2. Now the thing that worries me is that I have for example 700 soft 404 pages, and after some time I have 300 and I didn't change anything.

  3. Are they harmful for my site?

  4. Can this be fixed in using some rewrite rule in Apache to prevent them for been indexed at all?

4
  • you could fix it using a htaccess file however you would be better using a sitemap generator to see if you can pin point where these errors are coming from and which pages are generating the errors. Do you have wordpress on your site? Feb 6, 2014 at 11:33
  • 1
    If you don't have index.php on your site then presumably these pages return an actual 404 HTTP response code?
    – MrWhite
    Feb 6, 2014 at 11:39
  • You probably have some links to these pages somewhere, most probably on your site source code.
    – Zistoloen
    Feb 6, 2014 at 12:27
  • google changed there error reporting to report the url that is giving the error not the page it is coming from Feb 6, 2014 at 12:37

2 Answers 2

2

You have to identify where all these links come from.

Click on the problem URL and check "Linked from", if it's coming from external domain then you have to contact those sites to request removing their link. If those site owners are unwilling to remove or failed to respond, use the Google Webmaster Tools disavow links tool.

If linked internally (from your own domain) use the method discuss in the other answer and comments. You may also consider a redirect (to the main page) instead returning a 404 error.

After all, If you have too many 404 errors, Google will think your site is low quality and definitely will harm your search ranking.

1
  • Good info - the last part kind of depends, and is really based more on things like signals from bounce rate, and slowing the crawl rate for your site. The OP should look at this Google Webmaster Tools answer and this Official Blog.
    – dan
    Feb 7, 2014 at 15:54
1

If you ever see lots of 404 entries in your Apache or other webserver logs, then it is possible that malicious software has been used to try and attack the website - trying out a list of known URL paths and checking the HTTP response codes to try and identify any installed software such as content management systems or server administration tools, such as phpmyadmin.

Upon a successful hit, the software then proceeds to try and identify the version by looking at files such as README.txt, README.md or LICENSE.txt which might be present in the same folder. Upon identifying the version, if there are any known exploits for that software version, the malicious software then deploys the exploits and completes a successful attack of some kind.

I mention this simply to raise your awareness, perhaps it is also worth checking your webserver logs to see if a pattern like this has taken place. An IPS-enabled firewall or the Apache mod_security add-on can help address issues like this.

However in your case, and more specifically to answer your question, for the entries to be in Webmaster Tools the links must have been indexed at some point, and I think this article by Susan Moskwa (Google Webmaster Trends Analyst) should help put your mind at ease.

To me it mostly sounds like you have lots of soft-404's because your website when it gives end-users a 404 File not Found error page, it's actually possibly returning an HTTP 200 (OK) response code instead of the corresponding HTTP 404 response code in the HTTP headers.

To be certain, it's worth double-checking: Go here and select the HTTP Headers option on the left, and paste in one of the URLs from your Webmasters Tools 404-list and see if the headers returned include HTTP/1.1 404 Page Not Found. If not, perhaps you need to modify something in your website's page-not-found mechanism - for example in PHP to add the line header('HTTP/1.1 404 Page Not Found'); to the top of the script before any HTML code is output.

1
  • Nice answer, very detailed.
    – dan
    Feb 6, 2014 at 18:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.