Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

Good day,

I'm currently finding out why I have lost all of my website's rank in google. I don't even appear in google results by the domain. But other sites do link me and they appear in the google results.

I think it's all about leaving my site two months alone and finding out I had 20k in comment spam, which I completely deleted and fixed with filters and adding a new Disqus comment service.

Thing is, I added my site to Google Webmaster Tools and I'm finding out several awful things. For example, when I click in Google Fetch As GoogleBot. I receive this error message below in response to my request. And I don't even know what's the real problem and how to fix it. I simply don't get it. This is what appears:

Date: Wednesday, July 20, 2011 9:43:35 AM PDT

Googlebot Type: Web

Download Time (in milliseconds): 55

HTTP/1.1 403 Forbidden Date: Wed, 20 Jul 2011 16:43:36 GMT Server: Apache Vary: Accept-Encoding Content-Encoding: gzip Content-Length: 248 Keep-Alive: timeout=2, max=100 Connection: Keep-Alive Content-Type: text/html; charset=iso-8859-1

403 Forbidden

Forbidden

You don't have permission to access / on this server.

Additionally, a 403 Forbidden error was encountered while trying to use an ErrorDocument to handle the request.

Do you guys know anything about this problem ? I need to have Google crawl my site again. I used to have a really nice google result in the past three years. Now, there's nothing.

thanks,

share|improve this question
    
do you have an improperly created robots.txt file? –  Keith Jul 21 '11 at 12:23
    
I almost erased all of my robots.txt The only content it has is the following : User-agent: * that's all. –  UXdesigner Jul 21 '11 at 12:57
    
how does your .htaccess look? –  Keith Jul 22 '11 at 2:12
    
also, if you are using wordpress, under settings then privacy, does it have 'I would like my site to be visible to everyone, including search engines (like Google, Bing, Technorati) and archivers' checked? –  Keith Jul 22 '11 at 2:15
add comment

3 Answers

Ran into the same issue and the problem stemmed from a line of code in my .htaccess file.

Particularly, I defined a page for "DirectoryIndex" that did not exist, making it impossible for Google to access a non-existent page.

Be sure to do a careful review of this file if you have problems.

share|improve this answer
add comment
up vote 1 down vote accepted

thanks for all your help. They were all very helpful and gave me more information that I didn't know. I ended up checking the .htaccess file , and suddenly found some "Deny" commands with IP addresses after it.

One of the ip addresses was blocking all connections with Google.

So it was being blocked, maybe by a "Bogus" Plugin from Wordpress, or maybe someone possibly hacked into my server and changed the settings, which I find unlikely coz if they could access my account they could have easily been able to erase the website and do something funny. Doesn't seem too unreal as my website has a very high traffic demand and is about politics in my country.

So, the solution is to remove the following line from the ".htaccess" file - see below:

deny from 66.249

Thank you all.

share|improve this answer
add comment

Install the user agent switcher extension for Firefox and create a user agent for Googlebot. Then view your site and see what happens. If you get the above error then your site is configured wrong somewhere. If you don't then the issue lies with Google and you should go to their forums for support.

share|improve this answer
    
Very good approach, clever ! I just followed your instructions and saw the website as the Googlebot would do. . . no errors whatsoever. I have no clue now on what is going on over there. I will check the wordpress configuration and re-check with Dreamhost to see if is there any changes or something. I got to say not even the sitemap.xml is being crawled by Google. Something's really wrong. –  UXdesigner Jul 20 '11 at 20:51
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.