Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I get 18,000 "access denied" errors showing in Google Webmaster Tools every day!

It's strange that it shows for URL's with www but not for those without it (i.e., just domain.com instead of www.domain.com).

Fetch as Google works perfectly for pages got that error.

Google starts to downgrade my website - impressions have dropped from 35,000 to 18,000.

I am using CrowdFlare as a CDN and .htaccess mod_rewrite.

Any help will be extremely appreciated as I am really losing control.

share|improve this question

3 Answers 3

The access denied errors in Google Webmaster Tools that you're seeing for only URL's containing www is likely because the Googlebot can't access them.

Some common reasons for that are listed here: Google Webmaster Tools: Access denied errors

Specific to your situation, since this affects only URL's containing www and not those without, it's likely there's an issue either with your web server's configuration or DNS settings pertaining to how requests for URL's with the www subdomain are being handled.

I would suggest using Webmaster Tools' Fetch as Google function again to view the HTTP status codes for some of these problematic URL's, and then checking your web server's configuration (i.e., if 301 redirects are missing or incorrect), and your DNS server settings to see if the DNS record for www is specified correctly too.

It would also be wise to confirm that there aren't any other DNS server issues by using online testing tools like: DNS Health Check

share|improve this answer

Some things can happen, I tell you 3 I thought:

  • A bot is crawling your site, to know this, check your access log and see if is always the same IP (or almost the same IP)

  • Someone wants to deface your site or troll you. He may have a site with good traffic and have put an iframe inside his site pointing at your site so all the users that enters his site loads your site in the background. Also he may bought cheap adf.ly traffic and added your site. To know this, just check your referrals and check from where the traffic is coming.

  • You site has been linked somewhere in a high traffic famous site. To know this, again check your referrals.

I don't know what server have you installed so I can't tell you how to check IPS and referrals. By the way, you can edit the index PHP script making it save IPS and referrals in a log.

share|improve this answer

I have an idea that it is related to not being able to crawl urls with cdn-cgi (just a hunch). If so, please see this.

If not, please see the following information about crawl errors and CloudFlare.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.