I get 18,000 "access denied" errors showing in Google Webmaster Tools every day!

It's strange that it shows for URL's with www but not for those without it (i.e., just domain.com instead of www.domain.com).

Fetch as Google works perfectly for pages got that error.

Google starts to downgrade my website - impressions have dropped from 35,000 to 18,000.

I am using CrowdFlare as a CDN and .htaccess mod_rewrite.

Any help will be extremely appreciated as I am really losing control.

3 Answers 3


The access denied errors in Google Webmaster Tools that you're seeing for only URL's containing www is likely because the Googlebot can't access them.

Some common reasons for that are listed here: Google Webmaster Tools: Access denied errors

Specific to your situation, since this affects only URL's containing www and not those without, it's likely there's an issue either with your web server's configuration or DNS settings pertaining to how requests for URL's with the www subdomain are being handled.

I would suggest using Webmaster Tools' Fetch as Google function again to view the HTTP status codes for some of these problematic URL's, and then checking your web server's configuration (i.e., if 301 redirects are missing or incorrect), and your DNS server settings to see if the DNS record for www is specified correctly too.

It would also be wise to confirm that there aren't any other DNS server issues by using online testing tools like: DNS Health Check


Some things can happen, I tell you 3 I thought:

  • A bot is crawling your site, to know this, check your access log and see if is always the same IP (or almost the same IP)

  • Someone wants to deface your site or troll you. He may have a site with good traffic and have put an iframe inside his site pointing at your site so all the users that enters his site loads your site in the background. Also he may bought cheap adf.ly traffic and added your site. To know this, just check your referrals and check from where the traffic is coming.

  • You site has been linked somewhere in a high traffic famous site. To know this, again check your referrals.

I don't know what server have you installed so I can't tell you how to check IPS and referrals. By the way, you can edit the index PHP script making it save IPS and referrals in a log.


I have an idea that it is related to not being able to crawl urls with cdn-cgi (just a hunch). If so, please see this.

If not, please see the following information about crawl errors and CloudFlare.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.