Webmasters Stack Exchange is a question and answer site for pro webmasters. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I need a sanity check on this .htaccess (from a WordPress site) I inherited from a 5 month+ old site. What's the symptom? Google + Bing crawl, but don't index any of the pages. Let me be clear: I'm not mad about "not ranking high." I think something is (accidentally) rejecting search engine indexing.

I am not an expert on .htaccess, but one part especially looked funny, the Limit GET POST line. Is it not weird to have both Allow and Deny all, with no parameters?

Also, I've ruled out robots.txt, but if I were you I'd want to see it, so here it is:

User-agent:  *
Crawl-delay:  30

And here's the more suspect .htaccess:

# temp redirect wordpress content feeds to feedburner
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} !FeedBurner    [NC]
RewriteCond %{HTTP_USER_AGENT} !FeedValidator [NC]
RewriteRule ^feed/?([_0-9a-z-]+)?/?$ http://feeds.feedburner.com/anonymousblog  [R=302,NC,L]

# temp redirect wordpress comment feeds to feedburner
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} !FeedBurner    [NC]
RewriteCond %{HTTP_USER_AGENT} !FeedValidator [NC]
RewriteRule ^comments/feed/?([_0-9a-z-]+)?/?$    http://feeds.feedburner.com/anonymous_comments [R=302,NC,L]

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]

IndexIgnore .htaccess */.??* *~ *# */HEADER* */README* */_vti*

<Limit GET POST>
order deny,allow
deny from all
allow from all
order deny,allow
deny from all

php_value memory_limit 32M

Adding header by request:

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta name="robots" content="noindex,nofollow" />
<meta name="description" content="buncha junk i've deleted." />
<meta name="keywords" content="keywords i've deleted" />
<meta name="viewport" content="width=device-width" />
share|improve this question
up vote 1 down vote accepted

Post your website's header code too. You want to make sure you're not doing something silly like this:


share|improve this answer
Added it, and boy it does look suspicious! According to what I just Googled, you're right on the money. But why the heck didn't Google Webmaster Tools mention it... – jose Jun 12 '12 at 20:35
Delete this line: <meta name="robots" content="noindex,nofollow" /> ...and you're good to go. If there is no robots <META> tag the default is "INDEX,FOLLOW". – utt73 Jun 12 '12 at 20:43

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.