Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I need a sanity check on this .htaccess (from a WordPress site) I inherited from a 5 month+ old site. What's the symptom? Google + Bing crawl, but don't index any of the pages. Let me be clear: I'm not mad about "not ranking high." I think something is (accidentally) rejecting search engine indexing.

I am not an expert on .htaccess, but one part especially looked funny, the Limit GET POST line. Is it not weird to have both Allow and Deny all, with no parameters?

Also, I've ruled out robots.txt, but if I were you I'd want to see it, so here it is:

User-agent:  *
Crawl-delay:  30

And here's the more suspect .htaccess:

# temp redirect wordpress content feeds to feedburner
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} !FeedBurner    [NC]
RewriteCond %{HTTP_USER_AGENT} !FeedValidator [NC]
RewriteRule ^feed/?([_0-9a-z-]+)?/?$ http://feeds.feedburner.com/anonymousblog  [R=302,NC,L]
</IfModule>

# temp redirect wordpress comment feeds to feedburner
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} !FeedBurner    [NC]
RewriteCond %{HTTP_USER_AGENT} !FeedValidator [NC]
RewriteRule ^comments/feed/?([_0-9a-z-]+)?/?$    http://feeds.feedburner.com/anonymous_comments [R=302,NC,L]
</IfModule>

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

IndexIgnore .htaccess */.??* *~ *# */HEADER* */README* */_vti*

<Limit GET POST>
order deny,allow
deny from all
allow from all
</Limit>
<Limit PUT DELETE>
order deny,allow
deny from all
</Limit>

php_value memory_limit 32M

Adding header by request:

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta name="robots" content="noindex,nofollow" />
<meta name="description" content="buncha junk i've deleted." />
<meta name="keywords" content="keywords i've deleted" />
<meta name="viewport" content="width=device-width" />
share|improve this question
add comment

1 Answer

up vote 1 down vote accepted

Post your website's header code too. You want to make sure you're not doing something silly like this:

<META NAME="ROBOTS" CONTENT="NOINDEX">

share|improve this answer
    
Added it, and boy it does look suspicious! According to what I just Googled, you're right on the money. But why the heck didn't Google Webmaster Tools mention it... –  jose Jun 12 '12 at 20:35
    
Delete this line: <meta name="robots" content="noindex,nofollow" /> ...and you're good to go. If there is no robots <META> tag the default is "INDEX,FOLLOW". –  utt73 Jun 12 '12 at 20:43
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.