My website, let's say www.example.com
ranks fairly well ranking 1-5 for 6 keywords. On the other hand, my blog on the same website keeps on getting de-indexed.
The Website is HTML (www.example.com), and the blog (www.example.com/blog) is hosted on WordPress.
Some information about the blog: It has extremely good quality original content, got ~700 views organically for each individual blog post. So there's no chance of a copied content or a Google penalty. Google webmasters still shows no message.
There are no server crashes. As hosting is same for both website and blog, we can rule out the possibility of de-indexing because of server crashes.
There's no .htaccess file.
I checked Robots.txt for possible mistake that might have been blocking the search engines. I even cross checked the url using GWT's testing tool that says www.domain.com/blog is not blocked by robots.txt
I have a sitemap.xml.
Although, I am yet to set the preference for www or non www for my domain in GWT. But this isn't something that should have resulted in de-indexing.
I am using https:// for the entire site including blog. While all page appear to be secure, blog does have some sources which are not secure. But, the blog has been that way since a year and nothing happened.
Also, when I add url to fetch as Google, the submit to index option doesn't shows up.
NOTE: My Wordpress settings mistakenly were set to stop search engines from indexing the blog. I changed that a month ago, the blog was indexed, but the pages were not. Shortly, the blog got de-indexed again, I re-indexed it, and got de-indexed again, while other pages never got indexed.
Now the blog is not indexed, however, one of the blog post is indexed. But a google search display the meta description as "A description for this result is not available because of this site's robots.txt – learn more."
What should I do now? I have tried everything. Nothing seems to work.
<meta name="robots" content="index, follow">
? because that does not look like your loading that via wp_head but rather a manual entrie.