Common value for a HTML robots meta tag.

learn more… | top users | synonyms

4
votes
3answers
9k views

How to stop certain urls from being indexed

When I type site:example.com(using my domain obviously), I get several link errors showing up in the listing. Typically, they are of the form: /some/fixed/path/admin/unblockUser/11 I am thinking of ...
6
votes
1answer
142 views

How does “Noindex:” in robots.txt work?

I ran across this article in my SEO news today. It seems to imply that you can you use Noindex: directives in addition to the standard Disallow: directives in robots.txt. Disallow: /page-one.html ...
7
votes
4answers
6k views

Hide a Subdomain from Google/Search Engine SEO Result?

I have a subomain which I do not want to be listed in any search engine result. Let say I have: http://www.example.com http://child.example.com How can I hide all URLs of child.example.com domain ...
4
votes
8answers
768 views

Should I block indexing of my mobile site?

I notice that Yahoo and Bing have started indexing the mobile version of my website, which is located at /mobile from the root. All of the mobile content is just a reduced layout version of the real ...
4
votes
2answers
590 views

Should I prevent search engines indexing tag/category pages?

On my site, I currently have no special rules for search engines. It is a blog, statically generated using a Python program. When I search for some of my articles on Google, there is usually a tag or ...
3
votes
2answers
318 views

NOINDEX large number of pages to fix Google thin content manual action?

I've just been hit with a site wide manual spam action on Google due to a significant percentage of 'Thin content with little or no added value'. I have a large number of posts that are affiliate ...
3
votes
2answers
132 views

Not index page that doesn't have relevant content?

I have a large software website and on each application we let users add comments, reviews and so on. Each of these pages are called for example "Comments About Firefox", "Firefox Reviews" and so on. ...
2
votes
3answers
357 views

Should category pages be noindex regarding panda?

I have seen this question, but I believe the situation has changed a lot with the recent Panda updates. I have created a set of category pages to facilitate navigation on my site. It is a proper ...