9

My wife works at a high school in Germany. I recently noticed that it's extremely hard to find that school's homepage using Google. I looked at the source code of the page and I believe I've found the reason: There are two <meta name="robots"> tags; one is

<meta name="robots" content="all">

and the second one, further down in the page, is

<meta name="robots" content="noindex, follow">

My guess is that the second one trumps the first and keeps Google from indexing the site. Am I right? I wouldn't want to tell the webmaster that I've found the problem and make an idiot of myself (because I don't really know anything about how to build a proper webpage, let alone do SEO).

By the way, if you happen to find any other goofs on the site, I would be glad if you could report them. My next candidate would be the utter lack of relevant meta tags that convey information about the page's content.

4
  • 1
    German Wikipedia states that <meta name="robots" content="all"> is semantically equal to <meta name="robots" content="index, follow">, so the both meta tags are contradicting in any way. I would notify the webmaster. I also guess that Google will choose the stricter policy and therefore not index the page to prevent copyright issues. Commented Apr 26, 2013 at 7:38
  • 3
    I also saw that at the end of the page two requests are sent to third party servers (ajax.googleapis.com and ssl.google-analytics.com). This can be a legal issue againt the privilege of informational self-determination in Germany. I would recommend to avoid that. Commented Apr 26, 2013 at 7:40
  • 1
    There are also two title elements. And the same is repeated on other pages, not just the homepage! In fact, there does not appear to be any pages indexed, apart from a handful of PDFs!? Conflicting modules in the CMS perhaps?! Bit of a side issue... As stated in the answers, the most restrictive rule will be applied by the search engines - not necessarily the later rule. However, a function such as PHP's get_meta_tags() will only return the later rule - so there is an immediate (potential) problem with having multiple meta tags for the same data.
    – MrWhite
    Commented May 2, 2013 at 21:12
  • @w3d: Thanks for the insights! This is very helpful. Commented May 3, 2013 at 5:24

2 Answers 2

7

While the German Wikipedia claims that all is a valid value, I've no idea where that information comes from, and German Wikipedia does not cite any source. It is AFAIK wrong, and IMHO a good example for why one should not rely on Wikipedia for this type of information.

If you want to use the robot meta tag to direct googlebot, the best information about valid values is Google webmasters central. It lists the following six values as valid: noindex, nofollow, noarchive, nosnippet, noodp, none.

In other words, for googlebot:

meta name="robots" content="all">

is a syntax error and will be ignored, while:

<meta name="robots" content="noindex, follow">

is valid syntax and will be obeyed (so this explains why it is extremely hard to find that school's homepage using Google).

To answer your second question, what happens if two valid robot meta tags conflict, Google says (same page as the one linked above):

If content values conflict, we will use the most restrictive.

As for meta tags, here is an overview of Meta tags recognised by Google.

2
4

The behaviour for two conflicting meta tags isn't defined, as far as I know, but most likely the most restrictive rule will win.

This happens the similar case of robots.txt file vs meta robots tag. If robots.txt prevents indexing a page and meta-robots doesn't, the page will not be indexed. And if robots.txt allows a page but meta-robots blocks it, it will still not be indexed.

So logic would dictate that if there are two conflicting meta tags, if one blocks indexing it will override the other (regardless of their respective locations on the page).

You should ask the webmaster if they are blocking search engines for a particular reason. If not, remove both meta tags (the default is to allow search engines so no reason to have the tag at all).

As for the other part of your question, site reviews are outside the scope of Pro Webmasters. But I'll add that the meta description tag is probably the only one you will find useful. The meta keywords tag is not used by any search engine.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.