I'm setting up a personal website with a blog that I would like to be indexed by search engines. But the blog will link to CV with personal information that shouldn't be indexed.

Can it be reliably achieved without publishing that personal content in non-document format (e.g. image)?

  • Do you want to keep the page out of search engines, or just the information on it? Yes, it turns out there's a difference. Commented Jan 15, 2012 at 14:42
  • If you want the information to remain private (i.e. not available online) do not post it online - scraper bots do not respect robot exclusion protocols.
    – danlefree
    Commented Jan 17, 2012 at 14:11

1 Answer 1


If it's a HTML document, include the following meta tag in it:

<meta name="robots" content="noindex" />

This will tell Google and every other search engine that understands the tag (which should be pretty much all of them, since it's been around since 1996) not to index the page.

For non-HTML documents, you can configure your webserver to send the HTTP header X-Robots-Tag: noindex instead. This is a somewhat newer feature, but it's understood at least by the major search engines like Google, Yahoo! and Bing.

Yet one more way to keep search engines away from a page is to disallow it in your robots.txt file. However, there's an important caveat: Google, at least, treats robots.txt as a prohibition against fetching the disallowed pages, but not against including their URLs in their index if they're linked from another page they've crawled. And you can't effectively combine robots.txt with the meta tag or HTTP header based methods, since disallowing a page in robots.txt will keep Google from ever even seeing any meta tags or HTTP headers on it.

(However, if you disallow a page in your robots.txt and send Google a removal request for it, they'll keep it out of their index. But that's a somewhat laborious and error-prone method.)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.