Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

Possible Duplicate:
Blocking Search Engines to Crawl Parts of a Website?

In general, I want my site to be indexed, but there are certain files I don't wish to be searchable from Google, Yahoo, etc...

I know you can block your site from being indexed using a robots file, but what if I only want certain files to not be indexed?

share|improve this question

marked as duplicate by danlefree Dec 26 '11 at 1:36

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

1 Answer 1

For this you can use robots.txt too. Then you would write several disallow statements. Here are some examples:

Disallow: /tmp
Disallow: /logs

For more information see the specification of the robots file.

But be aware that not every spider respects this file. So maybe you need other methods like e.g. the nofollow meta tag if you really want to be sure.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.