How to exclude a website from real-time Google search results?

Question

Google bots have recently indexed my test web site which is not intended for users. How can i exclude my JSP web site from google's search result?

Mikael Svenson · Answer 1 · 2011-01-22 19:07:27Z

Add a robots.txt file on the root of your server with:

User-agent: *
Disallow: /

You can also add this to your start page (or any page):

<html>
<head>
<title>...</title>
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
</head>

This should stop all indexing for crawlers who play nice.

migrated from stackoverflow.com Jan 24 '11 at 2:35