Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I've got a major problem in that emails sent to our clients are showing up in Google's search results. For confidentiality reasons, I can't post those links (for risk that they get indexed here), but the format is something like this


where xxhh1213131 is a unique "token" that for each of our recipients.

Any idea how this might be happening? We have a "view as webpage" link in our emails. Could this have happened from traffic to those links?

share|improve this question
up vote 8 down vote accepted

If the search engines can find the page those links are on and are not told to ignore those pages, they will index them like any other web document. <speculation>In your specific case they probably found them when the user went to view them. Either the browser or an extension in the browser sent tha information to Google and they then crawled those pages.</speculation>

If you do not want them indexed, and would like them to be removed, you need to tell the search engines to block them. There are several ways to do this:

1) Use robots.txt

2) Use meta tags

<meta name="robots" content="noindex, nofollow">

3) Use http headers

Header set x-robots-tag: noindex

4) Use rel="nofollow"

<a href="http://www.example.com/sample.html" rel="nofollow">Link to page I don't want indexed</a>

5) Place the content behind a login. Search engines (generally) do not submit forms nor create accounts at websites.

6) Block the bots of all of the major search engines using .htaccess but that would be tedious and prone to error if they change whatever it is you would use to identify them (e.g. ip address, user agent).

share|improve this answer
Thanks for the detailed suggestions! – Ray Jul 11 '11 at 18:51
Keep in mind that only #5 is really the secure. The other choices just hide the choices, but anyway with the right address (or a good guess) could still see the messages... – user1205 Jul 12 '11 at 2:53
Especially the world readable robots.txt, if I see a Disallow: /secret/mails there, I will have a closer look. ;-) – Perleone Feb 25 '13 at 17:23

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.