Webmasters Stack Exchange is a question and answer site for pro webmasters. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

At http://www.microsoft.com/robots.txt, why are there entries like:

Disallow: /*/mac/help.mspx
Disallow: /*/mac/help.mspx?
Disallow: /*/mactopia/help.mspx?

Could there be a real reason to not allow spiders to index those pages or is it something nefarious?

share|improve this question

closed as off topic by John Conde Apr 4 '11 at 12:01

Questions on Webmasters Stack Exchange are expected to relate to webmastering within the scope defined by the community. Consider editing the question or leaving comments for improvement if you believe the question can be reworded to fit within the scope. Read more about reopening questions here.If this question can be reworded to fit the rules in the help center, please edit the question.

I imagine they didn't like it when google brought them up when looking for mac. I bet that the pagerank of microsoft.com is rather high compared to apple simply due to market share. – Ape-inago Apr 2 '11 at 20:10
Can you explain why you want to know? I'm not sure how this related to running a website. – John Conde Apr 3 '11 at 13:50

It's because Microsoft has products for mac, but it wants to separate its primary products from Apple's when people are searching for 'mac'. This way you get Microsoft reaults as much as possible, unless you're searching for mac software from Microsort specifically.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.