At http://www.microsoft.com/robots.txt, why are there entries like:

Disallow: /*/mac/help.mspx
Disallow: /*/mac/help.mspx?
Disallow: /*/mactopia/help.mspx?

Could there be a real reason to not allow spiders to index those pages or is it something nefarious?

  • 1
    I imagine they didn't like it when google brought them up when looking for mac. I bet that the pagerank of microsoft.com is rather high compared to apple simply due to market share. – Ape-inago Apr 2 '11 at 20:10
  • 1
    Can you explain why you want to know? I'm not sure how this related to running a website. – John Conde Apr 3 '11 at 13:50

It's because Microsoft has products for mac, but it wants to separate its primary products from Apple's when people are searching for 'mac'. This way you get Microsoft reaults as much as possible, unless you're searching for mac software from Microsort specifically.

| improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.