Website is redirected to remove the www so www.example.com is redirecting to example.com.

Question is do I need to include the www in the sitemap for the robots.txt?

  • Not what your asking, but FYI adding sitemap to robots.txt in general is pretty useless, since you tell Google and Bing where the sitemap is in Webmaster Tools. May 12, 2016 at 14:01
  • Do I need to include the www? Short answer. No.
    – closetnoc
    May 12, 2016 at 14:46
  • @SimonHayter: … and all other search engines (current ones and those to come in the future) are out of luck and can’t find the sitemap? I’d say it’s the best practice to reference the sitemap in the robots.txt.
    – unor
    May 12, 2016 at 20:23
  • The other search engines such as DuckDuckGO have no problems crawling the site, those that can't are not worth considering anyhow. May 12, 2016 at 21:31
  • Found several articles claiming it to be best practice to include a sitemap reference in the robots.txt and submit via console for faster indexing.
    – CZorio
    May 13, 2016 at 13:06

1 Answer 1


In your situation, Don't include "www" because web crawlers that do read the sitemap directive from your robots.txt will attempt to access the "www" version of your sitemap and then be redirected to the non-www version of the URL.

Not only will you be wasting a couple extra bytes by including "www", but the crawlers that parse the sitemap directive properly will be making two requests to your server: one for the www version, and one for the non-www version.

  • Of course, if you were me, you would put in several strange redirects and www along with random versions of something like www1 in the sitemap just for fun.
    – closetnoc
    May 12, 2016 at 17:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.