Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I'm working on building sitemaps and am not sure what should and should not be added.

For example:

https://www.website.com            

https://www.website.com/en/        

http://www.website.com/index.php   

http://www.website.com/en/index.php

All of these URLs lead to the same page. https://www.website.com/ is the preferred address.

Should I write all of these variations in the sitemap and assign the nonpreferred ones an update frequency of never, priority 0. Or simply not include them in the sitemap at all?

Nearly every link on the site could be reached with a similar combination.

Similarly, I have many links that contain parameters https://www.website.com/en/catalog?utm_source=Blog&utm_medium=CTA&utm_campaign=Test

Should these be in the sitemap with the same update frequency never, priority 0 or simply not included?

The documentation I've read, says that the sitemap is used to teach the search engine about your site and give priority to pages, so it makes sense to me that I would want to add all of these links so I can explicitly educate the search engine NOT to bother checking these links.

Additionally

The site contains a catalog of products which can be filtered through in multiple ways (by color, and size for example). Each filter changes the url like such:

The main catalog:          https://www.website.com/catalog/

filtered for blue:         https://www.website.com/catalog/color-blue/

filtered for large:        https://www.website.com/catalog/large/

filtered for blue & large: https://www.website.com/catalog/large/color-blue/

With three types of filters and 5 to 15 options for each filter, this potentially creates 75 different links which essentially display the same content. Should all of these potential links be listed in the sitemap, only the main catalog link, or only the first level of filter links, or every potential URL?

share|improve this question

1 Answer 1

up vote 1 down vote accepted

For all pages that have more that one url, add only the preferred url on sitemap, and do not include the secondary urls. But you have treat these multiple versions, to avoid duplicate content.

For example: https://www.website.com/, https://www.website.com/en/, http://www.website.com/index.php, http://www.website.com/en/index.php all lead to the same place. https://www.website.com/ is the preferred address.

For all the urls that lead to same place, you can and must do one of the following:

Choose one url as the main url and then:

  1. Redirect 301 all other to this url
  2. Use the canonical meta tag to all other urls, pointing to the main url

I suggest 301 redirection, and I would use only the main url for internal or external linking.

Do not link this url: http://www.website.com/index.php, but link your preferred version, which should be http://www.website.com/.

Similarly, I have many links that contain parameters https://www.website.com/en/catalog?utm_source=Blog&utm_medium=CTA&utm_campaign=Test should these be in the sitemap with the same update frequency never, priority 0 or simply not included?

When u need extra GET parameters, to determine eg source or order, you cannot perform 301 redirection. You will have to treat this either by using a canonical url , or either by adding the noindex metatag

The canonical meta tag is the correct way to treat this.

You can find some more info about the canonical tag here

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.