There are some pages on our websites that we do not want indexed by Google or other search engines, so we leave them off of the XML sitemap we provide to search engines.
To be safe, is it also necessary for us to add noindex and nofollow tags to the individual pages that are excluded? Or should it be enough to rely on search engines respecting the site map?
The sitemap is just to help Google find pages that it doesn't know about and sometimes give them a bit more detail about files, especially on larger sites. It doesn't prevent a page being indexed, and isn't a requirement for a site to be indexed.
If you don't want a page to be indexed, you will need to add those meta tags (noindex,anyway) and/or block crawlers accessing the specific pages via your robots.txt file.
robots.txt prevents crawling, not indexing. Sometimes google indexes pages that it cannot crawl (in search results it will show the message "a description for this result is not available because of this site's robots.txt").
Commented
Jun 5 at 19:13
Based on my experience, you need to evaluate if the URLs are actually a duplicate or similar content of what you want to present in Search Results. IF there is a duplicate or similar content, then a canonical tag will do the work. If we talk about only pages or blog posts.
For a local business website, I will just let the sitemap active for pages and posts. Now if it's a website with a ton of educational posts, where each post require categories to find the information easy, then I will also index the categories.
