I read up on the limitations and how you can link multiple sitemaps. But my question is, are they to contain links to EVERYTHING or just the most recent X thousand changes?
Ideally your sitemap.xml contains:
- Every page on your site (at least every one that you want indexed; login pages, legal boilerplate, certain archives etc should perhaps be exluded).
- With a correct
<lastmod>date accurately reflecting when the content of the page last changed.
Now, if your site is very large, you might need to use the sitemap to only inform the search engines of the newest, most recently added pages on your site. Alternatively you can make multiple sitemaps to have all pages included while still keeping each sitemap below the size limit.
I usually have mine only contain links to every page on my site ... that I want to get indexed by search engines. Search engines are the only users who read the sitemap.xml, so I would not include pages that you do not want to get indexed.
- Admin areas
- Duplicate pages (like archives in wordpress)
Now the actual sitemap web page should contain a link to every single page in your website (or at least a way to get to every page).