1

As content fluctuates on our site, we will obviously add the titles to the new pages to our sitemap, and link list for search engine indexing.

Through time, certain links will become less relevant, and would like to know how to avoid them being crawled. The links themselves may not be removed - but we don't want to dilute the link list with less relevant links as time passes.

I'm guessing the status code of the page would change - to what? Should they also removed from the sitemap?

Improve this question

1 Answer 1

Reset to default
1

Sitemaps are to tell search engines where to find your pages so they can be indexed. Removing pages will not cause search engines to stop crawling those pages or have them removed from their index.

Additionally, the quantity of links in an XML sitemap will not hurt or help SEO or make search engines think pages are worth more (or less) if there are fewer links (or more links).

Don't remove pages from your site or search engine index unless the information on it is wrong or obsolete. Let the search engines determine how relevant a page is for a search result. That's what they do. And the more pages you have indexed* the better it is for your site as it is more opportunities to be found plus various other on-page SEO factors (internal linking, etc).

*crappy low quality/duplicate content pages not withstanding

Improve this answer
3
  • I see. What about when we deprecate content, or remove URL's - obviously 401. However if these are search results, a results page will simply show no results, or we may want to remove that results page from our sitemap and from being indexed any further - how would you accomplish that - status code?
    – ElHaix
    Jun 26, 2013 at 15:35
  • Pages with duplicate content should be using canonical URLs. That's both the proper and simplest way to handle them. Excluding them from your sitemap might not be a bad idea since those pages have no real SEO value. Pages that no longer exist should be getting a 40x status code and that's all you need to do for them.
    – John Conde ♦
    Jun 26, 2013 at 15:39
  • I meant 404 status code above. I'm not saying the pages will have duplicate content, only that I don't want them to be indexed any longer - I guess simply a rel noindex will do? Later, however, as relevant content increases, we may decide that those lower-priority pages, are now important again. So simply removing the noindex should do?
    – ElHaix
    Jun 26, 2013 at 16:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.