1

Basically, my site has a bunch of pages like Password Recovery, Sign Up, the page that says "Thank you!" after signing up, etc.

Those pages obviously don't have any marketing stuff in them and aren't intended as the landing pages.

Would it make sense to remove them from the sitemap.xml file?

  • Do you want your users to be able to easily find these pages? – John Conde Nov 27 '16 at 14:44
  • For the sitemap, if it is not necessary for Google to index it, then do not worry about. It really is up to you. Your site, you decide. Cheers!! – closetnoc Nov 27 '16 at 16:28
  • 2
    "the page that says "Thank you!" after signing up" should not even be accessible unless a user has just signed up!? – MrWhite Nov 27 '16 at 23:00
  • @w3dk why is that? I don't see how that would create problems for users or security – Victor Marchuk Nov 29 '16 at 13:39
  • It's not so much a "security" issue (or is it?) but a "usability" issue. If a user should stumble across a "Thank you for signing up" page when they've not actually signed up - don't you think that's a little confusing? Your analytics will show hits on the "thank you" page when users haven't actually signed up. It looks careless from a developers point of view (makes me wonder what else has been missed, what hidden vulnerabilities there might be....). – MrWhite Nov 29 '16 at 21:23
3

You should add the URLs of your pages which you want to get crawled and indexed.

Don’t you want your password recovery page to get crawled/indexed? Then don’t add it. (And if you want to disallow crawling of your password recovery page, add it to robots.txt; if you want to disallow indexing of your password recovery page, use noindex; don’t use both).

Should you want your password recovery page to get crawled/indexed? I would say yes. Some users might want to use a search engine to find this page (instead of visiting your site and finding a link to the page). But this is primarily a usability question, it doesn’t affect SEO much.

The same goes for the sign up page.

Your "Thank you" pages ideally wouldn’t exist in the first place (you’d typically display the message on the same page, or on the otherwise existing page you get redirected to after submitting the form), but if you need them, they would be good candidates for pages that shouldn’t get added to the sitemap. And you should noindex them, too.

2

There's no reason to announce them to Google unless you think your site is going to start to rank on keywords like "Thank you" or "Password Recovery".

-1

The sitemap.xml is expected to include absolutely all your public website pages. Google will sort them out as required.

I suppose your question is more likely to be in link with SEO than technology in itself. All my sites that rank well include all their public pages in their sitemap.xml. I really do not think Google cares. It would find them anyway, so having them in the sitemap.xml makes sense.

When I look at the statistics in the Google Webmaster tools, I can see that the it always says that about 10 to 20 pages are not included in their index. This is probably because they do not care about login, logout, thanks pages.

Since only robots are expected to read that file, it is safe to assume that if it includes boring pages, it won't hurt anyone.

However, if you can, add a priority and set it to a very low amount for those specific pages (like 0.1). That way it will probably rarely bother even looking at them. Without the sitemap.xml, you have a way to give a weight to the page. So I think it is a powerful way to tell Google, don't bother too much about those. As pointed out by Stephen in a comment, the frequency and priority parameters are now mostly ignored by Google. Instead the robot uses the last modification date to see what has changed compared against what they have in their index.

  • 4
    I wouldn't expect your sitemap to "include absolutely all your website pages." The sitemap you submit to search engines should include all the pages you want indexed. I would never put pages that are blocked from crawling or indexing into a sitemap built for search engines. – Stephen Ostermiller Nov 28 '16 at 0:49
  • Google has also said that it completely ignores then priority field because they found that it was not being used correctly in most cases. – Stephen Ostermiller Nov 28 '16 at 0:49
  • @StephenOstermiller I fixed the "all pages" into "all public pages". My mistake. Obviously there is no need for pages that Google could not access. Also I did not hear about the priority being dropped. Maybe you have a link to a video or other reference? – Alexis Wilke Nov 28 '16 at 0:55
  • @StephenOstermiller source? – Victor Marchuk Nov 29 '16 at 13:41
  • Google's John Mueller said so in a office hours hangout: seroundtable.com/… – Stephen Ostermiller Nov 29 '16 at 13:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.