1

I have a website that has some pages and then it has a folder that has the majority of the pages. All the pages that exist in this folder could only be accessed by using my website search engine, then I created HTML and XML sitemaps so the search engines could find the pages that are inside this folder. This worked well as search engines got very happy and indexed pretty much the entire website. BUT the html sitemap is appearing in SERPs and besides wanting search engines to crawl it and follow the links I do not want it to be in the SERPs.

What can I do so the links on my sitemap page are followed but it stops appearing in SERP's?

I saw some link rel no index meta tags and robots no index meta tags that seemed to be "designed" for this but some websites say they do one thing and other websites say they do other things. I don't want to screw this up and have my pages not being indexed again because it will be a pretty bad setback.

Besides this, when I made the sitemaps I made the XML and HTML and uploaded them to the server on the same day, it worked like a charm but if the xml sitemap is enough and i can remove the html sitemap with no negative impact to my number of indexed pages it would be a valid solution for me too because then i wouldn't have to bother with it appearing on SERP's so if anyone made this experience and has something to share it would be useful too.

Improve this question
2
  • Any reason you don't want your HTML sitemap in the SERPs? It doesn't hurt and can get you visitors if they see it in their search results.
    – John Conde ♦
    Commented Sep 22, 2011 at 0:29
  • @John Conde when google displays my main page in the SERPs it shows those "important site areas" beneath it and one of them is my sitemap, i wouldn't have a problem with it if the sitemap wasn't generating a huge lot of rejections instead of people continuing to navigate on my site. i wouldn't have a problem with rejections either if those rejections were from visitors that had seen the site already but they are happening a lot with the new visitors that are given a bad first impression of the site and i think it's bad. but maybe you're right, why not...
    – wxiiir
    Commented Sep 22, 2011 at 0:44

1 Answer 1

Reset to default
2

If you use the X-Robots-Tag tag, either as a meta tag or HTTP header with a value of noindex then that page will not appear in the SERPs. The links should be followed unless you use nofollow in the links, X-Robots-Tag meta tag, or X-Robots-Tag HTTP header on the HTML sitemap page.

Improve this answer
1
  • thank you for your answer, it was very helpful even if i might want to change my opinion because of your comment regarding this matter on this particular situation, this will still be a very useful piece of advice for later occasions when i might need it
    – wxiiir
    Commented Sep 22, 2011 at 0:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.