1

If i where to block a web page, say 404Error.aspx, in robots,txt (i see no reason for this page to be indexed by google). However, do i still include this page in the xml sitemap or not?

Edit:

If i wanted my page to be indexed (google follows the links in it), but not show up in search results, should i remove robots.txt and use the robots meta tag with "noindex" ?

Improve this question
2
  • How does the page 404Error.aspx ever get indexed? You shouldn't be redirecting to that page if there is a 404 error...
    – DisgruntledGoat
    Commented Nov 7, 2011 at 14:12
  • 404Error is a page i use to replace the default 404 error. I want it to be crawled as it contains links back to the homepage but i don't want it indexed for obvious reasons.
    – TheGateKeeper
    Commented Nov 7, 2011 at 18:37

2 Answers 2

Reset to default
1

If i wanted my page to be indexed (google follows the links in it), but not show up in search results, should i remove robots.txt and use the robots meta tag with "noindex" ?

Yes.

Improve this answer
1

If you want to stop Google and the other Search Engines to have in their index this page, I see no reason to have the link in the sitemap too. You should have the link only in your robots.txt

Improve this answer
3
  • Thats what i tought but i wanted to be sure, thanks friend
    – TheGateKeeper
    Commented Nov 7, 2011 at 13:25
  • @TheGateKeeper Glad to help you ;)
    – Aurelio De Rosa
    Commented Nov 7, 2011 at 13:27
  • I edited the question, please have a look
    – TheGateKeeper
    Commented Nov 7, 2011 at 13:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.