Webmasters Stack Exchange is a question and answer site for pro webmasters. It only takes a minute to sign up.
Sign up to join this community
I have a website example.com where I have a number of pages where comments can be made. Instead of using postback to post the comments back to the server, I use Ajax but it seems to have created additional pages.
http://example.com/page
http://example.com/page/postcomment
It has indexed the postcomment page rather than the one without the postcomment. How can I prevent Google from indexing postback function calls?
Look here:
http://www.robotstxt.org/meta.html
It tells you how to make search engines to NOT index pages you don't want it to index.
All you need to do is insert this into the HTML of your http://example.com/page/postcomment page is this between <head> and </head>:
<META NAME="ROBOTS" CONTENT="NOINDEX">
And make sure such line is removed from http://example.com/page
Also, I think google wanted to index http://example.com/page/postcomment because based on the URL, it appeared more interesting than just http://example.com/page.
And, to increase the odds of a specific page being indexed, add a meta description and a unique title to the page.
<head> section.
– Stephen Ostermiller♦
Dec 6 '16 at 16:39
robots.txt
– Alfred Wallace
Dec 9 '19 at 16:08
