Would adding noindex on index.html stop main domain from being indexed?

Question

For various reasons, I don't want my domain.com/index.html from being indexed because I have the same file with the same content but called domain.com/home.html instead.

Now I added the noindex meta tag in my index.html file so search engines would not index it. Although, I was wondering if by doing that i'm preventing my domain.com from being indexed. Since it's basically the same URL, typing both variants will get you to the same page.

Answer 1

Most likely yes, if there is a file which powers both URLs they will both have the noindex tag in the source code

Simple way to find out is add the noindex tag, then go to your htttp://domain.com/home.html and view the source code and search for noindex. If you can see it in the source code of your homepage, so will search engines and the page will be noindexed in time.

Another way to use noindex is in HTTP responses using .htacces, rather than adding it to the file code. Add the code below to your htacess to noindex just domain.com/index.html

<Files index.html>
   Header set X-Robots-Tag "noindex"
</Files>

Another way to avoid duplicate pages is 301 redirect the duplicate URL. You can do this by adding the code below into the .htaccess file on your site:

# Needed before any rewriting
RewriteEngine On

### Place after 'RewriteEngine On' and before any CMS specific rewrite rules

## 301 Redirects
# 301 Redirect 1
RewriteCond %{QUERY_STRING}  ^$
RewriteRule ^index\.html$ /domain.com/home? [R=301,NE,NC,L]

Another option is use canonical tags. Add the code below in the source code of the file the powers both URLs:

<link rel="canonical" href="http://domain.com/home.html " />

When search engines crawl http://domain.com/index.html, the tag tells them to count the URL as a single URLs, http://domain.com/home.html

And if for some reason you cannot do any of the above, you can block it in robots.txt, which might look something like this:

User-Agent: *
Disallow: /index.html$

This will block search engines from crawling http://domain.com/index.html, it won’t necessarily remove the page from the search results pages, but it will most likely be added to the omitted results and won't cause issues with duplicate content. This is the least favoured option though.

Also I might add, that if you have duplicate issues with http://domain.com/index.html and http://domain.com/home.html, then either of these pages are most likely a duplicate of your root home page, http://domain.com/. If so, then follow the same steps above, but so that all duplicates resolve too http://domain.com/