How does a webmaster tell search engines to remove any indexed HTTP URLs/pages allowing only the HTTPS protocol to both remain and be indexed later?

Question

Hello Webmasters of StackExchange and happy holidays!

I signed up an hour ago and have read all the rules for asking questions. I've used StackOverflow for years but this is my first time joining a separate part of the community and network. I am happy to have signed up. But I digress.

I have viewed all other questions that are related to my problem but there is no issue exactly like mine although some are very close. The problem with current question/answers is they simply do not provide a solution for my issue. If I have somehow overlooked one, my apologizes. Please point it out. I will ask my question and give you a brief scenario to help you understand the problem while keeping it all universally relevant to any webmaster.

My Question? How does a webmaster tell search engines to remove any indexed HTTP URLs allowing only the HTTPS protocol to both remain and be indexed later?

What Do I Mean? (The example is a typical scenario for many many webmasters. Scenario is meant to help you understand the prime problem as well as to make it relevant to all webmasters) I have a single page website that was created and indexed in Google. It was ranking in the top 5 spot for my keyword. It was a normal HTTP protocol site:website.com.

In the Google Webmasters tool I claimed/added both versions of the site: and the . (respectfully www and without but as a standard HTTP protocol).

After a few weeks I secured an SSL Certificate and activated it on the domain via my registrar and web host. I then too added both site variants to the Google Webmaster Tool. This time however with the HTTPS protocol. I added and .

I submitted my site to Google for index. The site was indexed that very night and was now in the #1 spot the page/URL was the HTTPS protocol. A clear bump in rank due to the SSL. I checked again a few days later and it was still in the #1 spot but it was the HTTP protocol URL/page that was ranking.

I ran a quick test to verify my suspicions: site:website.com and two pages where returned rather than just one. A single page site so just one should have returned. I now have in the Google Index two entries of the same page (duplicate content) using separate protocols.

• www.website.com (It is the HTTP version but in Google if it is not HTTPS it will show in the SERPs as seen above without the protocol prefixed to the URL.)
• https://www.website.com (the newly index version.)

How Would A Webmaster Fix This Using SEO And Other Best Practices? (The Actual Goal)

I would like to remove/de-index the HTTP version(s) of the website without hurting my rankings and do so in such a manner that no future HTTP pages are indexed. (This may have made a better title question. >.<)

TL:DR

• Properly, without effecting SEO, de-index the HTTP version of the site from Googles index.
• Using robots.txt, .htaccess, or other best practice, direct search engines to only index HTTPS pages for this domain/website now and in the future.

Thank you! I hope the question is short enough yet holds sufficient enough information that you are able to understand it and that the simple scenario I used (which happens to be the actual case) makes this even more universally helpful to other webmasters. Your answers are gratefully appreciated! I fear I have only a day - 72 hours before Google penalizes me for this as it's already all happened so fast.

I look forward to your answers!

Regards, DeveloperDan

Answer 1

1. First of don't block anything(http website) via anything(robots.txt or meta tags).

2. Use 301 redirection from http://domain.com to https://domain.com (Feel free to use www subdomain if you want). So both your user and googlebot will redirect to your https version, and Google will index the https version of webpage and will drop http version of that page from search result, the ranking will not change a more(Slietly it does initially, but you will get back in few months).

3. Use canonical link tag on http webpages and make sure it point to https version of website.

Migration from http to https is really easy, just don't block anything. Google already trying to access your https version of website when it is available, so if you provide same content on both protocol(http and https), then Google will pick only https version and will handle duplicate content it self, but setting proper 301 redirecion and canonical link tag really help to get your ranking position back quickly.

If you're site is really big then do 301 redirection in few section of website, and see your search console to get analytics data, but if it is small than full migration will be good choice.

Fore more reference read this epic guide from John Muller.

Answer 2

You are really asking the same old question with anxiety attached. There really is nothing new here. You are not unique in wanting your site to perform well and worry about making changes. The fact of the matter is that some changes require patience. You cannot get so wrapped up in your site and worry about the transition as not to see the payoff when a change is made. Transitions are a part of the proposition in making the change. It is a course of business on the web. So relax. It will all work out. You just have to accept the fact that there will be a transition period that is required and not worry about it.

The good news is, you have already decided what you want to do.

The next thing for you to do is not worry. Any change in sites is a disruption. However, you are fortunate that you are making this disruption early. It is a lot easier to take than if you decided to make a change a year or more down the road. The disruption would be much greater.

So how is this normally done?

Each unique URL and protocol is a website. Not to confuse, this would be example.com (HTTP), www.example.com (HTTP), example.com (HTTPS), and www.example.com (HTTPS). This is because of how search engines organize their index. Though not typical, in theory, each of these can serve different content therefore must be indexed as separate sites. Assuming that they do all serve the same content, only one can rank. This is why you have to make a choice.

Using www.example.com (HTTPS) for our example, one would create valid sites on their server for each of the remaining three and redirect them to www.example.com (HTTPS) using a 301 redirect. This requires a valid certificate for example.com (HTTPS). Any sitemap you create would be for the site www.example.com (HTTPS). You would create a property for each of the four in Google Search Console. You would select www.example.com (HTTPS) as the preferred site.

Optionally, you can put in a change of address using the Google Search Console from your HTTP site to your HTTPS site, though this is really not generally necessary.

Then sit back and wait.

You will see the site de-indexed using HTTP and indexed using HTTPS. Anytime you make a change like this, you will see a disruption. For newer sites, this is often a shorter disruption. Depending upon how often Google visits your sites pages, it would require each page to be fetched again. Optionally, if your site is small, you can use the Fetch as Google and Submit to Index options within Googles Search Console. However, I do often warn people not to get too carried away using this option. Submit your most important pages first, followed by others each day. If your site is a few hundred pages or more, I would not try and submit them all.

But what about your SEO question?

In time, your performance under HTTPS will be the same as HTTP with the exception of adding a bump in your trust score for using HTTPS. That is it. Any links you may have earned using HTTP will be passed to the HTTPS site. There is nothing to do except wait and be patient.