3

The Google Webmaster panel index says:

Update: As of 5/14/16, Index Status data precisely reflects the specific URL variant for your verified site (e.g., http://www.example.com data is distinct from https://example.com).

Is this saying the index count for http://www.example.com and https://example.com are now counted separately, and if so is it based on protocol or subdomain?

Clicking the Learn More link (https://support.google.com/webmasters/answer/6211453#indexed) brings me to a page that says

May 15, 2016
A data anomaly caused a dip in the data on this day.

Searching May 15th Google Index brings back a variety of user experiences all showing on that date something happened. The results since then have been varied though. Our site since that day has had a +1.5 million index dip.

Is this a real issue or just a reporting bug? Searching site:example.com brings back the same number the webmaster panel has (so I'm presuming not a bug).

  • 1
    Google treats http and https as separate sites just as it treats any subdomain www or without as separate. If your site is accessible via both means then you have duplicate content, unless your using canonical links and you have set your preferred site within Webmaster Tools. If you haven't already you should add all variations to Webmaster Tools, HTTPS WWW HTTPS NON-WWW, HTTP WWW and HTTP NON-WWW. You must use canonical links or a 301 redirect to avoid sites being treated as separate. – Simon Hayter Oct 10 '16 at 17:47
  • Ohhh, didn't realize the protocol was specific as well. Thanks, our canonicals are <link rel="canonical" href="/dir/page-title" /> we'll have to move those to absolute. So it could just be that on 5/14 Google separated the index counts for http and https? The index count for site:http://www.example.com and site:https://www.example.com is the same – chris85 Oct 10 '16 at 17:53
  • 1
    Google has always counted http and https separately. (They are different URLs.) If the index counts for both are the same then it would suggest you are not canonicalising your URLs (as Simon suggests). – MrWhite Oct 10 '16 at 18:12
  • 2
    @chris85 For what it is worth, canonical links should always be absolute. From: support.google.com/webmasters/answer/139066?hl=en - Avoid errors: use absolute paths rather than relative paths with the rel="canonical" link element. – closetnoc Oct 10 '16 at 18:13
  • Great, thanks. I'll go to work on this now. Perhaps the May 14th just separated the counts and made the issue prevalent. – chris85 Oct 10 '16 at 18:26
2
+50

As has been stated in the comments above Google treats sites differently based on protocol as well as if they have different sub domains (including www and non www sites). what this means is that...

http://example.com
https://example.com
http://www.example.com
https://www.example.com
http://some-other-sub.example.com

Are all treated as different sites for crawling and indexing purposes.

As a side note there was an unconfirmed update to Google's algorithm on the 10th May 2016, an update I add that Google has not announced or confirmed either, and so this error may have been due to that update as well. The message appears to have hit many other users as well around the same date.

Due to Google's lack of details on this incident there is no real way to know for certain if this was a temporary bug or if this was due to a change in the indexing algorithm. If it was a temporary bug then I would expect your sites index to return to normal, if not then chances are it has been adjusted by an algorithm change.

Additionally I note from your comment that you have canonical links on your site. You should ensure that your canonical links are an absolute URL not a relative path and that the root protocol and domain are the main ones that you want appearing in the Google index and that you want customers to access predominantly. In other words if you want the Google SERP to show your site as https://www.domain.com/whatever then you should make sure that all of your canonical links start with https://www.domain.com/.

  • The count for https://www.example.com and http://www.example.com are the same when searching with site: on Google. With the webmaster panel though there is a noticeable difference between the 2. Is there a way to use the site: search feature to get a live view of the index count? Currently we wait +1 week to see the changes we're trying to take affect. – chris85 Oct 27 '16 at 19:39
  • No there is no way through search to do what you are requesting. The site: flag is meant to allow the searcher to filter results to one site only. Additionally the index count isn't quite live, what happens is the crawler gets a copy of your page, stores a cache of it on Google's caching servers. Then the next step is that the indexing server go through the page and evaluate the appropriate context-specific keywords and add them to Google's inverted index (A generalisation of the process but basically accurate). This can take anywhere up to 2+ weeks to be completed, so not live at all. – Chris Rutherfurd Oct 27 '16 at 23:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.