2

on 9/23 we migrated our entire site to HTTPS. Almost 3 months later and Google has recently cached versions of http:// URLs in the index.

We have an http:// sitemap that is live with 100% 301 redirects to https://

We have an https:// sitemap that is live as well with >95% end-state URLs.

Curious on how Google is caching multiple http:// URLs as recently as 12/6.

I started this dive into the index after seeing that our indexation rate had stalled at about 50%. Has anyone dealt with Google being oblivious to a 301 response code? What were the steps taken to remedy?

Improve this question
9
  • 3
    You should never have URLs that redirect in a sitemap. Google wants to see only the URLs you desire to have indexed in the sitemap. I don't think correcting that would completely solve the issue though.
    – Stephen Ostermiller ♦
    Commented Dec 8, 2016 at 18:58
  • 3
    If Google is able to cache the HTTP version then it would seem to suggest the 3xx HTTP redirect is not working - otherwise, the HTTP version of the site is simply not available to be cached in the first place? How are you performing the 301 redirection?
    – MrWhite
    Commented Dec 8, 2016 at 19:01
  • that's my thinking as well @w3dk, the 301s are rewriterules in htaccess, and every non-cached request of the URLs properly redirect when cURLing as googlebot.
    – Zander Buteux
    Commented Dec 8, 2016 at 19:24
  • 3
    Your HTTP sitemap isn't doing anything to get your site crawled. It may be part of the problem. Google uses sitemaps to determine which URLs are canonical. Google expects not to have non-canonical URLs in any sitemap.
    – Stephen Ostermiller ♦
    Commented Dec 8, 2016 at 19:30
  • 1
    I agree. Do not create a sitemap for pages that should not exist! Remove the HTTP sitemap. As well, you want to check that Google as well as others are being properly redirected. One way to do this is to do a Fetch as Google one HTTP page that should be redirecting and watching the log file. It may take a few minutes to flush the log file cache, so you may not see your Google hit right away. Keep that in mind. That should work enough to know. You will see an entry in the log either way as a 202 or a 301 (generally speaking) after the cache flushes if not right away. Cheers!!
    – closetnoc
    Commented Dec 8, 2016 at 22:00

1 Answer 1

Reset to default
1

Review the whole codebase on existing of internally linked http urls. This is the most cause of remaining them indexed despite of redirect to https.

If http remains internally linked, Google means like "these redirects could be created erroneously - they are done with one code line in htaccess or similar, but if http remains internally linked, maybe they are truly correct and should remain indexed".

Beside of this, check whether your https certificate is correctly implemented (Chrome should fire errors if not)

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.