4

Let's say I've a page which is already indexed in Google Search. If I make a change to the content of that page, how would Google bot notice and updated it in the Search?

  • Does he compare the indexed and the updated versions and see a change?
  • Does he rely on the modification timestamp which is displayed in the page or in the sitemap file?
  • Something else?
  • 2
    Google used to use the HEAD request to check to see if a page has changed. However, that was when network bandwidth was more expensive. Today, Google simply requests the page and updates the cache. If the page has changed or not updates metrics that allows Google to better understand how often it should request any particular page and set a schedule for that page. This would be the page refresh rate. The more Google sees a page change, the shorter the refresh rate becomes or vice versa. – closetnoc Aug 9 '17 at 5:25
  • Googlebot recrawls every page on the internet periodically whether or not it has changed. It is unclear whether Googlebot stores old versions of pages or not. It really doesn't even need to know if it changed. It can just make sure that search index always contains the latest version that it crawled. – Stephen Ostermiller Aug 9 '17 at 9:08
6

Google uses an algorithm to index your page at calculated intervals. The more frequent your updates are - more often the bot will index your page.

  • The bot only needs to compare the html which is on average around 10kb. To do proper comparison the bot is quite likely to hash the content, so if you got extra space it will be accounted for.

  • I've never used any types of timestamps for the bot to recognise changes

  • You can manually request for re-index using Google Search Console which takes about a minute to take effect. (You will need to add the domain to your google account to manage it with google tools, just like Analytics)

Even if your site is not often busy with changes or stale all together, the bots will index it at least 3 times a week. Search Console also displays when google indexes your site. I got a year old site which is indexed few times a ~3 times a week, because it's not very resource intensive task for google.

  • I have seen many pages that Google doesn't index once in a week. please need to clear idea about it. – Mazharul Haq SEO Aug 9 '17 at 13:48
  • Google will crawl a page once a week once it gets enough PageRank. It would probably take PR 4 or 5 to get to once a week. Pages with less PageRank get crawled less often. For little to no PageRank it may be months between crawls. It may also depend on how many pages there are on your site and how frequently you update them. Google may crawl small sites more often and it may crawl frequently updated content more often. – Stephen Ostermiller Aug 9 '17 at 15:37
  • @StephenOstermiller Pagerank has nothing to do with how often a page is updated. Google uses something like a TTL (time to live) metric that is adjusted slowly over time either up or down as it discovers a page being updated or not. I had pages with absolutely no rank updated in the SERPs within 20 minutes routinely because the website was updated often and mildly active in the search results. All of this happened within about a month or so of being launched. – closetnoc Aug 9 '17 at 15:57
  • @closetnoc Googlebot certainly uses PageRank to determine how often to crawl pages. So much so that you can use the number of times a page has been crawled in a time period as a very accurate proxy measure for PageRank. When I was working with a PR9 home page, it got crawled every hour (even though it changed weekly). The complete back water pages got crawled every couple months. Lots of pages in between that got crawled at different frequencies that correlated perfectly with their toolbar PageRank. Google has taken away toolbar PageRank, but they still use the metric internally. – Stephen Ostermiller Aug 9 '17 at 16:01
  • Googlebot also has some other modes of crawling. It has a mode for new URL discovery where it will crawl up to thousands of new URLs even if none of them have PageRank. It also has a mode where it checks URLs that have been redirecting for years. I've seen it crawl thousands of redirecting URLs with no PageRank (and no inbound links) from our site. When it does that it crawls them in order of length: shorter URLs first. – Stephen Ostermiller Aug 9 '17 at 16:03
0

Update your website to google search console. Add your sitemap to google search console and put google analytics code to your web page could easy to understand. if you want to update your content update through google search console and this could be easy to find by google that content updating has been done.

  • How does this answer the question? – closetnoc Aug 9 '17 at 15:47
  • Alerting Google to changed content through search console is no very practical. You have to "fetch as Google" one page at a time. Google has other much better ways of figuring out when content changes on pages. – Stephen Ostermiller Aug 9 '17 at 16:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.