2

If Google picks up a page organically it means it has already found one link to a page. Fetch as Google is a great way to get your new pages indexed quickly. Is there a difference in terms of SEO?

Of course it's obvious that earlier indexing is good if you're dealing with bots or people who are copying your texts to other places. But this is not very common. In normal situations, could it be better to wait for Google to pick up pages organically or does it help to get your stuff indexed sooner?

I'd like to see some reputable sources here. A source at Google itself would be ideal but a quality blog post is fine as well.

1

In May, the Fetch as Google was updated. There are no details on the update other than adding the render feature. Prior to that, it was an older version of the googlebot moved outside of the queuing process. The differences were slight. It is good when you want a page indexed right away but not something I recommend unless it is important to update Google's index for expediency.

Historically, if you fetch a page using Fetch as Google, you would see that Googlebot will come along again in just a few days. This is because the Fetch as Google is intended to be just temporary and did not tie into the index the same way that the normal Googlebot did. It may do so now. I suspect the popularity of the tool had something to do with the update. There are advantages to this option such as getting your page into the index fast. Previously, the page will not fully perform as it normally would. The reason was that a certain amount of trust was given and the page could perform higher than normal- a pseudo boost in obvious search terms where as the page would actually perform differently overall if naturally discovered.

Some try and game Google with Fetch as Google. I do not suggest over using the tool for this reason. There are, however, times when getting a page in the index fast is a must. Then this option is perfect. Use it sparingly. But use it when it is important. Otherwise, I would just wait till the standard Googlebot comes along.

  • 3
    Is there some documentation about the fetch as Google being an outdated version of Google bot, and that a page wont perform the same after doing a fetch as Google, as it would if simply found by Google? – Max Oct 15 '14 at 11:08
  • @Max The Fetch as Google was updated in May with the Fetch and Render feature. Prior to that, it was simply a functional version of Googlebot that lived outside of the normal queuing process and was not updated for the simple reason- it was a tool. The differences were extremely slight at that point. Updating the answer. – closetnoc Oct 15 '14 at 15:37
  • 2
    It'd be great if you can add some links to sources for these statements. – kqw Oct 15 '14 at 19:51
  • @KasperSouren Not everything I know I read off of the Internet. Some of it comes from Google directly. Some of it comes with working on niche search engines. And some of it comes from experience. If there are references I can make them to back up my statements and often do. Google took some servers that were used in the Google fetch queue to build the Fetch as Google feature but did not update them to match googlebot as they aged. There was little difference between them anyway. Even then, the update in May may have only been to the website with no mention of the bot being updated. Who knows? – closetnoc Oct 15 '14 at 22:47
  • @KasperSouren I respect what you are saying. Thank You for the comment. One of the things I like is when people keep me honest. As part of the research group I am in, some operational, schema, and business rule details were shared with us and I evaluated these documents for the processes that we were concerned with such as developing trust networks, self-defending networks, and so on. In return, we shared developments back to Google. – closetnoc Oct 15 '14 at 22:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.