1

Google is not crawling my website. I want to be in the search results as soon as possible. How do I get Google to crawl faster?

  • We don't have a duplicate here that I can find. Having answers on this site is good. StackExchange is reliable and the voting means that the better answers are easier to find. I wouldn't discourage asking about things here even if they are well covered elsewhere. – Stephen Ostermiller Feb 12 '15 at 11:13
  • @ StephenOstermiller This is a similar question that was marked as a duplicate: webmasters.stackexchange.com/questions/76993/… – closetnoc Feb 12 '15 at 15:28
  • The short answer is that you cannot speed up Google. Don't even try. It makes them mad. You do not want a mad Google do you? – closetnoc Feb 12 '15 at 15:29
2

Just wait.

You're not the only website on the internet. You're not more important than everybody else, just wait your turn.

You can help them though, by some methods:

  • Make sure your /sitemap is proper
  • Make sure your sitemap.xml is correct
  • Upload your sitemap via Google Webmaster Tools
  • Try to get backlinks to your site, they find you sooner
  • Get internal linking to standards so when they do visit, they find all pages
  • Keep your content up to date and fresh

These things take time, think among the lines of 1 to 2 weeks.

2

The answer is kinda simple: Freshen your content, get comments + discussions, get returning users on the site. Google can speed up the crawl rate and time to index significantly.

Look at popular forums, often it takes less than 10 minutes for a post to make it into index even when the forum has millions of posts. Popular blog or ecom is similar, I've seen content to index in less than 2 hours for some larger sites of that nature.

The logic is that if your site is full of static pages that never change, Google knows it would be a waste of time to hit for fresh indexes every hour. You may see them visit once a day in this case. Whereas if your site is full of commented on pages that are part of a more dynamic discussion (or content often updated), it will send out the bots far more frequently. It might even start to send more bots to help so that its always being crawled by some 66.249.xxx.xxx IP nonstop all day and night.

  • For the record, I posted Google IP address ranges for bot activity here: webmasters.stackexchange.com/questions/71344/… with some explanation. Also your comment on freshness is such an important factor. One up-vote! – closetnoc Feb 12 '15 at 17:17
  • @closetnoc Thanks again man for that list :) For the OP: Checking it out, one of our multi-installs of Opencart has 47 Gbot 66.249's hitting it today. 5-8 of them per site pretty consistently. Cool! – dhaupin Feb 12 '15 at 17:29
  • The first list of IP ranges seems to be the most recent Googlebot IP address ranges. I got this list from activity I have seen since 2008! I looked at the activities from each IP address and evaluated them through time to create the list. It may be that another persons experience may differ some. But it should be fairly accurate. – closetnoc Feb 12 '15 at 17:59
1

You could try to speed up indexation of your website by submitting a unique video on youtube. It should be related to your website and the main keywords you would like to target and, obviously, it should provide some value to the prospective viewers. YouTube videos can help you in your case because videos can go viral within a very short amount of time (provided that they are of good quality and provide value to your prospective visitors), and they help you enrich your website with something else than just text copy and pictures.

1

There are few steps through which you can increase the rate Google crawl:

  1. Update your website content on a regular basis
  2. Create a sitemap
  3. Do not use duplicate content
  4. Reduce your site loading time
  5. Prohibit access to unwanted page to Google via robots.txt
  6. Optimize and monitor Google crawl rate via Google webmaster tools
  7. Use pinging service
  8. Submit your site to online directories
  • Bhoomika, this answer is much better than many of your other answers, so I upvoted it. Please write more answers as long as this one. – Stephen Ostermiller Jul 2 '15 at 12:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.