Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

Possible Duplicate:
Why isn't my website in Google search results?

I have added my website to the webmaster tools of some major search engines already, and supplied the sitemap for the website.

I have been waiting for exactly 3 weeks now and still no data in the webmaster tools.

I have also did some link building, and the click-trough link to my website could be found in the search results from day one.

How long did it take for your website to get indexed? And why does is take so long?

share|improve this question

marked as duplicate by John Conde Jul 24 '12 at 16:18

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

    
3 weeks sounds unusually long... I don't have any concrete data on hand but I'm pretty sure Google (for instance) picks up on changes to my sites much quicker than that. –  David Z Jul 26 '10 at 1:18
    
Are you seeing any problems reported in webmaster tools? –  Tim Post Jul 26 '10 at 13:06
    
I did not see any any data in none of the webmaster tools, so no problems either. The funny thing is the day after I posted this question the website got indexed. Still not everything is indexed but there is some data available already, something like 70%. So for those of you who found 3 weeks unusual, now you know, it can happen. Weird coincidence! –  Saif Bechan Jul 26 '10 at 22:59
add comment

8 Answers 8

up vote 5 down vote accepted

There is no official rule as to how long it takes. I've heard anywhere from "instantly" to "a few months later". There are many variables such as how many others are linking to you (and how important they are), and how relevant is your content, and how often does your site get updated. Also, I tend to think some of the spiders kind of "do the rounds" and you may get lucky and be one of the early sites it indexes, or unlucky and not get indexed till the end.

Don't forget, the internets is a big place...

When I first launched my blog, the first post didn't get indexed for maybe a few weeks. Then once I started posting more often, my site got crawled more often. When I'm busy and don't post much, the crawling slows down as well.

share|improve this answer
    
The relevance issue you stated could be the case, because the content I have on the website is not that new to the web. It's all content that is out there already somewhere with slight variations. –  Saif Bechan Jul 26 '10 at 23:05
add comment

Put a googlewhack (statistically unlikely phrase like "jellified monkey shoes" or "insanitary curmudgeons") in your site somewhere and test if you have been indexed by searching for it.

Remember that Google have highly opaque logic about what search results they return, so you are playing a guessing game in trying to get into their search results.

share|improve this answer
add comment

I think about 48 to 72 hours for some sites - and probably all less than 1 week. I usually announce any new websites on a blog or friend's blog that I know is crawled and it doesn't take long.

Recently I had a problem where although my robots.txt file said:

User-agent: *
Allow: /

I didn't realise that I had a Wordpress plugin that was inserting a META tag on each page instructing robots to no-follow. The Google webmaster help forum was very helpful (and quick) to find my oversight I hadn't found in weeks.

share|improve this answer
add comment

This is unusually long.

For webmastertools, don't forget that you can just create a sitemap and upload it. Once you verify your ownership of the site, it will kind of force Google to check out your pages.

share|improve this answer
add comment

I recently uploaded a video on my college website, and it appears in Google's search engine on the second day of my post. It even appears in the first page of the search and the keyword used for searching is only 2 words long (I swear I am not lying here). I myself thought it was unbelievable and wondered how Google manage to crawl the web so fast.

share|improve this answer
1  
I've asked a question on stackoverflow.com, then had the question show up in Google's search results five minutes later. –  delete Jul 26 '10 at 11:03
1  
Very popular sites like Stackoverflow are crawled much more often than common-or-garden sites. Plus John Skeet personally adds each new question to the index - meta.stackexchange.com/questions/20171/… –  Dan Diplo Jul 26 '10 at 19:54
    
Yes I have notices that the questions pop up on google almost instantly. –  Saif Bechan Jul 26 '10 at 23:00
add comment

The data in the webmaster tools can lag quite a way behind the actual indexing. Have you actually tried searching Google to see if pages from your domain appear in it? If you prefix you type site:http://www.yourdomain.com (obviously replacing this with your real domain) into Google you should see all the pages Google has in its index for your domain. For example, to see all the pages from this site you would type site:http://webmasters.stackexchange.com

share|improve this answer
    
I did the check also every day or so, but there were no results. –  Saif Bechan Jul 26 '10 at 23:01
add comment

On the Google Webmaster dashboard, under "Sitemaps", you will see a list of submitted site maps and a count of how many pages are indexed from each.

If you don't see your site map there with a green check box next to it, or the number of pages indexed for that site map is zero, you should check the diagnostic tools.

I rarely see the process take longer than a week. Something is surely wrong and its very likely something that can be easily corrected.

Google is a good place to start. Usually, but not always, other search engines will be having the same difficulties that Google is reporting.

share|improve this answer
add comment

It's unclear whether by "indexed" you mean just any page of the site, or whether you mean the whole site. If you have quite a few pages, it will take a very long time for search engines to index the whole site, and they may never index everything (particularly Bing/Yahoo).

The other thing to remember is that search engines need to revisit the pages linking to you, to find those links. That can take over a week in itself if it's a static page that has not changed for months.

Make sure you have some links that are not "nofollow" - if the links have rel="nofollow" in the code then Google will probably ignore them. Most user-generated content (e.g. Twitter, this site) adds nofollow to the links.

Even with all this in place, it can still take a few weeks for pages from your site to be indexed, so just be patient! You can check how many pages are currently indexed in each search engine in their respective webmaster tools sections, or simply by searching for site:example.com.

share|improve this answer
add comment

Not the answer you're looking for? Browse other questions tagged or ask your own question.