Tell me more ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.
up vote 1 down vote favorite

recently it takes 5 to 8 days for my site (www.StudyHorror.com) to be indexed. I'm updating the website everyday.

What are the possible causes for this problem?

share|improve this question
For Example, this url link hasn't been indexed on Google for 6 days. Any ideas? – studyhorror Aug 15 '11 at 1:23

3 Answers

up vote 1 down vote

I would like to add into your comment the option to adjust google crawl rate.

I think it's just a suggestion to Goolge how frequently we are confident for Google to update the content. If it's too high, it can consume heaps of our bandwidth.

Therefore, it's not possible to adjust Google crawl rate by setting that up. :)

share|improve this answer
up vote 0 down vote

If your site crawl slowly and you want fast crawling then you have to update your site's crawl rate in google webmaster tool.

here it is:

To change the crawl rate:

  1. On the Webmaster Tools Home page, click the site you want.
  2. Under Site configuration, click Settings.
  3. In the Crawl rate section, select the option you want.
share|improve this answer
up vote 4 down vote

Here is your reason

http://www.studyhorror.com/robots.txt

User-agent: * Disallow:

Remove that !

share|improve this answer
That should definately do it. – chrisjlee Aug 15 '11 at 17:01
??? OMG! why? I thought every site must have one robots.txt. Can you please explain for me? – studyhorror Aug 16 '11 at 4:25
@studyhorror this robotstxt.org/robotstxt.html and this en.wikipedia.org/wiki/Robots_exclusion_standard should help you – HackToHell Aug 16 '11 at 13:29
@HackToHell, Chris J. Lee: Thanks you guys for your help! All the best to you! Kind Regards! – studyhorror Aug 17 '11 at 0:05
The robots.txt appears to be blank now but having Disallow: with nothing after it doesn't block anything, so this definitely isn't the problem. – DisgruntledGoat Oct 24 '11 at 18:35

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.