I have a site that has around 12,000 pages. A couple of days ago I uploaded the site, I removed the
disallow option from my robots.txt, I run "fetch as Google" and I got the message:
URL and linked pages submitted to index.
The problem is that my site is not indexed yet although Google bot has started crawling it. When I search Google for my domain name I get:
A description for this result is not available because of this site's robots.txt
I have removed the
disallow option and in fact robots.txt is as follows for the last 2 days:
User-agent: Googlebot-Image Disallow: /
In fact, when I test the robots.txt file via the Webmaster Tools I get:
Detected as a directory; specific files may have different restrictions
Google is crawling it at a slow pace (~400 pages per day).
How can I can tell Google to index my site before finishing with the crawling process? If this is not possible, can I speed up the crawl?