When I first submitted my website to Google my website's robots.txt had disallowed the search engines from fetching the URL and so whenever I enter my websites domain to check on google the description shows as "A description for this result is not available because of this site's robots.txt". But now I modified my robots.txt file to allow search engines to crawl the website. How can I force google to re-crawl my website's description?

  • 1
    Possible duplicate of Need Google to recrawl the pages
    – MrWhite
    Dec 23, 2016 at 20:00
  • It will happen in time. However, if your site is not too large, you can always do a Fetch as Google and Submit to Index in Google Search Console. It will take a couple of days to update - it is not immediate. You can do quite a few pages each day. I do not suggest submitting all of your pages unless your site is small, however, I do suggest you submit your most important pages plus a few. Cheers!!
    – closetnoc
    Dec 23, 2016 at 22:10

1 Answer 1


Use the Submit to Index function of the Fetch as Google tool to re-crawl your URLs

Ask Google to crawl and index your URL

Click Submit to Index, shown next the status of a recent, successful fetch in the Fetches Table.

Select Crawl only this URL to submit one individual URL to the Google for re-crawling. You can submit up to 500 individual URLs in this way within a 30 day period.

Select Crawl this URL and its direct links to submit the URL as well as all the other pages that URL links to for re-crawling. You can submit up to 10 of requests of this kind within a 30 day period.

Click Submit to let Google know that your request is ready to be processed.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.