How much time for robots.txt to remove my website from google

Question

I created a robots.txt file in my root but the website is still indexed by Google. How much time for robots.txt to remove my website from Google?

Answer 1

Just use the URL removal tool found in Google webmaster tools to speed things up (after blocking with robots.txt)- this should take no longer then 24 hours (just enter / to remove an entire site)

https://www.google.com/webmasters/tools/url-removal

Your robots file should obviously contain one line telling all robots to not crawl your site:

User-agent: * Disallow: /

Answer 2

A robots.txt can disallow crawling, not indexing.

The next time Google tries to crawl your pages, it will probably check your robots.txt and notice that they are no longer allowed to crawl. This would stop Google from visiting your pages, but they don’t necessarily remove these pages from their index (nor does it mean that new pages won’t be indexed; they could find links to these pages somewhere else). Your pages could still be listed in their index (but without taking the title or the snippet from your page).

If you want to stop indexing, you’d need to use the meta-robots element or the X-Robots-Tag HTTP header. In that case, you’d have to allow crawling of these pages in robots.txt, otherwise Google would never be able to learn that you don’t allow indexing.

Answer 3

It depends. All robots.txt does is tell a crawler what content you want—or don’t want—indexed. But that just communicates to the crawler. How the owner of the crawler handles the indexing—and cleanup—of the content depends on their internal processes. Meaning, they can take as long as they want to handle that aspect of the process.

In practical terms I have seen results change as quickly as a few days to one full month. This unpredictability is why the whole concept of SEO (Search Engine Optimization) is based on web site owners/managers taking an “ounce of prevention versus pound of cure” approach to website management. Meaning the only thing you can control is your code. The rest of it is out of your hands so you need to present your pages as cleanly as possible.