1

In this old video Matt Cutts says that if you blocked the website in the robots.txt and want completely remove the website from search results, use removal tool.

How can I do actually this: just time in removal urls field:
www.example.com

  • I think Matt was a bit overly optimistic on the remove URL. It has been my experience that the remove URL can take months while using the robots.txt often takes 24 hours to just a few days depending upon how often Google visits your site. I would stick with the robots.txt method. – closetnoc May 30 '14 at 15:56
  • Thanks for feedback!!! So, you're saying that after blocking via robots.txt, the page eventually has gone from SERPs? I saw many examples when pages are blocked in robots.txt, but still appear in SERP for several months. So it looks like not enough for Google, and it want more for example using removal tool – Vasyl Nekohda May 30 '14 at 16:06
  • I cannot speak to everyone's experience, but I used the removal tool and waited almost three months before using the robots.txt file and the page was removed from the SERPs within less than a week. I have also read that the remove URL tool has been lagging and others here have had better success using robots.txt. It all depends upon how often Google visits the site. If something hung around in the SERPs for a period, I suspect these were sites not visited by Google often. Google checks my robots.txt file every day but then again, my site is large enough and changes very often. – closetnoc May 30 '14 at 16:15
2

Assuming your site is verified in webmaster tools. navigate to Google Index > remove URLS

Click > Create a new removal request. Enter nothing and press continue. That will be accepted as a site removal.

Ensure in the 'pending removals' it reads 'Site Removal' and the site will be removed from the index within a day or so, no longer.

It might get reindexed after 90 days, if you dont want that to happen, block Googlebot from the site.

  • OK. Counter-intuitively, leaving the box empty does seem to be processed differently from typing the home page URL, even though both appear as the same URL in the list. Even with it recognising the robots.txt exclusion and password protection it still shows as "Pending" after several hours though and the cache is still there. Does it have to try fetching every page in order to remove them from the cache despite the robots.txt? – Cedric Knight Jun 12 '15 at 21:28
  • No, it will be processed soon enough and fetching wont help. Look in search results to see if removed, as that will happen first and then the pending notice updated later. Shouldnt be more than a day or so. – user29671 Jun 13 '15 at 8:14
  • OK, took 6-7 hours to stop crawling and remove cache, similar to Bing. I note that www.google.com/webmasters/tools/removals shows the request as "Outdated page removal" implying it's just removing the home page, whereas tools/dashboard or tools/url-removal shows it correctly as "Site removal". Probably a Google bug. The Google help pages don't really document what to expect. I presume the robots.txt is enough to stop it trying to crawl the protected pages indefinitely. – Cedric Knight Jun 14 '15 at 13:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.