1

My site's robots.txt was blocking everyone for a year (The "Discourage search engines from indexing this site" option was checked in WordPress).

I guess Google will re-try to crawl it in the upcoming days (there is at least one site linking to it right now)? Or should I re-submit the site to Google? Here is what I had (and just removed today) in my robots.txt:

User-agent: *
Disallow: /

Thanks in advance.

  • 1
    Oooooooooooooooops! It'll just take time ;) – Simon Hayter Jul 2 '14 at 9:29
2

Google will fetch your updated robots.txt within the next few days and update their crawlers accordingly. Keep in mind this doesn't mean they will start crawling your site immediately and doesn't mean your pages will be indexed soon or rank well. It just means will know your pages are available for crawling but that is the first step to getting your pages into their index.

  • ...and submit a sitemap in GWT. – MrWhite Jul 2 '14 at 12:03
  • Start adding new fresh content and update your site as often as possible with new content. Show google you're making updates often. The more frequent the better, try and get some good natural links back to your site and that should all help along with the above. – Anagio Jul 2 '14 at 14:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.