Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

We recently updated the site to a single page site using jQuery to slide between "pages". So we now have only index.php.

When you search the company on engines such as Google, you get the site and a listing of its sub pages which now lead to outdated pages.

Our plan doesn't allow us to edit the .htaccess and the old pages are .html docs so I cannot use PHP redirects either.

So if I put in place a robots.txt telling the engines to not crawl beyond index.php, how effective will this be in preventing/removing crawled sub pages.

And rough guess, how long before the search engines would update?

share|improve this question
add comment

2 Answers

robot.txt is text file which just includes allow and disallow rules for any particular page of website. If you need to remove all pages from Search engine then you need to use Google web master tool. There is not particular time period need to remove website(updation process of search engine) from Google but approx 5 to 15 days needs to remove it from Google.

The important information about you website is that if you have removed website from Google then you can recover it in 30 days.

share|improve this answer
    
I definitely will do this, but doesn't Google Webmaster Tools only affect Google itself, or am I mistaken? –  Stefan Jul 10 '12 at 14:02
1  
Yes, Google Webmaster Tools only affects Google's index, not Bing, etc. –  mgkrebbs Jul 10 '12 at 21:16
add comment

Having a robots.txt to tell the search engine not to crawl certain pages should work but there are no guarantees because search engines do what they want. You could also try creating a removal request in Webmaster Tools. The delay for search engine updates vary from a day to months and this also depends on the search engine (I have heard Bing is the slowest).

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.