Webmasters Stack Exchange is a question and answer site for pro webmasters. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I've got a pretty unusual problem. A client has their site hosted on a friend's server that is basically misconfigured to redirect http://clientname.com/ to http://clientname.com/clientname. After this 301 redirect, the site works fine and subpages can be accessed via http://clientname.com/clientname/subpage.

However, the whole site is not indexed on Google and my FTP root folder resembles http://clientname.com/clientname/ which means I cannot put a robots.txt file on the top level. Moving to a different hoster is out of question for the client.

What can I do? I'm desperate.

share|improve this question

migrated from serverfault.com Jul 23 '11 at 23:24

This question came from our site for system and network administrators.

up vote 0 down vote accepted

If the whole site is not indexed a robots.txt isn't going to help you as that is used to stop the search engines from indexing your content. Not help them do it.

Sign up for a Google Webmaster Tools account and add their site. Then submit an XML sitemap to Google. That will tell them where all of their pages are. Also, get some links to their pages from other sites. That helps the search engines finds them and links can increase the importance of those pages in Google's eyes giving them more incentive to index them.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.