I have a domain, let's call it example.com, which has a subdomain, games.example.com. I maintain a games forum using phpbbseo which is located at games.example.com/forum. The problem is that the forum is not being crawled. I used Google's webmaster tools and tested that the page is seen by google.


There is a link from games.example.com to games.example.com/forum.

What can I do? How can I make google crawl my forum?


Sounds like your domain is in the sandbox, if so, you just have to wait until it gets accepted to the index.

I also recommend not using your subdomain, just use a subdirectory. This way your pages will be accepted and crawled directly and all these pages will increase your PR much more than a subdomain ever will. Subdomains are no longer a good SEO practice.

See also: Do subdomains help/hurt SEO?

  • Your comment about PR is incorrect. PR has nothing to do with subdomains or subdirectories. It is only a numerical representation of link popularity regardless of their origin. Subdomain, subdirectory, different website, etc.
    – John Conde
    Mar 15 '11 at 20:40
  • The Google algorithm is based on much more factors than counting links, for example domain age is one of them. In this instance that is relevant, because a subdomain is seen by Google as a separate domain. So using a subdomain is like creating a new website with no credibility build up at Google. So the subdomain receives little page rank. Even if you link back to your main website, they will contribute little PR value. Pages in a subdirectory on the other hand, will inherit the credibility of the main site and will directly contribute to the page rank of related pages on the domain.
    – Kdeveloper
    Mar 15 '11 at 21:18
  • I didn't say Google algorithm. I said PR. PR is a published formula based on only one thing: link popularity. It can't be debated how it works because the formula is public knowledge. As far as PR is concerned the location of the page does not matter at all.
    – John Conde
    Mar 15 '11 at 22:13
  • @John: The only relevant PR is the Google PR, that’s why most people refer to it as just PR. On the Wikipedia page you refer to the following can be read about it: "PageRank measures number of sites which link to particular page.[21] The PageRank of a particular page is roughly based upon the quantity of inbound links as well as the PageRank of the pages providing the links. Other factors are also part of the algorithm such as the size of a page, the number of changes and its up-to-dateness, the key texts in headlines and the words of hyperlinked anchor texts".
    – Kdeveloper
    Mar 16 '11 at 8:40
  • You're using the term PR incorrectly. PR stands for PageRank. If you mean "page rank", the ranking of pages, you can't call it PR as you confuse the subject. PR is Google only and no one uses it to refer to the ranking of pages by any search engine anywhere. It's important to get the terminology right or else misinformation results.
    – John Conde
    Mar 16 '11 at 11:57

You could try adding a sitemap to games.example.com that includes some of the main pages from the forum. Also refer to it in your robots.txt, like:


Also, Google doesn't have a problem with your subdomain, but because it's indexing games.example.com. It's either not seeing the forum or not wanting to index it for some reason.

Also, make sure the link to the forum doesn't have a rel="nofollow" on it.


Google will request the robots.txt file before trying to index any page within your site. And make sure you don't have a bit like "meta name="robots" content="noindex, nofollow"" else Google will ignore your page.

Google will index subdomains if you link to them from your main domain, but if your site is new, it will take time for Google to index everything.

  • robots.txt file is okay. some parts of games.example.com are being indexed, but games.example.com is specifically not being indexed. It's driving my crazy since there is a lind from games.example.com (which is indexed) to games.example.com/forum
    – fudge
    Mar 13 '11 at 17:05

Have you checked the site with a link checker? (xenu, a1 website analyzer or whatever. At least a1wa obeys nofollow, noindex and robots.txt - not 100% sure about xenu)

Not the answer you're looking for? Browse other questions tagged or ask your own question.