Google only index main page

Question

My site is a single page application built using AngularJS. My aim is to make it crawl-able (https://developers.google.com/webmasters/ajax-crawling/docs/specification).

This is what I have been using:-

• Pages without hash fragments
• AngularJS configured to use HTML5 URLs
• <meta name="fragment" content="!" /> inserted to the head

As mentioned in the specification, using the above method, crawler is able to read my URL,

www.domainname.com as www.domainname.com?_escaped_fragment_=

My URL looks like this:

http;//www.domainname.com/about

Forwarded the request if request include _escaped_fragment_ to my htmlfiles directory

RewriteEngine On

Options +FollowSymLinks

RewriteCond %{QUERY_STRING} ^_escaped_fragment_=$                              
RewriteRule ^(.*)$ /htmlfiles/index.html? [NC,L]  

RewriteCond %{QUERY_STRING} ^_escaped_fragment_=/?(.*)$
RewriteRule ^(.*)$ /htmlfiles/%1.html? [NC,L]

# Redirect Trailing Slashes...
RewriteRule ^(.*)/$ /$1 [L,R=301]

# Handle Front Controller...
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^ index.php [L]

With the above rewrites desired html files were able to be obtained as follows:-

www.domainname.com?_escaped_fragment_= receives www.domainname.com/htmlfiles/index.html

and

www.domainname.com/?_escaped_fragment_=about receives www.domainname.com/htmlfiles/about.html

Note: these tests were carried out by disabling javascript on browser

The sitemap submitted to Google is as shown:

<url>
<loc>http://www.domainname.com/</loc>
</url>
<url>
<loc>http://www.domainname.com/about</loc>
</url>

Problem

What Google did is that it only index my main page www.domainname.com correctly. However it does not index other pages, in this case, www.domainname.com/about.

Answer 1

It's impossible to tell without looking at your site, but I think you're using the AJAX crawling scheme incorrectly.

If you have http://example.com/about and add the fragment meta tag to that page (so when you curl http://example.com/about it returns the meta tag in the response), then the URL that would be fetched for indexing would be http://example.com/about?_escaped_fragment_= , not http://example.com/?_escaped_fragment_=about .

Answer 2

Consider using HTML5, CSS3, and pushstate. Here are a few resources and examples:

• http://html5.gingerhost.com/
• http://www.ocpsoft.org/java/seo-friendly-angularjs-with-html5-pushstate-rewrite-and-twelve-lines-of-code/