My site is a single page application built using AngularJS. My aim is to make it crawl-able (https://developers.google.com/webmasters/ajax-crawling/docs/specification).
This is what I have been using:-
- Pages without hash fragments
- AngularJS configured to use HTML5 URLs
<meta name="fragment" content="!" />
inserted to the head
As mentioned in the specification, using the above method, crawler is able to read my URL,
www.domainname.com
as www.domainname.com?_escaped_fragment_=
My URL looks like this:
http;//www.domainname.com/about
Forwarded the request if request include _escaped_fragment_
to my htmlfiles directory
RewriteEngine On
Options +FollowSymLinks
RewriteCond %{QUERY_STRING} ^_escaped_fragment_=$
RewriteRule ^(.*)$ /htmlfiles/index.html? [NC,L]
RewriteCond %{QUERY_STRING} ^_escaped_fragment_=/?(.*)$
RewriteRule ^(.*)$ /htmlfiles/%1.html? [NC,L]
# Redirect Trailing Slashes...
RewriteRule ^(.*)/$ /$1 [L,R=301]
# Handle Front Controller...
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^ index.php [L]
With the above rewrites desired html files were able to be obtained as follows:-
www.domainname.com?_escaped_fragment_=
receives www.domainname.com/htmlfiles/index.html
and
www.domainname.com/?_escaped_fragment_=about
receives www.domainname.com/htmlfiles/about.html
Note: these tests were carried out by disabling javascript on browser
The sitemap submitted to Google is as shown:
<url>
<loc>http://www.domainname.com/</loc>
</url>
<url>
<loc>http://www.domainname.com/about</loc>
</url>
Problem
What Google did is that it only index my main page www.domainname.com
correctly.
However it does not index other pages, in this case, www.domainname.com/about
.