4

I am trying to get Google to index an AJAX site (davidelifestyle.com). It's crawlable with JavaScript turned off and I have also recently implemented _escaped_content_ snapshot mechanism but all that's indexed is a home page and PDF files that are not directly available from the home page. Also when I use Fetch as Google in Webmaster Tools, it downloads the dynamic page but does not index it ("Submit to Index" just reloads the page). Any ideas what might be wrong?

Edit: Today Index Status in Webmaster Tools showed: Total indexed: 0, Not selected: 178. According to documentation, pages are "not selected" because they are regarded duplicates.

2
  • Have you any noindex tag in your web pages. Please check the source page of your web pages. Thanks
    – Smart Jazz
    Commented Oct 6, 2012 at 13:05
  • No, and I added explicit <meta name="robots" content="index,follow" /> tag but it does not change anything as this is what Google assumes by default.
    – Meisner
    Commented Oct 6, 2012 at 16:34

2 Answers 2

2

One of the issues I see on your site is that on the homepage links look like href="/fashion". They must be href="/#!/fashion" in order for google to crawl through it properly. Google couldn't get from the homepage to any of the pages listed in sitemap.xml.

Also, on this page: www.davidelifestyle.com/?_escaped_fragment_=/fashion some of links look like this

<a href="//#!fashion/szyte-na-miare"> 

which is obviously wrong — should be /#!/fashion/szyte-na-miare

Also, make sure you submitted your sitemap.

Other than that, everything looks correct.

7
  • As for the second thing you described, in the HTML snapshot version almost all links were messed up - either in the way you explained or by not having a hashbang in them. Enough to possibly prevent Google from following all the links. Links on the homepage are OK though as the site is supposed to degrade gracefully with JS turned off. They are changed to their "#!" equivalents with JS and in the HTML snapshots. I corrected all the issues and am waiting for the next crawl but Fetch as Google on AJAX pages does not work yet.
    – Meisner
    Commented Oct 6, 2012 at 16:21
  • @Omeoe In order for Google to crawl AJAX urls and show them in the search results (especially /#!/fashion/ and /#!/interiors/ pages), I'm confident the links on the homepage should have #!.
    – Azmorf
    Commented Oct 6, 2012 at 17:54
  • If so, then I'll have to give up no-JS graceful degradation and I've been trying to avoid that. However, SEO is now a priority even at the expense of diminished compatibility and accessibility and I can easily change back and forth and observe what happens.
    – Meisner
    Commented Oct 6, 2012 at 18:43
  • Changed. Indexing in Fetch as Google still not working though.
    – Meisner
    Commented Oct 6, 2012 at 18:45
  • 1
    I have posted another question about possibility of combining crawlability and website's ability to degrade gracefully: webmasters.stackexchange.com/questions/35400/…
    – Meisner
    Commented Oct 6, 2012 at 19:00
0

Your issue stems from your script: <script type="text/javascript" src="/js/js.js"></script>

function loadPage(uri) {
....
    $.post(
        uri,
        //"/index2.php?view="+uri.replace(/^\//,''),
        {bodyOnly:true},
        function (html) {
            pageChanged=true;
            if (uri!="") {
                window.location="/#!"+uri; //<-- url rewrite here
                lastHash=location.hash;
            }
            $("#wrapper").replaceWith(html);
            onPageLoad();
            if (uri!="/interiors/dla-prasy" || musicOff) {
                replaceLinks();
            }
            if (uri=="/interiors/o-nas") {
                if (window.location.href.match('davidelifestyle.com')) {
                    changeImageDescription("Esther Verheij - owner");
                } else {
                    changeImageDescription("Esther Verheij - właścicielka");
                }
            }
            $('link[rel=canonical]').attr("href",uri==""?"/":"/#!"+uri);
        }
    );
}

I really don't see what purpose the javascript url rewrite to prepend /!#/ serves, but if google can see your content at both /fashion and /!#/fashion then its going to consider it duplicate. Also you should avoid javascript url redirects whenever possible as that can be a problem with google since many spammer/malware sites use js redirects

1
  • This is for progressive enhancement of the site to AJAX version. It is viewable with JS disabled so the originally retrieved links need to point to regular, static addresses. All the pages have rel=canonical tag referencing the #! address (which is the best practice according to offical FAQ: developers.google.com/webmasters/ajax-crawling/docs/… ), so this should have handled the duplicate content issue.
    – Meisner
    Commented Oct 10, 2012 at 16:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.