5

I've seen a few references to Google now crawling JS sites on this forum but nothing definitive. Nor have I seen any updates to the recommendations from Google. We can vouch for this. It's happening. The question is what does it mean? Is it time to get off the snapshots train?

Background: We have used Ember.js to generate our HTML for over a year. We create snapshots of our pages (pre-render) and serve them to Google using "_escaped_fragment_=" best practices. Sadly, our SEO has been rather poor.

I've been digging into this recently and discovered that Google is indexing the live, JS version of our pages (even though we have been pointing it at our snapshots). So, should we stop pointing Google at our snapshots and rely on their crawling?

Case: The "Donald Trump" page on Countable

Here's the current version of the Donald Trump Page on Countable - this is the version that Google is fetching properly (above) and has indexed:

enter image description here

Here's the outdated cached-snapshotted version of the Donald Trump page on Countable (note the numerous differences):

enter image description here

Search for the rich snippet from the current version of the page - "Donald Trump is a Republican candidate for President and business magnate, investor, and television personality"

Fetch As Google correctly Renders the current version of the Donald Trump page on Countable

Unfortunately, and this is really lame, the "fetching" tab shows our "noscript" HTML which is absolutely not what Google is fetching, rendering or indexing:

So, as a recap, do we ignore Google's advice (and the evidence of the Fetched HTML tab) in favor of the empirical evidence that Google is, in fact, successfully rendering our live JS?

Thanks in advance for your help.

||improve this question
  • 2
    Google announced they support processing JS so it probably is ok to do. – John Conde♦ Aug 11 '15 at 20:10
  • Hey, sorry to bother you 2 months later, but you could elaborate a little on what you ended up doing and what the SEO implications were? We're having a similar issue and I wanted to know if you have more empirical evidence now. – Benjamin Gruenbaum Oct 8 '15 at 11:32
2

Google has gotten really good at reading & processing JavaScript-based content for web-search. For the most part, if the files (JS, CSS, as well as any AJAX/JSON/JSONP responses) aren't blocked by robots.txt and can be crawled normally, we'll be able to render the pages like a browser would, and will use that for web-search. I suspect at some point we'll deprecate our recommendation to use the AJAX-crawling proposal (escaped-fragment/hash-bang-URLs), though we'll probably support crawling & indexing of that content for a longer time.

The "Fetch as Google" tool in Search Console has two modes - just the raw response (which will return the HTML of the page) or the rendered view (which will show a screenshot of the rendered version). Depending on what you're trying to diagnose, one or the other will make sense. At the moment, there's no HTML view of the rendered page (you could approximate this by using a Googlebot user-agent in your browser though).

||improve this answer
  • Thanks so much John. That's incredibly helpful. One other area where there's inconsistency is the Google's "Cached" search engine results. Can they render JS? I'm seeing mostly blank "cached" pages and when I click View Source it shows the non-JS default version of our pages. Here's one of our urls: countable.us/bills/hr4505-113-dod-cloud-security-act Here's the cached result (which appears blank): webcache.googleusercontent.com/… – bartolah Aug 25 '15 at 17:24
  • I've used the cached pages to verify JavaScript based markup so it can return rendered html. – Tony McCreath Aug 26 '15 at 2:40
  • The cached page is usually just the raw HTML response. If your JS can be run on the Google domain, and execute in the browser, that can happen (and it can appear to be on the cached page). However, the rendered content won't be displayed on the cached page directly. – John Mueller Aug 26 '15 at 21:30
  • You're right John (not surprisingly). I did a test quite a while back and my notes state the cache showed rendered code. But I can't repeat it. I better stop telling people to check it :-O – Tony McCreath Aug 29 '15 at 3:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.