Best approach to have Google index content primarily generated by Javascript

Question

I'm developing an app where customers can drop in a Javascript tag onto their website, and it will generate content. What is the best approach to have Google index this content and associate it with the site's URL.

I know that Google doesn't execute JS, so I'm thinking of different angles. I could have Google index the content on the site where the content is comping from (not the customer's site) but then it obviously will be indexed with the wrong URL. Could I somehow get Google to display a different URL? Is there another approach to get Google to index content from another site but associate it with

The constraints are:

• The customer website can only drop in one section of code (ideally a JS tag).
• I have full control of the original content, albeit on a different site.
• The customer website can't do any server-side processing to specifically solve this.

Ideally, the web visitors would see the JS generated content when viewing the site, but Google bot would get fed content from another web page (which is the same content, but from the raw source). I just don't know how to do this without JS.

Answer 1

As of 2015, Googlebot is rendering pages including running JavaScript. It is then indexing all the content it sees: static and generated. To make sure that your JavaScript generated content is seen by Googlebot:

• Ensure that your CSS and JS files are not blocked by robots.txt
• Load all content for the page with the onload event. Googlebot won't see any content that loads only when a user clicks or scrolls. Googlebot doesn't simulate user actions in JavaScript. It only loads the page, runs the JavaScript, then examines the document object model (DOM).
• Use anchor tags with links (<a href="...) rather than relying solely for onclick for navigation. Again, Googlebot doesn't try to click anything to see if it causes navigation. Googlebot will pick links out the rendered page, even if users don't load new pages because an onclick handler loads the new content without fully loading a new page.
• Use the "Fetch and Render" feature of Fetch as Google in Google Webmaster Tools. Make sure that Googlebot can see the JavaScript generated content of your pages.

Googlebot has gotten so good at seeing JavaScript generated content, that I've had to ask how to prevent it.

Answer 2

Your "tag" can be a combination of an A element and snippet of javascript. The href of the A element points to a crawlable page with your content, intended to be seen by search engines and those without javascript. For users with javascript, the A element gets replaced with the dynamic content (it can parse the href to determine which content to serve, to simplify things for those who want to drop it into their site). Users without javascript (and therefore search engines) will see a link to the content.

Also, on the content page, you can use javascript to send users back to the dynamic page. That way, if someone arrives via search engine, they'll go to the page that linked them there.

It's challenging to set it up so it doesn't make it complex for people to drop your dynamic content into their site, but shouldn't be that hard.

Answer 3

From what I understand you are serving up content from your site to client sites, and you want the client sites to get the rep for the content?

Google does now parse some JavaScript, though not all of it and not consistently. Your best bit would be to follow the link @Shaz provided http://code.google.com/web/ajaxcrawling and work with your clients to implement that.

Unfortunately, that does place some restrictions on your clients' URL structures but I don't know of any other way to do this.