My website's main page has very little content on it until after a javascript function loads all the images to the page. The function is called on the window.onload event. My only problem is search engines are unable to index my site with all the content on it. It only stores a cache of the site with all the DOM elements, and none of the javascript created elements. How can I get search engines to index my pages with all the dynamically created elements along with it? Sites with less content make it harder to rank in results, and in return hurt site SEO.


Note: this answer is only about Google, I'm not aware of how others are handling this.

Google bot is now able to execute JS in crawled pages, by using a headless browser (fully automated browser instance). But since it takes more resources (hence, money) for them to run that browser, it will visit your pages less often than the regular crawler, which just parses the initial HTML.

In any case, if you want to have better SEO, several options are available, and it's up to you to decide which ones you are willing to take. Here are some:

Making your website work without JS

In most cases (and your website definitely falls into that category from what I can see), JS isn't actually required for filling your HTML with data. It's either for adding interactivity, or for getting data from an API, which could be done server-side. In this situation, it's always good to turn off JS in your browser, and see if everything still works. Content should already be on your page, even if it's not pretty, and it should work. JS should only be there to make everything better for the user. Links should work, forms should work. If you have buttons leading to other pages, they should be <a> links, with an actual URL, even if your application is a SPA.

This will help bots understand your page, and your site's architecture (i.e. sitemap). Not only, it will help people with impairment (blind, for example) who use a screen reader. And Google takes that into account, which is good.

Serving dynamic content

The first obvious option which comes to mind, is to process the data server-side, and render HTML pages on the fly. If you're only comfortable with JS, you could look into NodeJS, and create a server using a module like ExpressJS or HapiJS, for example. There are tons of tutorials you can find. Or, if you know other languages, you could do it in Python, PHP, Java...

Second option, if you really don't know how, or don't want to work server-side, you could look into static website generators, like Hugo. You write templates, define some data, and once you're done, generate HTML pages and host them on a static server. That does not prevent you from having Javascript in the front-end to enhance the page, by the way. But the content is already there.

Another solution, usually for more complex applications, is server-side rendering. You run the page on your server in a browser-like context (not really a browser, but JS runs in the context of your page), JS fills up the DOM, and the updated HTML is what is sent to the client. JS can then be used client-side to do more work. It's getting pretty popular with existing frameworks. For example, if you write an app using VueJS, you can use NuxtJS or Vue-server-renderer to pre-render the HTML in a state that is only obtained after running JS. If that's not something you want to look into, you can also use a paid service like SEO4Ajax, which will run your pages in a browser, and keep a cached version of the final HTML, for you to serve to bots like Google bot. We used that one at our company for a while, it worked nicely.

Leaving a good impression

Finally, if you want to be ranked higher, your site needs to be high quality. For example, right now, it does not support https. You're losing points here.

It does not have a lot of text, which makes it less valuable.

It does not have an <h1>, or really any page structure apart from divs.

It does not have Open Graph tags, which will help display nice thumbnails in Twitter, Facebook or other sharing services: enter image description here

It does not have structured data, to help crawlers understand what your page is about, and which could be used to display your website nicely in search results:

enter image description here

There are tons of other things to look after, but most importantly, know that crawlers become more and more clever. Hence, no matter what you do, if your website is low quality and does not have great content, people won't like it. And if they don't like it, there is a good chance Google won't either. Build your site with passion, look at what crawlers suggest for you to do to improve it (e.g. PageSpeedIndex, Google guidelines), and your website will eventually grow. It should be fast, it should be light. It should work nicely on mobile.

| improve this answer | |

You're probably best off generating the content on the server side before and include it in the response to the client instead of asking for it after the page loads. You can take a look at templating libraries for whatever server side framework you're using to generate the dynamic HTML.

If the content must be loaded on the client side, it is recommended that you keep your page load times below 4 seconds (source). You probably want to be faster than 4 seconds, though. Maybe include a timeout that can send some static content (maybe workouts cached on the server?) before the crawlers finish with your page. Another recommendation I have for this case is to create a static home page with these categories and workouts on a separate page. That way, crawlers can at least get a site description, a bit about the service you provide, site index, etc even if it won't find any categories or specific workouts.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.