Efficiency of generating a list of pages on the fly

Question

I am currently working on improving the extensibility of my website.

One of the things I am working on is automatically generating a list of pages for the navbar, sidebar, index, etc, rather than doing it mostly manually like I do currently.

I plan to use a master JSON file that contains a list of pages, titles, descriptions, URLs, etc, and then parse this file with PHP when the page is loaded in order to display the list of pages. The particular file format and language used is not applicable this question, it's just here for context.

The obvious problem with this is efficiency - every single page load is going to involve reading that JSON file and generating the HTML for the list of pages.

I know that caching is the usual solution to this, so that the page generation script is only run when needed, or the database may cache common queries, however what are the implications of not using any caching solution other than potentially lowered performance, DoS risk, etc?

Is not using a caching solution standard practise and the lowered efficiency is just accepted, or am I missing something when it comes to how pretty much every website in the world displays dynamic content?

Answer 1

The obvious problem with this is efficiency - every single page load is going to involve reading that JSON file and generating the HTML for the list of pages.

The question is rather vague, but if we are talking about parsing a static JSON file to output HTML, then unless the source file is enormous, this should be near "instantaneous" - at least "insignificant" in the overall process of getting a complete page, with all the connected resources, rendered on the user-agent.

In this instance, whilst caching (the rendered HTML) could be quicker, whether this makes a real-world difference is another matter. (The very act of maintaining that cache is not without its own overhead.)

what are the implications of not using any caching solution other than potentially lowered performance, DoS risk, etc?

If the lack of caching is slowing the site then, like you say...

• Increased server load
• Unable to handle as many users
• Frustrated users
• Drop in SEO (if your site is very slow)

You mention "DoS"... whilst yes, you'll potentially be unable to handle as many users. But simple caching is unlikely to help you against a real DOS/DDOS attack.

Is not using a caching solution standard practise and the lowered efficiency is just accepted, or am I missing something when it comes to how pretty much every website in the world displays dynamic content?

Caching takes many forms. Most sites will be using caching of one form or another, even if they don't realise it. However, I would say caching is more standard than not. Employ additional caching if you need it, don't if you... don't!

But many sites that depend on caching are doing considerably more than parsing a static JSON file.

---

Just a thought, since you mention JSON... don't do any processing server-side. Send the JSON to the client and let the client build the HTML instead? (Yes, that creates a JavaScript dependency - but this could be checked on first load and fallback to a server-side rendering if necessary.)