updating SQL tables every x seconds (the right way)

Question

I hope this is in the right topic area. Sorry if it isn't.

I want to ask the right way to do this task.

I have a few service API's I need to query every 30seconds or so that I would like to store in a mysql table then use this data on a web page.

Now I have done this using a python script to get all the data, format it and update the table and this works fine. Now at the moment I'm using a cron task to run the script every 30 seconds.

My question is: - is cron the right way to run this script in intervals or am I going against best practice? - Am I going about this whole thing completely the wrong way to get API data?

I was thinking about having the script trigger at the bottom of the HTML code, but if multiple people access the page it will trigger the script each time ending in tens if not 100's of python scripts querying the APIs

PS: I have done a search but couldn't see a post that answers this question.

Cheers in advance :-)

Answer 1

In this instance you have taken the option that is the best practice and is what is done by most websites that have to run tasks at regular intervals. What it boils down to is you are using the servers native task scheduling tool (cron) to run a regularly scheduled task, this is absolutely what you need to do. Running it every single time the page loads not only will cause a huge number of queries to be made to the API every single second, but will also slow down the page load as the user will have to wait while the server side code checks the API's for data and commits the data to the database. What you have done achieves everything you need, uses tools for what they where designed to do, and does not cause issues for your end users' experiences. Well done.