1

I have a website A whose content gets updated every minute(Its content is related to stock price). However, because displaying the content of website A requires expensive computation, I want to create a website B that shows a cached version of the website A.

Ideally, if cache is set to expire every 5 minutes, the following sequence of events must occur.

  1. User visits website B.
  2. Website B redirects the user to the website A.
  3. Website B stores the html file of the website A in server.
  4. If some other user visits a website B within 5 minutes, instead of redirecting the user to the website A, the website B should display the cached html file.
  5. After five minutes, cache expires, and if some user visits a website B, the website B should redirect the user to the website A.

I am completely new to nginx and server development. Currently I have created the website B, but the website B just redirects the user to the website A, and I do not know how to extract html file from the website A in nginx. I have implemented cache.conf in the website B, but the website B does not store any cache file because it cannot extract any html file from the website A.

Is there any good, simple, cost-friendly way of doing this? Thank you for your help.

6
  • 1
    What you are wanting does not sound logical - indeed it sounds complex and wasteful of bandwidth (Once visitor 1 has visited Website A, the data needs to be pushed to website B as well as provided to user 2...) . I think you probably want to set Website B up as a reverse proxy, and ensure Website A sets a 5 minute cache time for dynamic content.
    – davidgo
    Commented Jan 18, 2022 at 7:52
  • @davidgo I think they are using the term "redirect" loosely. It sounds like "proxy" would be what they actually want there. Am I correct John? Commented Jan 18, 2022 at 10:52
  • Is website A still going to be publicly available? Do you need an entirely separate website (website B)? Why not implement the server-side caching directly on website A?
    – MrWhite
    Commented Jan 18, 2022 at 18:10
  • @MrWhite My boss wants to create website B because he wants a separate website for just visualizing the data. Website A processes stock information and shows some strategies, but since the computation takes so long, he wants to create a separate website just for visualizing the data, which is the website B. Also, he will add more features to the server A, so he think it is a good idea to create a separate server B just for visualizing the data, which is responsible for the website B. I am not sure if my explanation makes sense. If you are confused let me know.
    – John Doyle
    Commented Jan 20, 2022 at 0:20
  • @davidgo I looked up proxy server in internet, and it seems to do the caching job well, but my boss still wants to create a separate new server just for visualizing the data, which ideally should cache the contents of the website A. I think my explanation above will help understand better.
    – John Doyle
    Commented Jan 20, 2022 at 0:24

1 Answer 1

1

Your initial requirement would be relatively easy to answer with a Content Delivery Network (CDN) or some sort of caching layer via a reverse proxy within your infrastructure (I'm just going to use "caching layer" from now on) that the user would visit that then sends requests to your web server (origin) to get the content.

  1. User visits a page on your public site www.example.com - this is served by the caching layer servers.
  2. If the caching layer has a valid cache of the requested page, it returns that to the user.
  3. If the caching layer doesn't have a valid cache of the page (too old, or not cached) the caching layer forwards the request to your web server, gets the response, caches it and returns it to the user.

However, going by the comment discussion your actual request is:

We'd like to continue serving basic stock price information from Website A, as this is known service, updated frequently and can be served cheaply.

We'd like to have a second service, Website B, that pulls some computationally expensive visualizations from Website A, caches that for a period of time and serves the cached data to visitors.

There are a few technical considerations within this, but none of them are insurmountable.

Typically, your users will be viewing "Website A". You'll need to then update links to the visualization content to point to the caching layer (in your question "Website B") - that would then work as above, but only for requests to visualization content. That content would need to have links back to Website for the "normal" content.

To make your life easier, you may want to try and have all the visualization content under a common folder, so that you can have some redirect rules set up on Website A that redirect "normal" requests to the caching layer, and similarly requests outside of that folder on the caching layer can be redirected back to Website A.

Your caching layer will typically be sending some additional headers on the forwarded requests, Website A will use those to know that the request is coming from the caching layer and allow the request through to the expensive page.

The caching layer can either use the cache headers that are returned (expires, cache-control) by the visualization pages or just apply it's own default caching for 200 status responses.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.