How can a search engine crawl a dynamically generated website?

Question

A dynamically generated website doesn't have most of its content hard coded on the HTML file. Sometimes it waits for a user to login or wait for an input to generate its views. So it would look something like this.

 <section id = 'blog-content' class="section">
     <div class="columns is-centered">
         <?php echo $content ?>
     </div>
 </section>

In that example, the variable $content contains paragraphs and a title that someone wrote on a certain topic for their blog fetched from their database. How does a crawler know the content of that page if its content is dynamically generated? Since the content is not hard coded, the crawler wouldn't know the content exist because all it would see is the code i post above, right?

EDIT

To expand on what stephen said, the url would be something like this.

www.example.com/blog/$title

The $title variable has the same value as the one in the $content variable above.

Answer 1

Short answer: That PHP code is run on the server before sending the response to the crawler, so by the time the page reaches the crawler, all that info is already populated.

For sites written using server-side languages such as your example, here's the full lifecycle when a user visits a page:

1. The user's browser sends an HTTP request to the server for a certain path (such as /an/example/page/).

2. The server receives the request and determines the appropriate server-side code to run to generate the page. It executes this code, if any (or none if it's a static site).

3. The server sends the final generated, by that point static HTML page back to the user's browser.

Note that all the code is finished running on the server before the server actually sends any information back to the user's browser (or web crawler).

Things are a little different when the page is generated in part by client-side code (JavaScript) instead, which is a topic for a different discussion.

Regarding waiting for a user to log in or take action, generally search engine crawlers are cookieless and take no user actions, so anything hidden behind a login won't get crawled. Stuff hidden from crawlers behind logins like this is called the deep web, which is a cool term if you ask me.