You are conflating three things:
- what happens at the DNS level
- what happens at the TLS level
- what happens at the HTTP(S) level
In this case, the DNS is just there to map a name to an IP address. It can be a direct map, through an A or AAAA record, or some links of
CNAME records. But at the end of the day, when connecting to
test.example.com the browser (or any DNS client) gets an IP address.
Job of DNS system stops there.
Now the client starts a TLS connection to that IP address. By now all modern TLS clients use the SNI extension to add the name they want to connect to during the TLS handshake (remember, here we started with the IP only, as TLS is over TCP/IP, there is no more names involved). So whatever is listening at this IP address will be getting a "please open a TLS connection" from the client and getting the name the client wants to connect to.
The server can then react differently depending on his configuration and if it knows the name under which the client want to access it. SNI is typically used so that the server can handle back the proper certificate (that needs to have the name of the host in it, in the SAN extension typically) which the client will use to verify it does indeed trust this server (because the certificate will be checked as being having issued from a CA so the client is trusting). But the server can as well react as: the client is asking me about name X, and I know nothing about it, so I abort the TLS stream.
If the TLS part worked (both from server accepting under which name it was called and other parameters, and from the client having verified the certificate received from server), now it is time for HTTP. HTTP has headers, and one of them is
host, which is again the name the client wants to access (the hostname part of the URL). It will be in most cases what was in the SNI extension at the TLS handshake time as well (except in specific cases like domain fronting). Again at the HTTP level the server can then decide to handle the request, sending back a redirect, or just denying servicing it.
I get an error saying 'Connection not private'.
Look (and show to others if you want to get meaningful help you need to provide the exact data you are looking at) at the HTTP exchange. Use a command line client like
wget and see the HTTP exchanges. I suspect there is some redirections happening. Connection not private typically means the server had to revert back to HTTP (port 80) instead of being able to do HTTPS (port 443), which could happen as consequences of what I described above (if the TLS handshake or HTTPS exchanges were refused because the server got called with a name for which it is not configured to provide service).