How can I stop Google from indexing example.azureedge.net
content? We have the website from last two years but just recently we found out that Google has started to index all the URLs on the azure CDN subdomain website, and most of them have already started ranking. We have been using this subdomain CDN for all our assets. So not sure what caused Google to start indexing all the main URLs all of a sudden.
I have amended robots.txt
file, and has blocked Google from crawling it but somehow it is still crawling them. If someone know any other way to direct Google straight to custom domain and if anything needed to be changed at Azure dashboard.