The solution is uploading a separate robots.txt file for CloudFront URL. This will not block the CloudFront URL but will keep it away from google search
Create robots_cf.txt file that contains this text:
User-agent: *
Disallow: /
Now add this rule to .htaccess
file (say Cloudfront URL is aaabbbcccddd.cloudfront.net
):
RewriteCond %{HTTP_HOST} ^aaabbbcccddd.*$ [NC]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*robots\.txt.*\ HTTP/ [NC]
RewriteRule ^robots\.txt /robots_cf.txt [NC,L]
Eventually, Google will get this and stop crawling your CloudFront URL.
d1234.cloudfront.net
you are serving a canonical URL oncdn.example.com
? – jrtapsell Jul 9 '18 at 15:29