2

I've discovered that when I do Google Webmaster Tools --> Fetch as Google on a page on my site (let's call it http://example.com), it shows the static assets (js, css, and images) as Temporarily unreachable.

My static assets are behind an Amazon CloudFront CDN distribution, with the origin being my site, http://example.com. I have CNAMEd the CloudFront to assets.example.com.

That's exactly what is showing as "Temporarily unreachable" - all the css and js hosted at assets.example.com (for example assets.example.com/assets/application.css). The thing is, I can reach all those files just fine. I fear that googlebot is not seeing the js and css of my site, since when I do a Fetch and Render it says "This is how Googlebot saw the page:" and sure enough it's the page as it would look with no css or js.

I went through the responses on this somewhat similar question, but my CloudFront distribution doesn't even have a robots.txt that could be blocking anything, and the response headers when I do a curl -I on my css & js don't show anything alarming.

What could be causing googlebot to say that my static assets are Temporarily unreachable? Thanks!

Improve this question
2
  • Does your main site have a robots.txt file? The answer(s) on that other question seem to suggest that CF reads the robots.txt on your main site and serves that - although I think I may have misunderstood something??!
    – MrWhite
    Commented Apr 16, 2015 at 10:30
  • @w3d Yes definitely, main site has a normal robots.txt file. I have CloudFront set up such that it can only access anything in the assets/* directory, as to not duplicate all pages of my site with the assets.example.com URL.
    – DelPiero
    Commented Apr 16, 2015 at 14:36

1 Answer 1

Reset to default
2

Solved it!

My assets.example.com CloudFront distribution needed a robots.txt file after all. After adding one following these instructions, googlebot is able to fetch & render my static assets successfully.

It no longer reports them as Temporarily unreachable, and instead shows Complete.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.