2

I have a site built in AngularJS. Most of the dynamically-loaded content comes from a Wordpress back-end that is separate from the AngularJS site. In fact, it's a completely different domain and we do not want the public going to any of its un-styled pages/posts/media/etc. Thus, in the Wordpress settings, we've checked the search engine visibility setting to "Discourage search engines from indexing this site"

Now, if I try to fetch and render one of the pages from our AngularJS site, Google Search Console is blocking/not rendering any of the content coming from that Wordpress site.

Can anyone help find a solution to getting Google to render this AJAX content? Once again, the Wordpress is only supposed to be a "back-end" and we DO NOT want people visiting any of its URLS. We just want to fetch the data from that back-end and render it on our AngularJS site.

Please let me know if you have any possible solutions, advice, etc.

Thank you so much!

Improve this question
1
  • It would be helpful if you edited the question to describe how the AngularJS site is fetching the data from the WordPress site...but it sounds like the Fetch as Google tool is trying to crawl the back-end WordPress site while rendering the front-end and seeing Disallow: / in the robots.txt file there, which disallows crawling everything, and is what the setting "Discourage search engines from indexing this site" will add to it. Again, it's hard to suggest something specific without knowing more, but modifying the robots.txt in your WordPress site to be more specific might help.
    – dan ♦
    Mar 22, 2017 at 4:22

2 Answers 2

Reset to default
1

You should not block resources that are used in creating/rendering the page you want to index to avoid being negative in google's compilation.

But if you are working on domain x and loading something from domain y that you don't want crawler to index. Then you should not block the resources from y, instead on pages from Y, you can add meta noindex, which will let crawler crawl this from domain x, but won't list it in its search page results and would be unreachable for users using organic channel.

This is what i think you want to achieve.

Improve this answer
0

Google needs to be able to fetch the content from the WordPress site to be able to render the pages from your Angular.js site. Checking the "Discourage search engines from indexing this site" disallows crawling in the site's robots.txt file. This prevents Googlebot from fetching the content, even when called from another domain.

Instead, you want to allow Googlebot to fetch the content but tell it about the preferred URL where it can be viewed. You can use a meta rel canonical tag on each page of the WordPress site to point Googlebot to the corresponding page on the Angular.js site. For example, wordpress.example.com/some-page might have the tag <link rel="canonical" href="http://angular.example.com/#some-page" />

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.