We use AJAX to include some dynamic, context based content into a normal HTML page. The answer of this HTML call is an HTML segment without HTML headers.

The problem is that Google is indexing these pages.

We have tried using robots.txt to avoid indexation, but Google complains in (GSC) because it is actually some content (not too interesting though), and still indexes and gives these URLs in results.

We have tried adding a "noindex" HTTP response header to the AJAX page, but that lead to unindexing the main HTML page, as it inherited the HTTP X-Robots-Tag.

So, what is the proper way to doing this?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.