My page has an ajax call that loads fresh html into the mother page.

Google has now spidered these ajax urls and runs it's search console algorythm over them as if it were a stand alone page.

In the live environment, the html returned from the ajax script picks up styles and viewport declarations from the motherpage and renders as expected.

When Googlebot takes a look and spiders the url as a stand alone page, the resulting html fails on mobile userbility (because the styles and viewport info is not available)

Should I prevent Googlebot from spidering the ajax urls using the robots file or is there a better solution to minimising Search Console errors ?

Hunting around for a solution / answer to this question has come up with nothing directly. The closest possible option is for the ajax rendered html to be sent via JSON - but I might have misunderstood the purpose of using JSON ?

  • First you need to decide if you want the ajax call executed by Google. Then you can stop it or tailor it to return something different. Commented Mar 2, 2023 at 8:39
  • Thanks Rohit - The content returned is good for indexing but only within the mother page. Do I really need to format the ajax return within it's own url ?
    – M Corby
    Commented Mar 2, 2023 at 9:06
  • I made an answer to clarify it. Commented Mar 2, 2023 at 10:22

1 Answer 1


Google only needs to see a sample of your page, not all variations of it.

Creating a large number of slightly different lists of hotels is redundant, because Googlebot needs to see only a small number of lists from which it can reach the page for each hotel. For example:

From Crawling-Indexing

You do this either by detecting the robot or using the canonical with params.

URL Parameters helps you control which URLs on your site should be crawled by Googlebot, depending on the parameters that appear in these URLs. This functionality provides a simple way to prevent crawling duplicate content on your site.

From Improved Handling of URLs

Use these parameters to tell your back end to neutralize the relevant ajax portion. This would only work if your page stands on its own without the additional information provided by the ajax calls.

You can neutralise it by

  1. Changing the params of the ajax calls embedded in the page so that they return blank perhaps.
  2. Omitting the ajax calls altogether in your page.

I would suggest option-11

  • Rohit - Again, thanks for your time on this. Have I understood your reply correctly by saying an option is to add a canonical header ? If so, this won't work becasue the mother page could be any number of urls. "Omitting the ajax calls altogether" - Without this call, the page won't update as items are added to a bespoke cart. Hope I'm on the right track ?!
    – M Corby
    Commented Mar 2, 2023 at 11:36
  • The canonical is to tell the bot, which version of the page you want to be indexed. You don't have to omit the ajax call - it's an option. It is up to you to decide how to handle the ajax call. If you already have params on the mother page to make into several pages, then you can add another param in the canonical url. Commented Mar 2, 2023 at 13:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.