1

We are using a cookie on our website which is to fetch some information through an API. The cookie becomes invalid after a very small duration.

Google Bot uses the same cookie again and again to crawl our website and we see numerous of exceptions and errors in the API server.

Please suggest me the way to handle the scenario gracefully.

Improve this question
2
  • Googlebot doesn't crawl using cookies at all as far as I know. I've also never seen Googlebot try to use an API. I'm surprised to hear this. Can you tell us more about this situation? How did you publish the API in a way that Googlebot finds it? Are there links to get data via the API? Is this part of some AJAX rendering for your page?
    – Stephen Ostermiller ♦
    Commented Nov 19, 2018 at 11:21
  • @StephenOstermiller Yes, we are rendering one section of the page as react component through AJAX call followed by an API. There are links to get via API. Can you please redirect me to a document which explains about Google-Bot not using cookie or API to crawl a page.
    – abhishek shrivastava
    Commented Nov 19, 2018 at 14:18

1 Answer 1

Reset to default
2

As far as i know Google is not using cookies or API when crawling a website, the issue should be something else check again your API errors or servers logs to determine the issue.

Google states:

HTTP cookies, local storage, session storage are cleared between page loads.
All features requiring user permissions (like Notifications API, clipboard, push, device-info) are disabled.

Read here more about Googlebot rendering

And a moz guide here

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.