I have a single page app that I want google to index. However, I can see from the search console that Google also crawls /api/* urls. Search console does not tell wether google bot calls them directly or trough the web app. Should I disallow /api/* urls in robots.txt or let them be? Does disallowing prevent Google from calling then trough the SPA as well? I want google to call them like regular user would, but prevent from crawling and especially indexing them individually.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.