My WordPress site in cPanel is serving the
admin-ajax.php file which is wasting CPU resources. How can I solve this problem? Should I use this in my robots.txt file or not?
User-agent: * Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php
Webmasters Stack Exchange is a question and answer site for pro webmasters. It only takes a minute to sign up.Sign up to join this community
This file is often used front-end by themes and plugins. It was changed in WordPress robots.txt file a couple of years ago to allow crawling for this reason, after an issue with google being unable to crawl it.
The WP Core ticket thread can be found here and offers further information https://core.trac.wordpress.org/ticket/33156