7

My WordPress site in cPanel is serving the admin-ajax.php file which is wasting CPU resources. How can I solve this problem? Should I use this in my robots.txt file or not?

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Improve this question
3
  • What is the reason your are considering crawlers to access admin-ajax? If that is only used by your site administrators, then you would disallow robot access to it. Even if you did that though, I don't see how it would help with CPU resources. Presumably it only does anything for a logged-in administrator.
    – Stephen Ostermiller ♦
    Commented Oct 18, 2017 at 17:06
  • If it is being used by something other than an admin, it is possible that hackers are targeting it. If that is the case robots.txt isn't going to help at all. Robots.txt is like a sign that says "please", but it isn't enforced at all and hackers will just ignore it.
    – Stephen Ostermiller ♦
    Commented Oct 18, 2017 at 17:20
  • 1
    @StephenOstermiller The reason for including admin-ajax.php is because that is how Wordpress' framework derived AJAX system works, even for the public side. So, if you code a theme or plugin using the Wordpress framework that uses AJAX, access to admin-ajax.php must be public. I think it's a stupid situation personally, but that's how it is. Also, I'm not a fan of outing what folders to disallow, especially since there would never be any links to that folder.
    – Steve
    Commented Dec 20, 2018 at 18:40

2 Answers 2

Reset to default
7

This file is often used front-end by themes and plugins. It was changed in WordPress robots.txt file a couple of years ago to allow crawling for this reason, after an issue with google being unable to crawl it.

The WP Core ticket thread can be found here and offers further information https://core.trac.wordpress.org/ticket/33156

Improve this answer
0

If by “allow” you mean to allow Google to crawl it, yes. Google wants to see how your site is constructed including CSS, JS, and Ajax files.

Improve this answer
1
  • I wouldn't think that /wp-admin/ URLs would be used for CSS, JS, or AJAX. The admin files are usually used by the website owner to write new content, approve contents, and such. It sounds like this URL might be an exception to that. If that is the case, it would probably be worth explaining that in your answer.
    – Stephen Ostermiller ♦
    Commented Mar 1, 2020 at 11:41

Not the answer you're looking for? Browse other questions tagged or ask your own question.