Can user-agents be listed together, followed by their common rules like so in a robots.txt?

User-agent: Googlebot
User-agent: ia_archiver #Alexa
User-agent: BingPreview #Microsoft
User-agent: bingbot #Microsoft
User-agent: MSNBot #Microsoft
User-agent: Slurp #Yahoo
User-agent: Ask Jeeves/Teoma #Ask
Disallow: /adm30_buds/

1 Answer 1


Yes. Assuming that your agent names are specified correctly, it looks like this should work. Here is a resource if you want to read more. https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt

  • 2
    The original robots.txt spec agrees: "If more than one User-agent field is present the record describes an identical access policy for more than one robot." Mar 18, 2014 at 15:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.