How to serve a corresponding robots.txt file for each website in the same directory?

Question

I couldn't find the exact answer for this particular issue.

I have two add-on domains https://domain001.com and https://domain002.com hosted in /public_html/domain001/ and public_html/domain002/

A little background: I had to place robots.txt file for domain001.com in /public_html/ because for some reason I hadn't been able to verify the robots.txt file in google search console when I placed it in /public_html/domain001/.

Now domain002.com is using robots.txt file of domain001.com, although I have a separate robots.txt file for domain002.com in /public_html/domain002/.

As I've learned from similar topics here, I need to conditionally serve a different robots.txt file based on which domain has been accessed.

How can I do that? What code should I place in .htaccess?

I'm not a developer. So I would greatly appreciate if you provided more details in your answers.

Thank you for your time.

P.S. I'm using wordpress.

Answer 1

You need to test the domaine in your .htaccess to serve the right robots.txt

for example :

RewriteCond %{HTTP_HOST} ^(dev|test|www){1}(\.){1}(domain002\.){1}(com){1}$ [NC]
RewriteRule ^(robots\.txt)$ /robots-domain002.txt [L,R=301]

I use this for multiple instance in magento but it might solve your problem Of course, you must name in this case the second robots.txt with the name you provide in the rewrite rule