1
vote
1answer
236 views

Redirect Google crawler to different robots.txt via .htaccess

I have googled for the answer all day and still couldn't find an answer. I have a virtual subdomain www.static.example.com which is a mirror site of www.example.com. It means I have just one root ...
0
votes
1answer
196 views

Need help with mybb forum robots.txt file

I have installed the mybb forum in subdomain of my site. The link to my site is here. but when I search Google even the pages that are disallowed are also index. What's wrong with the robots.txt ...
1
vote
1answer
214 views

Search engines crawling non-existent subdomains

A few years ago, a previous developer added my company's website to a link farm he ran. My company has a wildcard subdomain, so any subdomains will return the user to the homepage. As such, the dev ...