Should I disallow the age gate page in robots.txt?
I need to make sure the crawlers can get through the gate.
This question came from our site for professional and enthusiast programmers.
Sign up using Google
Sign up using Facebook
Sign up using Stack Exchange
1 year ago