0
votes
0answers
96 views

Disallow age gate/verification page in robots.txt?

Should I disallow the age gate page in robots.txt? I need to make sure the crawlers can get through the gate.
1
vote
2answers
159 views

Adult website crawlability issue

I got a mission and asking for some help with this scenario for a new website. The site is an adult escorts advertising service. It is developed fine, works and looks good, but the issue is with ...