New answers tagged crawl-errors
Your PHP code isn't modifying the robots.txt file. You've merely commented out the link to the page. Try creating a robots.txt file in the main public directory of your website. Then add the following: User-Agent: * Disallow: /terms-and-conditions/ You could also add a noindex meta tag to the file in question: <meta name="robots" ...
Top 50 recent answers are included