New answers tagged

3

From the Google Search Console help docs: If you want Google to exclude the image-name-here.png that appears on your site at www.example.com/images/image-name-here.png, add the following to your robots.txt file: User-agent: Googlebot-Image Disallow: /images/image-name-here.png The next time Google crawls your site, it will see this directive ...


3

It is already answered in Stackoverflow, you just need to define the useragent name there. Here we only specify Googlebot, so that rule is not applied to other search engines. user-agent: Googlebot Allow: /$ Disallow: / As Google said (at the end of page). Use $ when you want to Match URLs that end with a specific string. Here /$ will say allowed to index ...


Top 50 recent answers are included