This is simple question but also I have a confusion.
Below is the code which I wrote in my robots.txt:
User-Agent: * Disallow: /phpmyadmin/ Disallow: /login/ Disallow: /newsletter/ Disallow: /articlesubmission/ Allow: / Sitemap: http://www.example.com/sitemap.xml
I want all bots to follow my site but they should not follow these URLs:
http://www.example.com/phpmyadmin/..... http://www.example.com/login/..... http://www.example.com/newsletter/..... http://www.example.com/articlesubmission/.....
My doubt: Is my above robots.txt code is correct?
I followed this code from StackOverflow robots.txt.