I have a small website, for which the current robots.txt
looks like so:
User-agent: *
Disallow:
Sitemap: https://www.myawesomesite.com/sitemap.xml
Now, I want to add another rule like so (with the aim to disallow SomeStupidBot
for crawling my website):
User-agent: SomeStupidBot
Disallow: /
Sitemap: https://www.myawesomesite.com/sitemap.xml
I am unsure how I combine the above two rules
into one robots.txt
.