2
votes
2answers
88 views

Robots.txt Blocking Blog Tags?

Tags are causing duplicate content on my website and I want to make sure that I block them from being indexed. Is this the right syntax to use for blocking those Tags from being crawled? User-agent: ...
4
votes
2answers
548 views

Should I prevent search engines indexing tag/category pages?

On my site, I currently have no special rules for search engines. It is a blog, statically generated using a Python program. When I search for some of my articles on Google, there is usually a tag or ...