There are many reasons not to block your feed, but only you can know if they are relevant for you. For example:
- There may be bots that especially look for feeds, e.g., feed search engines.
- There may be bots that use feeds to discover new content.
- There may be other cases where bots would like to access your feeds, now and in the future.
- Some web search engines might index feeds resp. feed URLs, so that they can give it as a result if users search for
example.com feed
, site:example.com inurl:feed
, etc.
- Some user agents, e.g., feed readers, might follow rules in robots.txt.
I think most search engines will not be confused when they find a feed containing similar content to the front page of the website, as feeds are very common (almost every blog has them, news sites, forums, …). Make sure to link them with rel
-alternate
and give the corresponding MIME type in the type
attribute:
From the HTML5 spec:
If the alternate
keyword is used with the type
attribute set to the value application/rss+xml
or the value application/atom+xml
The keyword creates a hyperlink referencing a syndication feed (though not necessarily syndicating exactly the same content as the current page).
If your feeds contains the same content (i.e., the same number of posts and the same or less of the content) from a page of your site, you could use the canonical
link type as HTTP header:
Link: <http://example.com/>; rel="canonical"
But it should not be necessary.