Impact on SEO for scraped news content

Question

I'm making a finance website and apart from our core features, I also want to show the latest news.

These news articles are scraped from RSS feeds (full-article scraping) of some reputed news sources (ex.CNN), but posts also have rel=canonical tags to their original source. All links in posts are redirecting to original source, but news will be mostly scraped word-for-word.

Is it going to negatively impact my SEO? Should I consider using nofollow in robots.txt? I just want to curate news articles for the ease of our users. I really don't want to get any SEO benefits from these articles as our core business is not showing the news.

Answer 1

rel=canonical is a good idea. That lets search engines know about the original source of the content and should prevent most penalties that could come from syndicating content.

robots.txt does not have any nofollow directive. You could disallow those URLs in robots.txt. Doing so would be wise as it would prevent Google from even seeing duplicated content on your site. Of course, if Google can't see the content, it also won't see the rel=canonical, but that would probably be fine.

Instead of robots.txt you could consider using a noindex meta tag in the page to prevent Google from indexing them. Google occasionally indexes URLs blocked by robots.txt without ever seeing the content on those pages. Allowing crawling, but using noindex is a sure fire way to prevent the URLs from ever appearing in Google's search index.

The biggest threat to your site from this is from a legal perspective. Make sure you get proper licenses from the content creators to syndicate their content. If you are scraping and publishing content without permission, the news sites will not be happy and will try to get your site shut down. They may file DMCA (Digital Millennium Copyright Act) takedowns with your host or with Google.