Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I know how to code. I know how to set up a news/RSS aggregator site. But I don't know if Google will crawl it. Will Google crawl such a site?

The site will have a listing of all the aggregated news with the title, a short snippet of the news, and an image (not all will have an image) of the original news article.

When a user clicks the article the user will get redirected to the source.

Something like drudgereport. I haven't seen a drudgereport link when I search so I can't tell. (Maybe because it has something to do with my location I don't know.)

Anyway any ideas?

share|improve this question
add comment

1 Answer

up vote 1 down vote accepted

Yes, Google will crawl the page. Unfortunately for most news stories you'll never have a chance of showing up, because the article's you link to will be ranked higher. If you Google "Drudge Report", it should be the first result - So you know it's being crawled and indexed. Even if you Google "CONDOLEEZZA RICE STAKES STAGE AT CONVENTION", you should get a result that is a sub-page on the same domain.

The problem is that the content on the page is so thin, and it's almost all outgoing links. Generally Google doesn't like it when a landing page just directs users to another website, they'd rather send the user directly to that website.

With the snippets that you add, that might make the content a bit thicker, but at the same time it will all be considered duplicate content because the originating sites will have it published first.

share|improve this answer
    
"If you Google "Drudge Report", it should be the first result - So you know it's being crawled and indexed" you do have a point haha. I'm just amazed of the stats that drudge displays (033,755,500 PAST 24 HOURS) (900,910,156 PAST 31 DAYS) (11,789,455,190 PAST YEAR) I would assume majority of that was from robots but when you look at his quantcast stats it still shows the same similar amount of visits. I'm wondering if its direct or search traffic. Anyway thanks for the tips! –  joespina Aug 30 '13 at 2:43
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.