Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I have four websites that I am wanting to link to from one website. But the kicker here is that beyond just linking to the sites I was looking for a way to display updates from those sites on my site. I know this is possible through RSS syndication, but is this possible without RSS?

In other words as far as I can tell the four websites do not have an RSS feed. But I was still hoping to aggregate updates from those websites in one place.

UPDATE:

There are four websites that customers have to go to and manually check for updates. Rather than have my customers check each site manually by clicking on their links from my website I was hoping there was a way to build a widget or something that would display snippets of content from those four sites.

My website would need to display the latest content from the other sites. Sort of like Google Reader can do with RSS feeds. But those 4 websites do not support RSS.

share|improve this question
    
Do you mean you want to link to the articles/new content on their website? –  John Conde Sep 17 '10 at 19:26
    
@John, check update in original post. –  Webs Sep 17 '10 at 19:47
add comment

4 Answers

up vote 4 down vote accepted

I can think of 3 ways - 1) Google Spreadsheets' ImportHtml function can help you selectively scrape content from any website. You can then embed this spreadsheet data as a widget

2) Use YQL. Requires a little programming but the grunt work is simplified

3) Check this article on creating an RSS feed of any website without requiring software or having to write a single line of XML code. Using this method, you don't even have to be the owner of the site to create the feed.

share|improve this answer
    
On a related note.....Page2RSS.com provides a service that helps you monitor web sites that do not publish feeds. It will check any web page for updates and deliver them to your favorite RSS reader. –  mvark Sep 23 '10 at 2:46
    
I've looked in page2rss and it didn't seem to be able to do what I needed. I'm thinking option 2 or 3 is going to be my best bet. At the same time as suggested below I'm going to contact the site admins and see if they can work on creating an RSS feed. Hopefully one of the three works out. –  Webs Sep 27 '10 at 14:01
add comment

As mentioned above without RSS or any API you can tap into you will be forced to grab and parse those websites. You'll probably want to use a cron job to run your crawler on a predetermined basis. It should be no more more frequent then those sites are updated.

You should also verify that the other sites' owners do not object to have their websites scrapped. If nothing else it's good form to ask for permission to do so ahead of time.

See this answer for the beginnings of a PHP based solution.

share|improve this answer
    
Thanks for the answer John. I will look into something along those lines. –  Webs Sep 17 '10 at 20:29
add comment

if they don't support RSS, your best is probably writing a script that can parse the HTML from these pages, and get the relevant content.

share|improve this answer
    
Do you have any links or resources that could help me with this? I have HTML experience, but this seems to be more complex then what I have done in the past. –  Webs Sep 17 '10 at 20:14
add comment

You could also email those websites and ask them to publish RSS feeds.

share|improve this answer
    
I like the idea, but considering they are government websites that is going to be hard to do. Everything with the government takes awhile. But strangely enough I might be able to parse their code. –  Webs Sep 18 '10 at 2:13
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.