1

I have a list of about one million URLs that I would like fetched and would rather not run this process on my own web server. I am wondering if there are any commercial solutions to this - i.e. any services that will take my list of URLs and return the HTML for each (running it behind proxies, etc., if necessary).

1

A Linux virtual private server (some priced at less than $10/mo) can be used to complete this task with minimal configuration - see the wget man page for instructions if you choose to go this route.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.