1

I have a list of about one million URLs that I would like fetched and would rather not run this process on my own web server. I am wondering if there are any commercial solutions to this - i.e. any services that will take my list of URLs and return the HTML for each (running it behind proxies, etc., if necessary).

Improve this question

1 Answer 1

Reset to default
1

A Linux virtual private server (some priced at less than $10/mo) can be used to complete this task with minimal configuration - see the wget man page for instructions if you choose to go this route.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.