Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I have a TB of compressed media files that I need to get to a remote web server with a bit of urgency. (from my office)

Our upload rate is only 500KB/s, which would suggest ~23 days to upload.

The new hosting company does not accept drives-by-mail.

Is there a good place to go in major metro areas to borrow a fatter pipe?

share|improve this question
8  
I know this isn't an answer, or even particularly helpful, but if your new hosting company is not willing to work with you on this you need to find a new new hosting company. Not accepting a drive from a customer (for a fee of course) is pretty sucky. –  voretaq7 Mar 15 '12 at 22:45
    
Check my answer it's a data storage company in Tulsa –  Anagio Mar 15 '12 at 22:53
    
It's got good answers here, but shouldn't this have been transfered to ServerFault? –  Mark Hurd Mar 16 '12 at 2:53
1  
Perhaps, but I view serverfault as a place to discuss the configuration of hardware and software, whereas this is the usage of whatever webmasters have at their disposal to achieve a more composite task –  George W Bush Mar 16 '12 at 3:36

10 Answers 10

up vote 15 down vote accepted

I would contact some of your local data-centers (rack-hosting, managed service providers, etc) and see about renting a vps or virtual machine for a day or two. Just be sure they allow you to bring in a disk for the initial transfer and they are aware of your bandwidth needs.

You might also just ask if they would simply help you with the upload.. They often tend to be opportunistic, inventive, and might be willing to accommodate you for the chance at getting some of your business.

share|improve this answer
3  
There's also a remote data-center option: Amazon AWS! Ship them hard drives, and they charge (as of this writing) $80 per device, plus $2.50 per data-loading-hour, plus whatever it costs to store the data as long as it's in AWS plus data transfer out. Possibly not the most attractive option, but just throwing it out there. –  fennec Mar 16 '12 at 1:10
    
This is great to know. We use AWS quite a lot! Do you know where I can read about this service? It's hard to find on google (at least the key words I'm using) –  George W Bush Mar 16 '12 at 1:45
1  
aws.amazon.com/importexport –  Rich Mar 16 '12 at 8:07

You could if you have friends or employees willing to lend you a hand with their home connection, set up a torrent for the large file and set up your employees/friends as seeds. Of course you'd have to cart and copy the file to everyone. Even then, you could split the file into .rars and have "teams" send portions (taking care to make sure they don't download the files they don't have). Crowd sourcing ftw!

share|improve this answer
3  
This would be particularly effective if you have friends who are in college. Not only do they have access to nice high speed university connections, but it wouldn't be hard for them to ask a couple of friends in different dorms to help with the upload. –  Lèse majesté Mar 16 '12 at 7:36

Find a regional ISP that's locally owned and bring a nice fat external drive that can hold all this. Walk into their office with $300 in cash, credentials for your server, and a 12-pack of beer. Pray that they have a gigabit uplink because a maxed-out 100mbit will take you over 24 hours.

That failing, look for local internet internet exchange point and find any AS that's connected to it. Make really good friends with somebody local from NANOG. Odds are you'll need to be creative. Try looking for any co-location near you.

share|improve this answer
2  
This is the path I would take. Beer tends to lubricate business interactions. –  Evik James Mar 15 '12 at 23:09
1  
I think this is quite possibly the best path-of-least-resistance solution here. –  jeremiahd Mar 19 '12 at 8:39

If the files are text files or otherwise cmpress well you can zip them up before uploading which could reduce the file size by up to 90%. Then you can decompress them on the server.

You can also try splitting the uploading between multiple locations. Each one then can give you their maximum speed.

share|improve this answer

Get a temporary hosting company you can mail your drive to, and then transfer from the temporary host to the permanent host at a faster rate.

share|improve this answer
3  
Or just switch to a web host that does accept drives by mail. Most web hosts with decent customer service will allow this. –  Lèse majesté Mar 15 '12 at 21:14
    
I assume the current hosting company was selected because of price or some other long-term criteria, and it would be more costly to change on a permanent basis. –  xpda Mar 17 '12 at 19:11

These guys are in Tulsa, which you have in your profile so if you're in the area call them or stop by and let them know your situation.

http://www.datastorageinc.com/

share|improve this answer

I just want to add to the existing answers something that might improve your transfer once you've got your hard drive connected to a server with good bandwidth.

I needed to transfer large amounts of data several times between two web servers and instead of using common FTP or WGET commands to upload or download the data, I have been using the Linux SCP command. You can find more info here: http://www.go2linux.org/scp-linux-command-line-copy-files-over-ssh

You will need SSH access to both servers and then you can copy one file, or many files recursively.

So, if you only have large files instead of many small ones, I think it will copy fast enough.

At a transfer rate of 11MB/second between servers the transfer would last about 25 hours.

Good luck! ;)

share|improve this answer
1  
Better yet, use rsync if the target host supports it. (Many do.) –  Ilmari Karonen Mar 15 '12 at 21:46

http://aws.amazon.com/importexport/

Ship your device to AWS and they will take care of the rest.

share|improve this answer
    
"...and they will take care of the rest"--no, they won't, not unless your destination is AWS. If it's not, then they will only transfer the data to an S3 bucket. And you're the one who has to get it from the S3 bucket to your storage destination, in which case it's the same as xpda's answer. –  Lèse majesté Mar 16 '12 at 8:56
2  
Oh absolutely, I should have commented as Rich did (Wasn't there by the time I had posted) on InChargeOfIt answer. Since Hamlin11 especially showed an interest in AWS solution and was looking for the link for this service I posted it. A comment over there would have been more appropriate I agree. –  subiet Mar 16 '12 at 9:05
    
Well, I think if you can edit the answer to tie in to that comment, and note that for non-AWS-destinations, you just need to initiate a server-to-server transfer afterwards using Amazon's very fat pipe, it'd still be a good answer. –  Lèse majesté Mar 16 '12 at 10:23

If the data is in your office on the slow connection you are in a pickle. If the data is on or at your old host you could try a server to server transfer. Using the ISP pipes instead of your tight office pipe.

If you did it in chunks via a wget from ssh it would be a hell of a lot faster than shipping a drive for install. That said, I have never had to move more than 250GB.

share|improve this answer
    
The urgency portion of the question came from a catastrophic downtime event at the "old host". We didn't have a warm-backup host, but we did have recent backups locally. –  George W Bush Mar 15 '12 at 23:15

As the other questions suggest, it mostly boils down to find someone around you with a good connection.

One group you could talk to are university or college admins. Just make sure, that the guy you're talking to is authorized to let you pipe some TB through their wire, and I'm sure you can get to an agreement.

Many faculties at universities, e.g. Particle Physics, post petabytes of data around, so this might be another possible option.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.