So here is the use case:
Our company has customer data files stored in S3. Their website is hosted with a third party hosting company (Lets call it Hosting X). They are planning to provide a new offering where the customer can select a cloud space he has account with like dropbox, skydrive etc.Then the company will transfer the customer data to that cloud space account using the cloud space's API calls.
We have done cloud uploads in the past, but the data servers were ours, so we just had a bunch of scripts running in the data servers. The scripts would fetch the data files locally and upload them to the desired cloud space.
In the current scenario I am thinking the following are two possible options:
Option 1. Download data files from S3 to Hosting X. Then run script in Hosting X which will upload the local files to desired cloud space.
Option 2. Host the upload script in EC2. When customer requests a data transfer, a script from Hosting X will hit the upload script in EC2. The upload script in EC2 will fetch the files from S3 and upload to the cloud space.
I believe Option 1. will have a recurring cost associated since every data transfer from S3 to outside AMZ infrastructure costs money.
In any case I am not sure if these solutions are possible. Can EC2 and S3 communicate so? Will EC2 need to download the data files from S3 or can it access them as if they are "virtually local"?
Anyone has any experience with such a scenario? Any suggestions, tips are GREATLY appreciated!!