...
- All data is stored on S3 as our hosting partner.
- All data will be served over SSL/HTTPS.
- We will have one Identity and Access Management (IAM) group for read-only access all datasets.
- We will generate and store IAM credentials for each user that signs any EULA for any dataset. The user will be added to the read-only access IAM group.
- We never give those IAM credentials out, we only use them to generate pre-signed S3 URLs with an expiry time of an hour or so.
- With these pre-signed URLs, users are able to download data directly from S3 using the Web UI, the R client, or even something simple like curl.
- Our Crowd groups are more granular and tell which which users are allowed to have pre-signed URLs for which datasets.
- The use of IAM allows us to merely track in the S3 access logs who has downloaded what.
- Users can download this files to EC2 hosts (no bandwidth charges for Sage) or to external locations (Sage pays bandwidth charges).
- For users who want to utilize Elastic MapReduce, which does not currently support IAM, we will add them to the Bucket Policy for the dataset bucket with read-only access.
Sage Employee Use Case
Download Use Case
EC2 Cloud Compute Use Case
Elastic MapReduce Use Case
Assumptions
Where
Assume that the initial cloud we target is AWS but we plan to support additional clouds in the future.
...
The Pacific Northwest Gigapop is the point of presence for the Internet2/Abilene network in the Pacific Northwest. The PNWGP is connected to the Abilene backbone via a 10 GbE link. In turn, the Abilene Seattle node is connected via OC-192 192 links to both Sunnyvale, California and Denver, Colorado.
PNWPG offers two types of Internet2/Abilene interconnects: Internet2/Abilene transit services and Internet2/Abilene peering at Pacific Wave International Peering Exchange. See Participant Services for more information.
...