Synapse provides physical storage for files using Amazon S3, however, you can configure your own custom storage locations as well. For example, data files can physically reside in your own S3 bucket, Google Cloud Storage Bucket, or a local file server using a proxy servers. Creating a custom storage location allows you greater ownership and control of your files, especially when you have a large amount of data or when additional restrictions need to be set on the data.
...
| Code Block |
|---|
{
"Statement": [
{
"Action": [ "s3:ListBucket", "s3:GetBucketLocation" ],
"Effect": "Allow",
"Resource": "arn:aws:s3:::thisisthenameofmybucket",
"Principal": { "AWS": "325565585839" }
},
{
"Action": [ "s3:*ObjectGetObject", "s3:PutObject", "s3:DeleteObject", "s3:AbortMultipartUpload" ],
"Effect": "Allow",
"Resource": "arn:aws:s3:::thisisthenameofmybucket/*",
"Principal": { "AWS": "325565585839" }
}
]
}
|
...
For convienance, AWS Cloudformation can be used to provision a custom AWS S3 bucket for use with Synapse. Using this approach will result in the exact same bucket as described in Setup with AWS Console.
Instructions:
Download the CF template.
Use the AWS Command Line or AWS Console to execute the template which will automatically provision the bucket.
Example using the the awscli:
...
Please see the REST docs for more information on setting external storage location settings using our REST API.
See also:/wiki/spaces/DOCS/pages/2048426057