Expose STS-related APIs
Description
is a collection of tasks related to allowing access to select Synapse folders via AWS S3 APIs and tools. For example it allow a user to get temporary credentials (STS tokens) to access a folder. This issue is to expose the new APIs in the Python client. See who architected the feature and implemented the back end services. is a representative end user.
One approach might be to automatically detect 'sts configured' Synapse folders when uploading or downloading files, and automatically use boto when Synapse.store() or Synapse.get() is called (without needing to pass the STS token back to the caller).
See sample Python code that calls the new APIs here: https://github.com/Sage-Bionetworks/synapseDocs/pull/713/files?short_path=82e8674#diff-82e867462b91e301ff6befdfefbabffa
Environment
Activity
Unless someone can argue that the difference is important, I'm not concerned about being only 95% as fast as 'pure S3'. (And if it really is important, we can consider revisiting our strategy of requiring the MD5 digest, which would require back end as well as client changes.)
This is included in the current RC, 2.1.0.24.
I tested this feature using the default synapse storage as well as an external bucket that Jordan setup for me. After a few suggested changes were made to the docs, I feel comfortable that this feature is easy enough for advanced Synapse users who want to use the AWS CLI instead of the Synapse client to move large datasets. I recommend exploring the possibility of extending the duration of the session token to the maximum 36 hours (https://docs.aws.amazon.com/STS/latest/APIReference/API_GetSessionToken.html).
per William’s suggestion for allowing longer lived tokens, is the current 12 hour lifetime (which is also the STS get-session-token default) a security consideration or could a future services revision allow for longer lifetime (possibly parameterized)?
On the back-end, we call AssumeRole, not GetSessionToken, as GetSessionToken cannot be called by an IAM role. The maximum duration for AssumeRole is 12hours: https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html
Assignee
Reporter
Labels
Validator
Development Area
Release Version History
Epic Link
Sprint
Fix versions
Priority
