Expose STS-related APIs

Description

is a collection of tasks related to allowing access to select Synapse folders via AWS S3 APIs and tools. For example it allow a user to get temporary credentials (STS tokens) to access a folder. This issue is to expose the new APIs in the Python client. See who architected the feature and implemented the back end services. is a representative end user.

One approach might be to automatically detect 'sts configured' Synapse folders when uploading or downloading files, and automatically use boto when Synapse.store() or Synapse.get() is called (without needing to pass the STS token back to the caller).

See sample Python code that calls the new APIs here: https://github.com/Sage-Bionetworks/synapseDocs/pull/713/files?short_path=82e8674#diff-82e867462b91e301ff6befdfefbabffa

Environment

None

Activity

Show:
Dwayne Jeng
June 10, 2020, 5:33 AM
Edited

On the back-end, we call AssumeRole, not GetSessionToken, as GetSessionToken cannot be called by an IAM role. The maximum duration for AssumeRole is 12hours: https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html

Jordan Kiang
June 10, 2020, 3:00 AM

per William’s suggestion for allowing longer lived tokens, is the current 12 hour lifetime (which is also the STS get-session-token default) a security consideration or could a future services revision allow for longer lifetime (possibly parameterized)?

William Poehlman
June 10, 2020, 1:35 AM

I tested this feature using the default synapse storage as well as an external bucket that Jordan setup for me. After a few suggested changes were made to the docs, I feel comfortable that this feature is easy enough for advanced Synapse users who want to use the AWS CLI instead of the Synapse client to move large datasets. I recommend exploring the possibility of extending the duration of the session token to the maximum 36 hours (https://docs.aws.amazon.com/STS/latest/APIReference/API_GetSessionToken.html).

Jordan Kiang
May 29, 2020, 3:32 PM

This is included in the current RC, 2.1.0.24.

Bruce Hoff
May 20, 2020, 7:53 PM

Unless someone can argue that the difference is important, I'm not concerned about being only 95% as fast as 'pure S3'. (And if it really is important, we can consider revisiting our strategy of requiring the MD5 digest, which would require back end as well as client changes.)

Fixed

Assignee

Jordan Kiang

Reporter

Bruce Hoff

Labels

None

Validator

William Poehlman

Development Area

None

Release Version History

None

Slack Channel

None

Epic Link

Sprint

None

Fix versions

Priority

Major