AWS Batch job unable to retrieve object from S3 bucket
Description
User runs code to download a file on EC2 and the file downloads, but when run on AWS batch receives a 403 Client Error: Forbidden Access Denied Error.
#User Code
```
import synapseclient
syn = synapseclient.Synapse()
syn.login(username, password)
entity = syn.get("syn18759102")
filepath = entity.path
```
User Issue Filed on Synapse at: https://www.synapse.org/#!Synapse:syn2580853/discussion/threadId=6934
User Provided logs from EC2 and AWS batch downloaded are attached.
Environment
AWS Cloud User's own docker container
Activity
Changes described in PRs merged and deployed and the user reports the issue is resolved.
Related PRs:
Per a Slack discussion I wil look at addressing this in the provisioning infrastructure as soon as I can.
Thanks ! This sounds great to me, I’ll consult with Khai and see if he has any feedback on the change. If not, do you know what that change would look like specifically in this yaml file?
It looks like this user is using an S3 VPC Endpoint and so the bucket policy is excluding them even though it is intra-region because traffic via an endpoint is routed internally rather than via a region associated public IP in the whitelist.
instead of using the below as one of the deny conditions on this bucket policy:
could we switch to:
This would allow any internally routed VPC endpoint traffic (which should by definition be intra-region, since endpoints do not currently support cross region) rather than just computevpc traffic, which I think aligns with the intention of this rule (allow any traffic as long as it has no egress costs). I’ve verified that a cross region request using a VPC endpoint was denied using a similar rule.
Alternately we could just add the user’s specific VpcId but that would not solve the general case.
Assignee
Reporter
Labels
Validator
Development Area
Release Version History
Slack Channel
Components
Fix versions
Affects versions
Priority
