User getting 503 error when downloading
We have a user in the AMP-AD project who is using the python client (v1.3) to download a lot of BAM files that are several GB each. They are getting about 2MB/sec transfer rate and the job failed with a 503. They are running the download from their internal hardware (not cloud).
Comments from user:
It managed to pull 48/60 BAM files before timing out:
ConnectionError: HTTPSConnectionPool(host='repo-prod.prod.sagebase.org', port=443): Max retries exceeded with url: /repo/v1/entity/syn4487045/version/1/file (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 503 Service Unavailable',)))
I don’t see a way to resume. I re-ran it earlier and it seemed to just overwrite existing files.
Phil is correct that the speed was consistently about 2 MB/s. It took 2 whole days to pull the 48 BAM files (400 GB).
This looks like throttling on the request to Synapse for a signed S3 URL. Retry was bypassed in this case; fixed as SYNPY-290.