download failure with syncFromSynapse
Report from user, see attachments:
I'm trying to downloading the round 2 data using Python synapseclient. I've tried several times and all failed after a few bam files downloaded.
The script I used was the following:
syn = synapseclient.Synapse(debug=True)
files = synapseutils.syncFromSynapse(syn, 'syn10141118')
This script stoped downloading after running for about 25 minutes. The STDOUT outputs are the attached. Would you please to have a look on them and give me some guide? Thank you so much .
Hi Ziming, it is sensitive data, but file sizes are going to be 10-20 GB each.
If possible, I'd like to know the synapse Id of the file downloaded so I can try to replicate the issue. If it has sensitive data, just let me know the file size and I can test on a dummy file of that size instead.
From reading the logs, It looked like the client had a inconsistent connection to amazon's S3. By the time it was able to connect, the provided presigned url had already expired. At the very end of the log (work.liwenhui.sh.20170721-0805.nohup), the client managed to download 140MB of data before the connection abruptly ended.
If you haven't already done so, please try the download again making sure that you are on the latest version of the python client (1.7.2 improved the download retry logic), and are using a wired Ethernet connection (WiFi can be flaky at times for larger file downloads).