We're updating the issue view to help you get more done. 

Error messages on upload retry inconsistent with behavior

Description

When uploading large files and the retry is triggered the message is inconsistent with what the client does. For example: warnings.warn("Attempt to complete upload failed: " + status['errorMessage'])

Seems to indicate that the upload failed but then it continues and finishes. See output below.

Uploading [###################-]95.26% 42.2GB/44.3GB jhu-usc.edu_PANCAN_HumanMethylation450.betaValue.tsv HTTPSConnectionPool(host='s3.amazonaws.com', port=443): Max retries exceeded with url: /proddata.sagebase.org/372127/fb6e25d4-8857-4c07-ae8b-bcfe1b53b9b9/jhu-usc.edu_PANCAN_HumanMethylation450.betaValue.tsv/8650?AWSAccessKeyId=AKIAIV5XCDRXPWB67YRQ&Expires=1433807733&Signature=HXnOUiCvYMwF0rPOJEIEeD%2FnkIA%3D (Caused by <class 'httplib.BadStatusLine'>: '')
retrying exception: ConnectionError HTTPSConnectionPool(host='s3.amazonaws.com', port=443): Max retries exceeded with url: /proddata.sagebase.org/372127/fb6e25d4-8857-4c07-ae8b-bcfe1b53b9b9/jhu-usc.edu_PANCAN_HumanMethylation450.betaValue.tsv/8650?AWSAccessKeyId=AKIAIV5XCDRXPWB67YRQ&Expires=1433807733&Signature=HXnOUiCvYMwF0rPOJEIEeD%2FnkIA%3D (Caused by <class 'httplib.BadStatusLine'>: '')

... Retrying in 1 seconds...
Uploaded Chunks [####################]100.00% 44.3GB/44.3GB jhu-usc.edu_PANCAN_HumanMethylation450.betaValue.tsv Done...
Waiting for Confirmation -/gluster/home/lomberg/Dropbox/Sage/synapsePythonClient/synapseclient/client.py:1940: UserWarning: Attempt to complete upload failed: java.lang.RuntimeException: Timed out waiting for S3 object to become visible: Key: 372127/fb6e25d4-8857-4c07-ae8b-bcfe1b53b9b9/jhu-usc.edu_PANCAN_HumanMethylation450.betaValue.tsv/9047 storageLocationId: null token: org.sagebionetworks.repo.model.file.ChunkedFileToken [uploadId=gUa28DrvCYw6x3lQiB1Dh.mg8AbkHXquclAmK253mNdGNas8NvjoFDZ_e6C5uLwP965ubSLx37b1kshNToF6jpEmahIU_kx_St5K4R4G0.0dkFHiUvqo34T3wD_ezAwy contentMD5=b54dda1e309112dd0230166e34e10296 fileName=jhu-usc.edu_PANCAN_HumanMethylation450.betaValue.tsv contentType=text/tab-separated-values key=372127/fb6e25d4-8857-4c07-ae8b-bcfe1b53b9b9/jhu-usc.edu_PANCAN_HumanMethylation450.betaValue.tsv storageLocationId=null ]
warnings.warn("Attempt to complete upload failed: " + status['errorMessage'])
Upload completed in 3 hours, 38 minutes, 6 seconds.

Environment

None

Status

Assignee

Chris Bare

Reporter

Larsson Omberg

Labels

None

Validator

Larsson Omberg

Release Version History

None

Components

Fix versions

Priority

Minor