presigned upload URL expired warnings using Python client sync function

Description

User uploading here (https://www.synapse.org/#!Synapse:syn9874666) - large files of ~200GB each from an institutional cluster.

Getting THOUSANDS of warnings:

“The presigned upload URL has expired. Restarting upload...”

Advised user that with large files, warnings are expected but should not affect performance.

Engineers on the users's side are investigating to check on performance, will post any followup here.

Environment

None

Activity

Show:
Meredith Slota
August 30, 2017, 4:54 PM

Did 's upload actually succeed but he got misleading messages? I'm confused – I understand that we chunk up big files for upload but ... did anything fail here or is everything working as-intended except for warning messages?

Larsson Omberg
August 30, 2017, 4:56 PM

No, he got a message that the upload was being retried but he was receiving too many of them.

Kenneth Daily
August 30, 2017, 5:00 PM

I think so, ! I haven't heard back from the user, but am checking with him today. I noted that two 200GB files had been uploaded to the above mentioned location - I think he has one or two more to go.

To be clear from 's response, nothing failed, just too many warning (which should be addressed somehow, TBD).

Ziming Dong
January 9, 2018, 12:27 AM

To validate:

Download a very large file. We don't really have a way to change the aws presigned-url expiration time from the python client so I can't think of an easier way to validate.

Kenneth Daily
January 12, 2018, 7:23 PM

I think you meant to test I should upload a very large file. I did, and no warnings reported.

Assignee

Ziming Dong

Reporter

Kenneth Daily

Labels

None

Validator

Kenneth Daily

Development Area

None

Release Version History

None

Fix versions

Priority

Major
Configure