downloaded files are reset when disk space exhausted


From the Synapse forum:!Synapse:syn5637528/discussion/threadId=7464

I am trying to download via CLI, and using a recently updated client, the downloaded files keep erasing themselves and starting over once they get to 100MB. It will do this with six files simultaneously, and the "Downloaded" status bar will reverse, and the downloaded size will get smaller, not bigger(?!). Eventually, after about an hour, it will error out with:

ValueError: File download failed during sync
synapseclient 2.2.0 from bioconda




Bruce Hoff
October 16, 2020, 6:15 PM

Sorry for the delay. I validated using a docker container:

I verified that the download 'hangs'. In fact it made the Docker engine hang then crashed my computer.

Next step:

Nice: Exited cleanly with a clear message. Did NOT crash the Docker engine or my computer!

Jordan Kiang
October 14, 2020, 11:02 PM

Oh I see the version of Python you were using was 3.4 which is not supported by synapseclient (and hasn’t been since synapseclient 1.8.2). A python version with a reasonably modern setuptools will not allow you to even attempt the install of an unsupported version from pip, but this from your output indicates that this old python 3.4 installation in Ubuntu 14.04 LTS (Ubuntu 14.04 LTS is itself no longer supported) is using the older distutils which doesn’t honor the python_requires option and attempts the install anyway.

Sorry for the trouble but the validation steps should work on a supported Python version.

Alternately here is the Mac equivalent of the above steps (Python 3.7 is the default on Mac Catalina).

Jordan Kiang
October 14, 2020, 9:55 PM

An equivalent Mac command for a ramdisk would be

The volume would then be /Volumes/ramdisk

However the setuptools issue itself is concerning. I just installed it on a fresh ubuntu ec2 venv without aparent issue.

Bruce Hoff
October 14, 2020, 9:50 PM

I'm not going to be able to finish this today. I can try again Friday. If you need it sooner, please ask someone else to validate.

Bruce Hoff
October 14, 2020, 9:48 PM

Tried another ubuntu instance:

... Looks like I have to update setuptools.

Your pinned fields
Click on the next to a field label to start pinning.


Jordan Kiang


Bruce Hoff


Bruce Hoff

Development Area

Synapse Core Infrastructure