Versioning broken for sftp files

Description

When updating a file that is stored on an external sftp server the client does not re-upload the file. The metadata is updated however. An easy repro:

synapse add ~/bye.txt --parentId syn5761826
echo bye >> ~/bye.txt #Change file
synapse add ~/bye.txt --parentId syn5761826 #File is not uploaded

Looking through previous clients both v1.4 and v1.5 are broken but 1.2dev2 works.

Doing a full git bisect shows that the regression was introduced at:

Bisecting: 0 revisions left to test after this (roughly 1 step)
[923185806fb720664c73fd1240b0649329be1731] replace use of old cache with use of new cache for

Environment

None

Activity

Show:
Xavier Schildwachter
June 1, 2017, 10:59 PM

Does not seem to work for me:

A new version was created but it still contains the externalURL. So in portal, v2 still appears as an external file and when attempting to download it tries to go to the URL.

Ziming Dong
June 2, 2017, 2:59 AM
Edited

If the user was to do a syn.get() on the a FileEntity with an ExternalFileHandle then: externalURL="http://www.website.com/file.txt" path="/local/path/file.txt". It doesn't make sense that if they then did a syn.store() on that object that it would modify the file handle to point to the local file

Larsson Omberg
June 2, 2017, 4:42 AM

Ziming, the issue is that setting entity.path usually changes causes the file to get uploaded. If I undestand correctly, What you are saying is that since File with externalFileHandles have synapseStore=False it never should be uploaded to begin with? This will require some serious documentation to make sure that it is not enough to always change the path to update the fileType OR we change synapseStore when path gets set.

Ziming Dong
June 2, 2017, 5:45 PM

Yes, I think synapseStore should indicate whether or not we want to update the FileHandle information and use a different variable to change the FileHandle type.

I got some really weird behavior when I tried switching from a Synapse stored S3FileHandle to a ExternalFileHandle. I was only able to convert from S3FileHandle to ExternalFileHandle after I've modified the file and also set synapseStore=False. just synapseStore=False by itself did nothing (probably because we cache the last modified time). However once the FileEntity pointed to a ExternalFileHandle, I was unable to swap back to S3FileHandle even though I modified the file and set synapseStore=True.

Xavier Schildwachter
June 15, 2017, 2:36 PM

Verified fixed py-1.7 vs 185.
Q: What's the browser supposed to do with that url in portal?

Assignee

Ziming Dong

Reporter

Larsson Omberg

Labels

None

Validator

Xavier Schildwachter

Development Area

None

Release Version History

None

Components

Fix versions

Affects versions

Priority

Blocker
Configure