Support create External File Handle in Python client
Previous title: Allow files in custom storage locations to be uploaded using sync function
I was working on linking and annotating files that were stored in an external S3 bucket, and I made a manifest file. I found the manifest structure is similar to the ones for uploading files in a local directory. It made me wonder if we can allow files in external custom storage locations to be uploaded using the `syncToSynapse` utility function?
Below is a demo s3 manifest file.
Also tagging as this is a big use case for HTAN.
We should probably not use the deprecated S3 syntax though and would also suggest adding an optional URL parameter for exteranalObjectStore file handles.
Tagging , and as we had discussed the need for this functionality in a few places today. It looks like this functionality has been replicated by Phil, William, Kenny and Milen already.
I can imagine a syncFromS3.