Concurrent migration of entities sharing the same file handle can result in an error
The migrate_indexed_files function of synapseutils.migrate_functions can migrate entities concurrently depending on the threading configuration of the client. If two files/versions share the same file handle and a multipart copy occurs concurrently it can produce a DuplicateKeyException in the backend:
However this exposes that the utility should detect multiple files/versions sharing the same file handle and only copy that file handle once, updating the meta data of the additional files/versions to use the existing file handle id.
encountered this with a test project that had a file entity with multiple identical versions, but it's also possible to reproduce with separate files sharing the same file handle. It is reproducible with the following: