Migration tool erroring when it shouldn't

Description

I was doing some more testing of the migration tools and ran into another error. It doesn't look like it broke anything, but I'm not sure why it was thrown or if it's something that could be a problem later (particularly if I use `continue_on_error = True`).

This is the file that is causing the problem: file (I gave access). It appears that the file migrated just fine, but the client stopped and left an error in the database (see attached image). If I look at the entity, I see that the file handle Id has been updated to 72973081, as it should have been. If I run the index tool a second time to make a new database, the new database also shows the file in the new location.

In short, it migrated the file how it should have, but still threw an error.

Full callback (still using the Synapse Python Client in R via reticulate):

Environment

RStudio with R 4.0 and reticulate. Synapse Python Client v2.3, I believe.

Activity

Show:
Nicole Kauer
March 3, 2021, 12:35 AM

Works great! Thank you, !

Jordan Kiang
February 27, 2021, 11:12 PM

The updated release candidate should resolve this issue (2.3.0.192).

I used the below script to repro this issue:

So running the above on the previous RC will repro the issue, e.g. installed with:

But running on the latest should not and all files should be moved to the new storage location. It can be installed e.g.

Nicole Kauer
February 26, 2021, 8:26 PM

When I mark this as Resolved and build a new release candidate version I could share my repro steps with you if you’d like to validate it that way.

That works for me. Thanks!

Jordan Kiang
February 26, 2021, 8:15 PM

it’s not 100% deterministic, as it involves a timing issue when migrating with multiple threads.

What happens it that you have at multiple files and/or versions that share the same file handle in the source project/folder. When you migrate the first time the function sees a file handle it will migrate it and then it will defer migrating any other entities that share the same file handle until the first migration is finished so that all the migrated files can share the same copied file handle. Once the first is done it’s possible for shared file to get queued for migration multiple times, the first time will succeed but the second time it fails since it’s already migrated.

Since it’s not 100% deterministic it’s not super easy to repro but I have a script that creates a project with 50 versions of a file that use the same file handle which will reliably repro it. When I mark this as Resolved and build a new release candidate version I could share my repro steps with you if you’d like to validate it that way.

Nicole Kauer
February 26, 2021, 7:42 PM

, you mentioned on our call yesterday that you were able to repro this, but I am not quite sure how to. What would be your suggestion for how I ‘validate’ this is fixed.

Fixed

Assignee

Jordan Kiang

Reporter

Nicole Kauer

Labels

None

Validator

Nicole Kauer

Development Area

None

Release Version History

None

Slack Channel

None

Fix versions

Affects versions

Priority

Major