Implement the asynchronous validation of Entities based on a bound schema

Description

Implement the asynchronous validation of Entities based on a bound schema. This includes the following services:

See: https://sagebionetworks.jira.com/wiki/spaces/PLFM/pages/989167828/Schemas+for+extending+Synapse+Objects

Environment

None

Activity

Show:
Bruce Hoff
August 21, 2020, 3:36 PM

I validated the functionality of the three services, see https://www.synapse.org/#!Synapse:syn22311050.3

Reopening because clean-up is needed in the doc's:
https://rest-docs.synapse.org/rest/GET/entity/id/schema/validation.html
'id' parameter has no definition
(see https://rest-docs.synapse.org/rest/GET/entity/id/schema/validation/statistics.html which DOES have the required definition)

https://rest-docs.synapse.org/rest/GET/entity/id/schema/validation/statistics.html
Are the results eventually consistent? If so how do you tell if they are up to date. (See https://rest-docs.synapse.org/rest/GET/entity/id/schema/validation.html which has a very nice explanation on how to check for consistency.)

https://rest-docs.synapse.org/rest/POST/entity/id/schema/validation/invalid.html
'id' parameter has no definition
(see https://rest-docs.synapse.org/rest/GET/entity/id/schema/validation/statistics.html which DOES have the required definition)
Are the results eventually consistent? If so how do you tell if they are up to date. (See https://rest-docs.synapse.org/rest/GET/entity/id/schema/validation.html which has a very nice explanation on how to check for consistency.)

John Hill
August 24, 2020, 9:02 PM
Bruce Hoff
August 27, 2020, 10:25 PM

Thanks for updating the doc's. Regarding:
https://rest-docs.synapse.org/rest/GET/entity/id/schema/validation/statistics.html
the instructions for determining whether the statistics are up to date are now:

  • The validation of an Entity against its bound schema is automatic and
    * eventually consistent. Keep checking this method to get the latest validation
    * statistics for the given container.

I see that the returned value, https://rest-docs.synapse.org/rest/org/sagebionetworks/repo/model/schema/ValidationSummaryStatistics.html, has a field:

Is it correct to say:

The statistics can be taken as up-to-date if the returned generatedOn date is later than the modifiedOn date of the container and of the entities in the container (whose annotations are validated against the container's schema).

That is, is there an algorithm that the client can use to know when it has up-to-date statistics?

John Hill
August 28, 2020, 1:29 AM

:

The statistics can be taken as up-to-date if the returned generatedOn date is later than the modifiedOn date of the container and of the entities in the container (whose annotations are validated against the container's schema).

The statistics could be out-of-date even if that is true. A change to a schema or one of its dependencies, or a change in binding would make the statistics stale. I do no think there is an easy algorithm for clients to determine if the stats are stale.

I am providing the 'generatedOn' timestamp with the assumption that we might need to cache the statistics in the future. So if the 'generatedOn' timestamp is old then you know the results are old. However, if the 'generatedOn' is recent you cannot conclude that they are up-to-date. It is not perfect but it might be the best we can do.

Assignee

John Hill

Reporter

John Hill

Labels

None

Validator

Bruce Hoff

Development Area

None

Release Version History

None

Sprint

None

Fix versions

Priority

Major
Configure