Next Step Against JSON Schema API

Introduction

Due to the complexity of JSON schemas, it can be nontrivial to evaluate what can be added next to an annotation. This API will address this issue by providing the client all the fields that can be annotated unconditionally and immediately.

To understand the necessity of an API like this, we can take a look at one simple example.

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "country": {
      "enum": ["USA", "CA"]
    }
  },
  "if": {
    "properties": {
      "country": {
        "const": "USA"
      }
    }
  },
  "then": {
    "properties": {
      "state": {
        "type": "string"
      }
    }
  }
}

In this JSON schema we have a “country” field with an enumeration that gives us the choice of choosing “USA” or “CA”. If our annotations were empty, our proposed API would tell us that we can fill in the “country” field. If the annotations were to be updated with the country “USA”, a second call to our API would tell us that we can now fill in a “state”. The complexity of these schemas can grow quickly and it may not be user friendly for an annotator to parse the schema to understand what they can add next to the annotations and what becomes invalid upon updates to the annotations. This API is posed to solve this issue.

Use Case 1

The simplest use case is where we have an existing entity with a bound JSON schema, but it may or may not contain annotations on it. In this situation, using this proposed idea, a client can do repeated calls to this API to determine a next step in an iterative annotation process. The client may ask for the next step (meaning all unfilled fields that can be filled immediately) on an entity, and once the job returns the fields, the client can use these fields to render a form for the user to fill out. When the user fills out the form, the client can use existing API to add these new annotations given by the user to the entity. To handle this case, the missing API is figuring out this next step on the entity. This API will need to address this by evaluating the current annotations on the entity to its bound JSON schema to determine what fields are available to be immediately filled unconditionally with respect to the annotations present.

Use Case 2

The second use case is where the entity does not exist yet, but we know the destination that the entity will be uploaded to. This destination will contain a JSON schema bound to it, and we want our API to build the JSON annotations before the client decides to upload the entity to that destination and apply the annotations. This next implementation will allow the client to use the API to send a JSON representing the annotations and get back the fillable fields that can be rendered to the user to be filled out. These fields are determined by the JSON schema at the destination against the JSON annotations in the request. This implementation should be used to iteratively build up a JSON on the client side. At the end of this process, the client should have a JSON that they could put onto an entity that will be uploaded to that destination.

This API should handle both of our use cases with 2 implementations.

Proposed API

{
    "$schema": "http://json-schema.org/draft-07/schema",
	"$id": "org.sagebionetworks-NextStepRequest",
	"description": "A request to start an asynchronous job get the next step in annotating against a schema for some annotation state.",
	"implements": [
		{
			"$ref": "org.sagebionetworks.repo.model.asynch.AsynchronousResquestBody"
		}
	],
	"properties": {
		"requestDetails": {
			"description": "Required.  Must be one of the implementations of NextStepRequestInterface",
			"$ref": "org.sagebionetworks-NextStepRequestInterface"
		},
		"concreteType": {
			"type": "string",
			"description": "Required. For this type the value should be: 'org.sagebionetworks-NextStepRequest'"
		}
	}
}

{
	"$schema": "http://json-schema.org/draft-07/schema",
	"$id": "org.sagebionetworks-NextStepRequestInterface",
	"type": "interface",
	"description": "Required interface that must be one of the implementations of NextStepRequestInterface",
	"properties": {
		"concreteType": {
			"type": "string",
			"description": "Required. Should be the full package name of the details implementation, for example 'org.sagebionetworks-NextStepOnEntityRequest'
		}
	}
}

The above request object for starting the asynchronous job to get a next step will contain a reference to the interface, which will specify the implementation of the interface that will address use case 1 or use case 2.

{
    "$schema": "http://json-schema.org/draft-07/schema",
	"$id": "org.sagebionetworks-NextStepResponse",
	"description": "The results of an asynchronous job to get the next step for annotating against a JSON schema.",
	"implements": [
		{
			"$ref": "org.sagebionetworks.repo.model.asynch.AsynchronousResponseBody"
		}
	],
	"properties": {
		"responseDetails": {
			"description": "The response details type is determined by the request details type.",
			"$ref": "org.sagebionetworks-NextStepResponseInterface"
		}
	}
}

{
	"$schema": "http://json-schema.org/draft-07/schema",
	"$id": "org.sagebionetworks-NextStepResponseInterface",
	"type": "interface",
	"description": "Base interface that all next step responses implement.",
	"properties": {
		"concreteType": {
			"type": "string",
			"description": "Indicates the full package name of the response type."
		}
	}
}

Similar to the request object, the above response object will consist of a reference to the interface that will determine the implementation of our next step operation.

First Implementation and Solution to Use Case 1

{
	"$schema": "http://json-schema.org/draft-07/schema",
	"$id": "org.sagebionetworks-NextStepOnExistingEntityRequest",
	"description": "Implementation that finds the next step on an existing entity with a bound JSON schema.",
	"implements": [
		{
			"$ref": "org.sagebionetworks-NextStepRequestInterface"
		}
	],
	"properties": {
		"entityId": {
			"description": "ID of entity to get next step for annotating against the bound schema.",
			"type": "number"
		}
	}
}

{
	"$schema": "http://json-schema.org/draft-07/schema",
	"$id": "org.sagebionetworks-NextStepOnExistingEntityResponse",
	"description": "Implementation of the results for a job that finds the next step on an existing entity with a bound JSON schema.",
	"implements": [
		{
			"$ref": "org.sagebionetworks-NextStepResponseInterface"
		}
	],
	"properties": {
		"entityId": {
			"description": "ID of entity to get next step for annotating on.",
			"type": "number"
		},
		"nextStepSchema": {
			"description": "JSON schema of the next step",
			"$ref": "org.sagebionetworks.repo.model.schema.JsonSchema"
		}
	}
}

This implementation of the interface is to address use case 1, where we have an existing entity uploaded. This entity will have a JSON schema and possibly existing annotations on it. Using this implementation to address use case 1, the following is a possible solution and application of this API implementation by the client.

1. Start the asynchronous job indicating the NextStepOnExistingEntityRequest implementation of our request, providing the entity ID. This gives back to the client the job token to monitor the job.

2. The job will get the entity’s bound schema and current annotations, and find the fields that are available for annotating.

3. Client will track the job.

4. Once the job finishes, the client can take the JSON schema contained in the response and render it to the user.

5. Once input is taken by the user, the client can use existing API to update the entity’s annotations with the new annotations received from the user input.

6. Repeat until the JSON schema returned is empty and the annotations are valid against the schema.

Second Implementation and Solution to Use Case 2

{
	"$schema": "http://json-schema.org/draft-07/schema",
	"$id": "org.sagebionetworks-NextStepOnFutureEntityRequest",
	"description": "Implementation finds the next step against the destination's bound JSON schema.",
	"implements": [
		{
			"$ref": "org.sagebionetworks-NextStepRequestInterface"
		}
	],
	"properties": {
		"destinationEntityContainerId": {
			"description": "ID of the destination entity where the future entity will be created in.",
			"type": "number"
		},
		"annotations": {
			"description": "Annotation state that will be compared to the destination's JSON schema.",
			"type": "object"
		}
	}
}

{
	"$schema": "http://json-schema.org/draft-07/schema",
	"$id": "org.sagebionetworks-NextStepOnFutureEntityResponse",
	"description": "Implementation of the results of a job that finds the next step for annotations against some destination.",
	"allOf": [
		{
			"$ref": "org.sagebionetworks-NextStepResponseInterface"
		}
	],
	"properties": {
		"nextStepSchema": {
			"description": "JSON schema of the next step",
			"$ref": "org.sagebionetworks.repo.model.schema.JsonSchema"
		}
	}
}

This second implementation of the interface is to address use case 2, where we have do not yet have an entity uploaded, but we know the destination we intend to upload to. The client in this case will have to handle the future JSON annotations and build it iteratively using our proposed API. The following is a possible solution and application of this implementation for use case 2.

1. Start the asynchronous job indicating the NextStepOnFutureEntityRequest implementation of our request, providing the destination entity ID, and the JSON representing the future annotations. This gives back the job token to monitor the job.

2. The job will find the JSON schema bound to the location and find the fields that are available to be annotated.

3. The client will continue to monitor the job.

4. Once the job finishes, the client can use the JSON schema returned to render the next step fields to the user for user input.

5. Once input is taken, the client will update the future JSON annotations being built with the new input.

6. Repeat from Step 1 until the returned JSON schema is empty.

7. Now the JSON annotations are complete and the client put them onto an entity that will be uploaded to that destination.

The following document demonstrates handling of the next-step schema. Handling Next Step JSON Schema Cases