Page Comparison

NOTE: The Evaluation services were initially introduced as the Competition services

Overview

...

Synapse Evaluations are constructed around three primary objects: Evaluation , Participant, and Submission. These objects are defined by eponymous JSON schema files in the Synapse Repository Services. Image RemovedThe UMLet file used to generate the following image is available here.

Image Added

Evaluation
Object representing a Synapse Evaluation

ID - unique ID for the Evaluation
eTag - optimistic concurrency flag; changes with each update
ownerID - principal ID of the owner of the Evaluation
name
description
contentSource - ID of the data source (e.g. entity) for the Evaluation (string)
status - enum describing the state of the Evaluation, one of: PLANNED, OPEN, CLOSED, COMPLETED

...

ownerID - principal ID of the Participant
evaluationID
createdOn - join date

Submission
Immutable object to represent an entity submission to a Evaluation. A participant can create many Submissions for a given Evaluation.

ID - unique ID for the Submission
name
ownerID - principal ID of the Submission owner
evaluationID
entityID - ID of the submitted entity
version - specific entity version
createdOn - submission date
submitterAlias - the alias for the user or team creating the submission

SubmissionStatus
Object to track the status and score of a specific Submission. This object is generated at the time of submission, and can be modified by Evaluation administrators.

submissionID
eTag - optimistic concurrency flag; changes with each update
status - enum describing the state of the Submission, one of: OPEN, CLOSED, SCORED, INVALID
score - numerical score for this Submission (double between 0 and 1)
report - String describing this Submission and/or its scoring. May contain additional scoring information, error logs, etc.
annotations - Annotations object containing user-defined key-value annotations. Note: The contents of the annotations are mirrored to a second set of tables optimized for querying, as discussed here.

SubmissionBundle
Bundle object containing a Submission and its accompanying SubmissionStatus.

SubmissionStatusBatch

Used to upload a collection of Submission updates at once.

REST API

Evaluation

URL	HTTP Type	Description
/evaluation	POST	Create a new Evaluation
/evaluation/{evaluationId}	GET	Get a Evaluation
/evaluation/{evaluationId}	PUT	Update a Evaluation
/evaluation/{evaluationId}	DELETE	Delete a Evaluation
/evaluation	GET	Batch Get Evaluations (Paginated) Optional request parameters: limit (long), offset (long)
/evaluation/count	GET	Get the number of Evaluations
/evaluation/name/{name}	GET	Find a Evaluation by name Note that the name must be URL-encoded
/evaluation/available	GET	Get the paginated list of Evaluations in which the authenticated user may participate. This includes (1) Evaluations which the user owns and (2) those which the user has joined as a participant. Optional request parameters: limit (long), offset (long), status (OPEN,CLOSED,...)

Participant

URL	HTTP Type	Description
/evaluation/{evaluationId}/participant	POST	Join as a Participant in a Evaluation *
/evaluation/{evaluationId}/participant/{principalId}	POST	/evaluation/{evaluationId}/participant/	GET	Add another user as a Participant in a Evaluation . * Requires admin rights on the Evaluation .
/evaluation/{evaluationId}/participant/{principalId}	GET	Get a Participant
/evaluation/{evaluationId}/participant/{principalId}	DELETE	Delete a Participant
Batch get Participants for a given Evaluation(Paginated) Optional request parameters: limit (long), offset (long)	/evaluation/{evaluationId}/participant/count	GET	Get the number of Participants in a given Evaluation

*A Synapse user can only join as a Participant in an Evaluation if the Evaluation is OPEN. However, the Evaluation owner can add other users as Participants at any time, regardless of the Evaluation state.

Submission

URL	HTTP Type	Description
/evaluation/submission?etag=<etag>	POST	Create a new Submission (and corresponding SubmissionStatus object). Requires the current etag of the Entity in the Submission. Required request parameter: etag (string)
/evaluation/submission/{submissionId}	GET	Get a Submission Requires ownership of the Submission, or admin rights on the Evaluation.
/evaluation/submission/{submissionId}/status	GET	Get the status of a Submission
/evaluation/submission/{submissionId}/status	PUT	Update the status of a Submission. Requires admin rights on the Evaluation
/evaluation/submission/{submissionId}	DELETE	Delete a Submission. Requires admin rights on the Evaluation.
/evaluation/{evaluationId}/submission/ /evaluation/{evaluationId}/submission/bundle/	GET	Batch get my Submissions/SubmissionBundles for a given Evaluation. Results are paginated. Returns only Submissions owned by the requesting user. Optional request parameters: limit (long), offset (long)
/evaluation/{evaluationId}/submission/all /evaluation/{evaluationId}/submission/status/all /evaluation/{evaluationId}/submission/bundle/all	GET	Batch get Submissions/SubmissionStatuses/SubmissionBundles for a given Evaluation. Results are paginated. Requires admin rights on the Evaluation. Can be filtered by SubmissionStatusEnum. Optional request parameters: status (String), limit (long), offset (long)
/evaluation/submission/{submissionId}/file/{fileHandleId}	GET	Get a temporary pre-signed URL to a File contained within a specified Submission. Requires admin rights on the Evaluation. Optional request parameter: redirect (Boolean)
/evaluation/{evaluationId}/submission/count	GET	Get the number of Submissions for a given Evaluation.

Access Restrictions

Access Restrictions and approvals may be applied to Evaluations. For details on the API see: Data Access Control

...

Create an Evaluation

Code Block

curl -i -k -H sessionToken:xxxxxxxxxxxxxxxx -H Accept:application/json -H Content-Type:application/json -d '{
    "status":"PLANNED",
    "description":"description",
    "name":"my first evaluation",
    "contentSource":"contentSource"
}' https://repo-staging.prod.sagebase.org/repo/v1/evaluation

Get an Evaluation

Code Block
curl -i -k -H sessionToken:xxxxxxxxxxxxxxxx -H Accept:application/json 'https://repo-staging.prod.sagebase.org/repo/v1/evaluation/1588317'

Update an Evaluation

Code Block

curl -i -k -H sessionToken:xxxxxxxxxxxxxxxx -H Accept:application/json -H Content-Type:application/json -X PUT -d '{
    "id":"1588317",
    "createdOn":"2013-01-16T16:30:56.727Z",
    "etag":"eed22bca-d88e-4b4a-8b1a-35dca7b7b8db",
    "status":"OPEN",
    "description":"description",
    "ownerId":"1588313",
    "name":"my first evaluation",
    "contentSource":"contentSource"
}' https://repo-staging.prod.sagebase.org/repo/v1/evaluation/1588317

Delete an Evaluation

Code Block
curl -i -k -H sessionToken:okxkSRyl3v0iIf0eutQfXA00 -H Accept:application/json -X DELETE 'https://repo-staging.prod.sagebase.org/repo/v1/evaluation/1588317'

Participant

Create a Participant

Code Block

curl -i -k -H sessionToken:xxxxxxxxxxxxxxxx -H Accept:application/json -H Content-Type:application/json -d '{
    "ownerId":"123456"
}' https://repo-staging.prod.sagebase.org/repo/v1/evaluation/987654/participant

...

Delete a Participant

Code Block
curl -i -k -H sessionToken:xxxxxxxxxxxxxxxx -H Accept:application/json -X DELETE 'https://repo-staging.prod.sagebase.org/repo/v1/evaluation/1588317/participant/1588313'

Submission

Create a Submission

Code Block

curl -i -k -H sessionToken:xxxxxxxxxxxxxxxx -H Accept:application/json -H Content-Type:application/json -d '{
    "evaluationId":"1588317",
    "entityId":"1588315",
    "versionNumber":"1",
    "name":"some-name"
}' https://repo-staging.prod.sagebase.org/repo/v1/evaluation/submission

Update a SubmissionStatus

Code Block

curl -i -k -H sessionToken:xxxxxxxxxxxxxxxx -H Accept:application/json -H Content-Type:application/json -X PUT -d '{
    "id":"1589914",
    "modifiedOn":"2013-01-18T21:43:18.897Z",
    "etag":"13c88b1a-662c-4dd6-bb56-3976566516f7",
    "status":"SCORED",
    "score":0.42
}' https://repo-staging.prod.sagebase.org/repo/v1/evaluation/submission/1589914/status

Delete a Submission

Code Block
curl -i -k -H sessionToken:dPF2TqGUq121PR36AwJhXw00 -H Accept:application/json -X DELETE 'https://repo-staging.prod.sagebase.org/repo/v1/evaluation/submission/1589912'

...

Versions Compared

Old Version 46

New Version Current

Key

Table of Contents

Overview

REST API

Access Restrictions

Participant

Submission