Document toolboxDocument toolbox

Proposed Version API

Owned by John Hill
Last updated: Jun 17, 2011Version comment
6 min read

Introduction

The current Synapse schema is a hierarchy of entities such as:

  • Project
    • Dataset1
      • Location 1 (dataset's location)
      • LayerA
        • Location 2 (layer's location)

In this schema, raw data files are represented by the Location entities. These locations track the path to a physical file, type and MD5. If a user wanted to upload a new version of one of these files how would they do it? What metadata would Synapse need to track for the version and how can a user discover things like:

  • What is the current version?
  • What versions exist for this location?
  • Who created a version?
  • How do I download the latest version?
  • How do I download an older version?

Since all of our current use cases only required versioning to apply to leaf entities such as Locations, only leaf entities will be version-able in the first iteration of versioning. The purpose of this document is proposes an API for versioning of leaf entities such as Locations.

Version-able Entities

Any leaf entity that is version-able will need additional metada. Here is the JSON schema for these additional fields:

{
  "properties": {
    "versionLabel": {"type": "string"},
    "versionNumber": {"type": "number"},
    "versionUrl": {"type": "string"},
    "versions": {"type": "string"},
    "versionComment" {"type": "string"},
  },
  "type": "object"
}

Field Name

User Provided / Auto-generated

Description

versionLabel

User Provided

The user visible label for this revision. It is up to the user to provide a meaningful label

versionComment

User Provided

The user provided comment for this version. For example, what is this version, and why was it created?

versionNumber

Auto-generate

The contiguous number representing this version. The first version will be '1', followed by '2'...'n'

versionUrl

Auto-generated

The URL that can be used to fetch this version of the entity

versions

Auto-generated

The URL to list all versions of this entity

Version-able Entity Metadata

For version-able Entities, some metadata applies to the root object and is therefore version independent, while other metadata applies to the version and is therefore, version dependent. For example, the Access Control List (ACL) of an entity is version independent. This means it is not possible to apply different permissions to different versions of an entity. While the Annotations of an entity are version dependent. This means each version of the same entity can have different Annotations. The following table lists the various entity metadata and whether it is version dependent or independent.

Metadata

Version Dependent /Independent

Implications

Access Control List

Independent

Access to all versions is controlled with a single ACL

Name

Independent

One name applies to all versions of an entity

Description

Independent

One description applies to all versions of an entity

Created-On

Independent

An entity can only be created once

Created-By

Independent

An entity can only be created once

Parent-Id

Independent

All version of an entity must have the same parent

Annotations

Dependent

Each version of an entity can have its own annotations

Modified-By

Dependent

First set when a new version is created, and updated every time its version is change. Once a new version is created the Modified-By fields of older versions will remain static as older versions are immutable

Modified-On

Dependent

First set when a new version is created, and updated every time its version is change. Once a new version is created the Modified-On fields of older versions will remain static as older versions are immutable

Version API

Current API

The following table describes how versioning effects the current entity CRUD API:

URL

HTTP Type

Description

/{entityType}

POST

Creates a new entity of the given {entityType}. For version-able entities, this will create the first version of this entity. This entity will have a versionNumber=1

/{entityType}

GET

Get a list of all entities of the given type: {entityType}. For version-able entity types, this will list the current version of each entity of that type.

/{entityType}/{id}

GET

Get an entity using its type and id. For version-able entities this will return the current version of the entity.

/{entityType}/{id}

PUT

Update the entity identified by type and id. For a version-able entity this will update the current version of the entity.

/{entityType}/{id}

DELETE

Delete the entity identified by type and id. For a version-able entity this will delete the entity and all versions of the entity.

/{entityType}/{id}/annotations

GET

Get the annotations for the entity identified by type and id. For a version-able entity this will return the annotations of the current entity.

/{entityType}/{id}/annotations

PUT

Update the annotations for the entity identified by type and id. For a version-able entity this will update the annotations of the current entity.

Version-able Additions to the API

The following table describes the new methods for manipulating the versions of a version-able entity.

URL

HTTP Type

Description

/{versionable-entityType}/{entityId}/version

POST

Create a new version of a given version-able entity. The user should provide a versionLabel and Synapse will auto-generate a new versionNumber. When this called on a version-able entity, the annotations of the current entity will be copied and applied to the newly created version.

/{versionable-entityType}/{entityId}/version

GET

Returns a list of all versions of this entity. This will be a paginated list with a default sort on versionNumber descending

/{versionable-entityType}/{entityId}/version/{versionNumber}

GET

Get a particular version of a version-able entity using the given id and version number.

/{versionable-entityType}/{entityId}/version/{versionNumber}

PUT

Not supported! Only the current version of an entity can be updated using the existing API.

/{versionable-entityType}/{entityId}/version/{versionNumber}

DELETE

Delete a particular version of a version-able entity using the given id and version number.

/{versionable-entityType}/{entityId}/version/{versionNumber}/annotations

GET

Get the annotations of a particular version of a version-able entity using the given id and version number.

/{versionable-entityType}/{entityId}/version/{versionNumber}/annotations

PUT

Not Supported! You can only change the annotations of the current version.

Examples

The following are examples of how the new version API will work.

Create Version-able Location

Request

curl -i  -H sessionToken:YourSessionToken -H Accept:application/json -H Content-Type:application/json -d '{
  "md5sum": "b513a23fc54b7b0d65312e1a900af5a6",
  "parentId": "1181",
  "path": "mskcc_prostate_cancer.phenotype.zip",
  "type": "awss3"
  "versionLabel ": "1.0.0"
  "versionComment": "The initial location for this file "
}' https://staging-reposervice.elasticbeanstalk.com/repo/v1/location

Response

HTTP/1.1 201 Created
Content-Type: application/json
Date: Wed, 15 Jun 2011 16:43:36 GMT
ETag: 1
Location: /repo/v1/location/1182
Server: Apache-Coyote/1.1
Content-Length: 466
Connection: keep-alive

{
  "accessControlList": "/repo/v1/location/1182/acl",
  "annotations": "/repo/v1/location/1182/annotations",
  "creationDate": 1308156216963,
  "etag": "1",
  "id": "1182",
  "md5sum": "b513a23fc54b7b0d65312e1a900af5a6",
  "name": "default",
  "parentId": "1181",
  "path": "https://s3.amazonaws.com/data01.sagebase.org/mskcc_prostate_cancer.phenotype.1.0.0.zip?Expires=1308242617&AWSAccessKeyId=AKIAJBJUALFYA3TUKPJA&Signature=kFKH7KbY%2FXITRVYVmveUD6tZq90%3D",
  "type": "awss3",
  "uri": "/repo/v1/location/1182"
  "versionLabel": "1.0.0"
  "versionId" : "1"
  "versionUrl" : "/repo/v1/location/1182/version/1"
  "versions" : "/repo/v1/location/1182/version"
  "versionComment": "The initial location for this file "

}
Create a New Version of the Location

Request

curl -i  -H sessionToken:YourSessionToken -H Accept:application/json -H Content-Type:application/json -d '{
  "md5sum": "b513a23fc54b7b0d65312e1a900af5a6",
  "parentId": "1181",
  "path": "mskcc_prostate_cancer.phenotype.zip",
  "type": "awss3"
  "versionLabel ": "1.0.1"
  "versionComment": "Added more data to this file and removed extra spaces from the header files."
}' https://staging-reposervice.elasticbeanstalk.com/repo/v1/location/1182/version

Response

HTTP/1.1 201 Created
Content-Type: application/json
Date: Wed, 15 Jun 2011 16:43:36 GMT
ETag: 1
Location: /repo/v1/location/1182
Server: Apache-Coyote/1.1
Content-Length: 466
Connection: keep-alive

{
  "accessControlList": "/repo/v1/location/1182/acl",
  "annotations": "/repo/v1/location/1182/annotations",
  "creationDate": 1308156216963,
  "etag": "1",
  "id": "1182",
  "md5sum": "b513a23fc54b7b0d65312e1a900af5a6",
  "name": "default",
  "parentId": "1181",
  "path": "https://s3.amazonaws.com/data01.sagebase.org/mskcc_prostate_cancer.phenotype.1.0.1.zip?Expires=1308242617&AWSAccessKeyId=AKIAJBJUALFYA3TUKPJA&Signature=kFKH7KbY%2FXITRVYVmveUD6tZq90%3D",
  "type": "awss3",
  "uri": "/repo/v1/location/1182"
  "versionLabel": "1.0.1"
  "versionId" : "2"
  "versionUrl" : "/repo/v1/location/1182/version/2"
  "versions" : "/repo/v1/location/1182/version"
  "versionComment": "Added more data to this file and removed extra spaces from the header files."
}
List all Versions of the Location

Request

curl -i  -H sessionToken:YourSessionToken -H Accept:application/json 'https://staging-reposervice.elasticbeanstalk.com/repo/v1/location/1182/version'

Response

HTTP/1.1 200 OK
Content-Type: application/json
Date: Wed, 15 Jun 2011 16:43:46 GMT
Server: Apache-Coyote/1.1
Content-Length: 512
Connection: keep-alive

{
  "paging": {},
  "results": [
    {
      "accessControlList": "/repo/v1/location/1182/acl",
      "annotations": "/repo/v1/location/1182/annotations",
      "creationDate": 1308156216963,
      "etag": "1",
      "id": "1182",
      "md5sum": "b513a23fc54b7b0d65312e1a900af5a6",
      "name": "default",
      "parentId": "1181",
      "path": "https://s3.amazonaws.com/data01.sagebase.org/mskcc_prostate_cancer.phenotype.1.0.1.zip?Expires=1308242617&AWSAccessKeyId=AKIAJBJUALFYA3TUKPJA&Signature=kFKH7KbY%2FXITRVYVmveUD6tZq90%3D",
      "type": "awss3",
      "uri": "/repo/v1/location/1182"
      "versionLabel": "1.0.1"
      "versionId" : "2"
      "versionUrl" : "/repo/v1/location/1182/version/2"
      "versions" : "/repo/v1/location/1182/version"
      "versionComment": "Added more data to this file and removed extra spaces from the header files."
    },
    {
      "accessControlList": "/repo/v1/location/1182/acl",
      "annotations": "/repo/v1/location/1182/version/1/annotations",
      "creationDate": 1308156216963,
      "etag": "1",
      "id": "1182",
      "md5sum": "b513a23fc54b7b0d65312e1a900af5a6",
      "name": "default",
      "parentId": "1181",
      "path": "https://s3.amazonaws.com/data01.sagebase.org/mskcc_prostate_cancer.phenotype.1.0.0.zip?Expires=1308242617&AWSAccessKeyId=AKIAJBJUALFYA3TUKPJA&Signature=kFKH7KbY%2FXITRVYVmveUD6tZq90%3D",
      "type": "awss3",
      "uri": "/repo/v1/location/1182"
      "versionLabel": "1.0.0"
      "versionId" : "1"
      "versionUrl" : "/repo/v1/location/1182/version/1"
      "versions" : "/repo/v1/location/1182/version"
      "versionComment": "Added more data to this file and removed extra spaces from the header files."
    }],
  "totalNumberOfResults": 2
}