Document toolboxDocument toolbox

Proposed Changes to Location API

Owned by John Hill
Last updated: Aug 30, 2011Version comment

Add a Location to a Layer

First create the presigned a S3 URL to use for upload.

Request 

curl -i  -H sessionToken:YourSessionToken -H Accept:application/json -H Content-Type:application/json -d '{
  "md5sum": "b513a23fc54b7b0d65312e1a900af5a6",
  "fileName": "mskcc_prostate_cancer.phenotype.zip",
}' https://repo-staging.sagebase.org/repo/v1/createNewS3Url

Response 

HTTP/1.1 201 Created
Content-Type: application/json
Date: Mon, 29 Aug 2011 23:55:11 GMT
ETag: 0
Location: /repo/v1/location/8237
Server: Apache-Coyote/1.1
transfer-encoding: chunked
Connection: keep-alive

{
  "presignedUrl": "https://s3.amazonaws.com/stagingdata.sagebase.org/8237/mskcc_prostate_cancer.phenotype.zip?Expires=1314748511&AWSAccessKeyId=AKIAJO2DMB3LTOB6IJOA&Signature=%2BryKGGJujZxtliiynGDZHyzqETU%3D",
}

The PUT the data to S3

Then PUT the data to S3 via an HTTP Client that supports multipart upload. Note that you must:

  1. Add header Content-Type: application/binary
  2. Add header Content-MD5: <the base64 encoded md5 checksum>
  3. Add header x-amz-acl: bucket-owner-full-control

Here is an example with curl:

curl -v -X PUT -H Content-MD5:tROiP8VLew1lMS4akAr1pg== -H x-amz-acl:bucket-owner-full-control \
 -H Content-Type:application/binary  --data-binary @<localFilepath> \
'https://s3.amazonaws.com/stagingdata.sagebase.org/8237/mskcc_prostate_cancer.phenotype.zip?Expires=1314748511&AWSAccessKeyId=AKIAJO2DMB3LTOB6IJOA&Signature=%2BryKGGJujZxtliiynGDZHyzqETU%3D'

Create the location using the URL as the path

Request 

curl -i  -H sessionToken:YourSessionToken -H Accept:application/json -H Content-Type:application/json -d '{
  "md5sum": "b513a23fc54b7b0d65312e1a900af5a6",
  "parentId": "8236",
  "path": "https://s3.amazonaws.com/stagingdata.sagebase.org/8237/mskcc_prostate_cancer.phenotype.zip?Expires=1314748511&AWSAccessKeyId=AKIAJO2DMB3LTOB6IJOA&Signature=%2BryKGGJujZxtliiynGDZHyzqETU%3D",
  "type": "awss3"
}' https://repo-staging.sagebase.org/repo/v1/location

Response 

HTTP/1.1 201 Created
Content-Type: application/json
Date: Mon, 29 Aug 2011 23:55:11 GMT
ETag: 0
Location: /repo/v1/location/8237
Server: Apache-Coyote/1.1
transfer-encoding: chunked
Connection: keep-alive

{
  "accessControlList": "/repo/v1/location/8237/acl",
  "annotations": "/repo/v1/location/8237/annotations",
  "contentType": "application/zip",
  "creationDate": 1314662110686,
  "etag": "0",
  "id": "8237",
  "md5sum": "b513a23fc54b7b0d65312e1a900af5a6",
  "name": "8237",
  "parentId": "8236",
  "path": "https://s3.amazonaws.com/stagingdata.sagebase.org/8237/mskcc_prostate_cancer.phenotype.zip?Expires=1314748511&AWSAccessKeyId=AKIAJO2DMB3LTOB6IJOA&Signature=%2BryKGGJujZxtliiynGDZHyzqETU%3D",
  "type": "awss3",
  "uri": "/repo/v1/location/8237",
  "versionComment": null,
  "versionLabel": "0.0.0",
  "versionNumber": 1,
  "versionUrl": "/repo/v1/location/8237/version/1",
  "versions": "/repo/v1/location/8237/version"
}

Update an existing location

First create a new presigned S3 URL to use for upload.

Request 

curl -i  -H sessionToken:YourSessionToken -H Accept:application/json -H Content-Type:application/json -d '{
  "md5sum": "b513a23fc54b7b0d65312e1a900af5a6",
  "fileName": "mskcc_prostate_cancer.phenotype2.zip",
}' https://repo-staging.sagebase.org/repo/v1/createS3Url

Response 

HTTP/1.1 201 Created
Content-Type: application/json
Date: Mon, 29 Aug 2011 23:55:11 GMT
ETag: 0
Location: /repo/v1/location/8237
Server: Apache-Coyote/1.1
transfer-encoding: chunked
Connection: keep-alive

{
  "presignedUrl": "https://s3.amazonaws.com/stagingdata.sagebase.org/8238/mskcc_prostate_cancer.phenotype2.zip?Expires=1314748511&AWSAccessKeyId=AKIAJO2DMB3LTOB6IJOA&Signature=%2BryKGGJujZxtliiynGDZHyzqETU%3D",
}

Upload the file to S3, the same as above.

See: above

Update the location with the new URL

Request 

curl -i  -H sessionToken:YourSessionToken -H Accept:application/json -H Content-Type:application/json -d '{
  "accessControlList": "/repo/v1/location/8237/acl",
  "annotations": "/repo/v1/location/8237/annotations",
  "contentType": "application/zip",
  "creationDate": 1314662110686,
  "etag": "0",
  "id": "8237",
  "md5sum": "b513a23fc54b7b0d65312e1a900af5a6",
  "name": "8237",
  "parentId": "8236",
  "path": "https://s3.amazonaws.com/stagingdata.sagebase.org/8238/mskcc_prostate_cancer.phenotype2.zip?Expires=1314748511&AWSAccessKeyId=AKIAJO2DMB3LTOB6IJOA&Signature=%2BryKGGJujZxtliiynGDZHyzqETU%3D",
  "type": "awss3",
  "uri": "/repo/v1/location/8237",
  "versionComment": null,
  "versionLabel": "0.0.0",
  "versionNumber": 1,
  "versionUrl": "/repo/v1/location/8237/version/1",
  "versions": "/repo/v1/location/8237/version"
}' https://repo-staging.sagebase.org/repo/v1/location

Response 

HTTP/1.1 201 Created
Content-Type: application/json
Date: Mon, 29 Aug 2011 23:55:11 GMT
ETag: 0
Location: /repo/v1/location/8237
Server: Apache-Coyote/1.1
transfer-encoding: chunked
Connection: keep-alive

{
  "accessControlList": "/repo/v1/location/8237/acl",
  "annotations": "/repo/v1/location/8237/annotations",
  "contentType": "application/zip",
  "creationDate": 1314662110686,
  "etag": "0",
  "id": "8237",
  "md5sum": b513a23fc54b7b0d65312e1a900af5a6",
  "name": "8237",
  "parentId": "8236",
  "path": "https://s3.amazonaws.com/stagingdata.sagebase.org/8238/mskcc_prostate_cancer.phenotype2.zip?Expires=1314748511&AWSAccessKeyId=AKIAJO2DMB3LTOB6IJOA&Signature=%2BryKGGJujZxtliiynGDZHyzqETU%3D",
  "type": "awss3",
  "uri": "/repo/v1/location/8237",
  "versionComment": null,
  "versionLabel": "0.0.0",
  "versionNumber": 1,
  "versionUrl": "/repo/v1/location/8237/version/1",
  "versions": "/repo/v1/location/8237/version"
}