Code Integration

...

R Convenience Functions

saveCode(localFolder, codeRepo)

parameters

localFolder: folder or file on the local file system

codeRepo short, easily remembered 'nickname' of code repository system. If omitted, user's default is used.

TODO: Do we need additional repository parameters (e.g. a folder within the repository) or can those be made part of the codeRepo object?

side effects

Register folder with code repository, if not already registered.

Commit code.

Create metadata in Synapse, if not already created.

TODO: What if not all steps are successful?

returns

ID of the object created in Synapse

Note: Code version is not recorded in the code metadata.

saveAndRun(localFolder, codeRepo)

parameters

localFolder: folder or file on the local file system

codeRepo short, easily remembered 'nickname' of code repository system. If omitted, user's default is used.

side effects

Register folder with code repository, if not already registered.

Commit code, retaining the version information.

Set a global "code.version" variable to the current version.

Create metadata in Synapse, if not already created.

"Source" code into R session.

Call "run()".

TODO: Check if code has changed before committing, to avoid making new version each time code is run.

returns

ID of the object created in Synapse

Note: Code version is not recorded in the code metadata.

loadCode(id, localFolder)

parameters

id: ID of the object in Synapse

localFolder: location for local copy of code. If omitted, then a default location is used.

side effects

Check out code from code repository, if not already checked out.  Otherwise, updates local copy from code repository.

"Source" code into R session.

returns

location of local copy of code

TODO: What if not all steps are successful?

deleteCode(id)

parameters

ID of the object in Synapse

side effects

Delete from the code repository.

Delete metadata from Synapse.

returns

Nothing.

TODO: What if not all steps are successful?
TODO: Should we delete from the local disk? From the R session?

TODO: Can we defer the convenience function for loading a specific code version?

Web Services

These manipulation only the metadata, stored in Synapse.
Note: It is assumed that code versions are represented by refererences.

Create Code

POST https://repo-staging.sagebase.org/repo/v1/code
{"parentId":"1234", "codeRepoId":"4567", "codeRepoLoc":"/trunk/analysisProject/R"}

Note: "parentId" is the ID of a Synapse Project; "codeRepoId" is the ID of a Code Repository object.

Successful Response:

HTTP/1.1 201 Created
{"id":"8901", "parentId":"1234", "codeRepoId":"4567", "codeRepoLoc":"/trunk/analysisProject/R"}

Get Code

GET https://repo-staging.sagebase.org/repo/v1/code/8901

Successful Response:

HTTP/1.1 200 OK
{"id":"8901", "parentId":"1234", "codeRepoId":"4567", "codeRepoLoc":"/trunk/analysisProject/R"}

Update Code

PUT https://repo-staging.sagebase.org/repo/v1/code/8901
{"id":"8901", "parentId":"5432", "codeRepoId":"4567", "codeRepoLoc":"/trunk/analysisProject/R"}

Successful Response:

HTTP/1.1 200 OK
{"id":"8901", "parentId":"5432", "codeRepoId":"4567", "codeRepoLoc":"/trunk/analysisProject/R"}

Delete Code

DELETE https://repo-staging.sagebase.org/repo/v1/code/8901

Successful Response:

HTTP/1.1 204 No Content

Create Code Repository

POST https://repo-staging.sagebase.org/repo/v1/codeRepo
{"codeRepoType":"svn", "nickname":"sageSvn", "codeRepoURL":"https://sagebionetworks.jira.com/svn"}

Successful Response:

HTTP/1.1 201 Created
{id:"9876", "codeRepoType":"svn", "codeRepoURL":https://sagebionetworks.jira.com/svn}

TODO: Error response if 'nickname' is already used.

TODO: GET, PUT, DELETE

Create Code Repo Credentials

POST https://repo-staging.sagebase.org/repo/v1/codeRepoCredentials
{"codeRepoId:"9876", "username":"foo": "password":"bar"}

Note: This request must be authenticated and the credentials are created for the authenticated user. The given "username" need not be the same as the user's Synapse username.

Successful Response:

HTTP/1.1 201 Created
{"id":"5432", "codeRepoId:"9876", "username":"foo": "password":"bar"}

TODO: Should the password not be 'echoed'?
TODO: GET, PUT, DELETE

Scenarios

...

Scenarios

The development cycle for preexisting code:  The code is Downloaded from Synapse, and Loaded into the R session.  The user repeatedly Runs, Edits and reLoads the code.  When satisfied, she may Store the code back to Synapse.  To help capture provenance, the Load and Store functions put the Code's ID and version into the current Step.  For the provenance record to be accurate, the user must Store prior to running.

  Image Added

Discover, retrieve, load and run code

A user, browsing Synapse, sees a piece of code of interest (mentioned in a discussion or used in an analysis), and clicks on it.  She sees:

# Load in Synapse R Client:
entity.3733 <- loadCodedownloadEntity(3733)

and begins by executing this in R.

(Note:  "Repo" is the code repository client.  The interaction between the repository client and server is not shown.)

Retrieve existing code, modify and save.

Image Removed

Develop new code, then

...

(Per Matt F., this is an important use case for data curators.)
Image Removed

Run code and capture provenance info.

...

Store

Image Added