NOTE: These instructions run into problems with backslashes for R Client versions older than 1.12. Please ensure you have an up-to-date version of the software.
This page contains the instructions to be followed by the Access and Compliance team for managing access restrictions on data in Synapse.
The design details for the system are here: https://sagebionetworks.jira.com/wiki/display/PLFM/Data+Access+Control
In the initial release of Data Access Control ("Governance"), restrictions are managed from the command line rather than through the Synapse web interface. Here the commands are presented in the form of "R" commands. In all cases the user performing the Governance tasks must have administrative access to the object being managed.
Update: To implement PLFM-1730, we allow multiple access restrictions to be placed on an entity. Synapse records the order in which you create access restrictions. If there are multiple, unfulfilled restrictions, the Synapse Web interface will display the first of these, along with instructions for the user to seek approval for that restriction. If all restrictions are fulfilled, then Synapse will show the first one created. The data restriction level shown for the Synapse entity is the most restrictive of all the restrictions attached to the entity. For example if one Restricted (Terms of Use) restriction and one Controlled (ACT Approval) restriction are applied to an entity, then the entity will be displayed as Controlled.
How to set up the Synapse R client on your machine:
# these two lines install the R Synapse Client. You only need do this once
source('http://depot.sagebase.org/CRAN.R')
pkgInstall("synapseClient")
# This has to be done each time you start up R:
library(synapseClient)
# one time call:
synapseLogin("username","password", rememberMe=TRUE) # thereafter you can simply call "synapseLogin()" without typing your credentials
How to remove the 'default' access requirement and create new restrictions
library(synapseClient)
synapseLogin(<your user name>, <your password>, rememberMe=TRUE) # later you can log in with "synapseLogin()"
#######################################
#Add Restricted (Tier 2) Access Control to syn1446624
########################################
entityIds<-list("syn1446624")
##########Step 1. Remove the access requirement set by JIRA when the data owner requested restrictions.
#Automatic restriction is: "Access restricted pending review by Synapse Access and Compliance Team."
#Identify the 'restriction id' for the above restriction
currentRestrictions<-synRestGET(paste("/entity/", entityIds[[1]], "/accessRequirement", sep=""))
#$results[[1]]$id
#[1] 1426942
#remove this restriction
requirementId<-currentRestrictions$results[[1]]$id
synRestDELETE(paste("/accessRequirement/", requirementId, sep=""))
###AT THIS POINT, THE DATA IS COMPLETELY OPEN BECAUSE YOU REMOVED THE FORMER RESTRICTION AND HAVE NOT ADDED THE NEW ONE.
# Step 2. Create the new restriction.
subjectIds<-lapply(entityIds, function(x){list(id=x,type="ENTITY")})
ar <- list(concreteType="org.sagebionetworks.repo.model.TermsOfUseAccessRequirement",
subjectIds=subjectIds, accessType="DOWNLOAD",
termsOfUse="The terms under which this dataset may be accessed require that you agree to use this data only for neuropsychiatric research performed for non-commercial purposes.")
ar<-synRestPOST("/accessRequirement", ar)
#######################################
#Add Controlled (Tier 3) Access Control to syn1446623
########################################
# use the synapse identifier for the data object to be put under access control
entityIds<-list("syn1446623")
##########Step 1. Remove the access requirement created when the data owner requested restrictions.
#Automatic restriction is: "Access restricted pending review by Synapse Access and Compliance Team."
#Identify the 'restriction id' for the above restriction
currentRestrictions<-synRestGET(paste("/entity/", entityIds[[1]], "/accessRequirement", sep=""))
#$results[[1]]$id
#[1] 1426942
#remove this restriction
requirementId<-currentRestrictions$results[[1]]$id
synRestDELETE(paste("/accessRequirement/", requirementId, sep=""))
###AT THIS POINT, THE DATA IS COMPLETELY OPEN BECAUSE YOU REMOVED THE FORMER RESTRICTION AND HAVE NOT ADDED THE NEW ONE.
# # create the Tier Three access requirement
subjectIds<-lapply(entityIds, function(x){list(id=x,type="ENTITY")})
openJiraIssue<-TRUE # set to true or false to control whether the web portal allows the user to open a JIRA issue
actContactInfo<-"Access to these data is controlled at the request of the data contributor and due to the sensitive nature of the data. <br/> <br/> Please contact the Access and Compliance Team for instruction, by clicking below, if you are interested in applying for access to these data."
ar <- list(concreteType="org.sagebionetworks.repo.model.ACTAccessRequirement",
subjectIds=subjectIds, accessType="DOWNLOAD",
actContactInfo=actContactInfo,
openJiraIssue=openJiraIssue)
ar<-synRestPOST("/accessRequirement", ar)
How to add a 'restricted' access requirement to a file:
# log in to Synapse
library(synapseClient)
synapseLogin(<your user name>, <your password>)
# use the synapse identifier for the data object to be put under access control
entityIds<-list("syn987654")
# create the Terms of Use access requirement
subjectIds<-lapply(entityIds, function(x){list(id=x,type="ENTITY")})
ar <- list(concreteType="org.sagebionetworks.repo.model.TermsOfUseAccessRequirement", subjectIds=subjectIds, accessType="DOWNLOAD", termsOfUse="The terms under which this dataset may be accessed require that you agree not to transfer or disclose the Data, in whole or part, or any material derived from the Data, to others, except to persons within your Institution.")
ar<-synRestPOST("/accessRequirement", ar)
When someone tries to download the data from the Web interface, they will be presented with a agreement which they must accept before they are able to complete the download.
How to add a 'restricted' access requirement ('click wrap') to a Team:
Putting access requirements on the team will result in the terms of use being displayed as a click-through when a participant presses a (properly configured) register button.
# log in to Synapse
library(synapseClient)
synapseLogin(<your user name>, <your password>)
# use the synapse identifier for the Team to be put under the 'click wrap'
teamIds<-list("1234567")
# create the access requirement
subjectIds<-lapply(teamIds, function(x){list(id=x,type="TEAM")})
ar <- list(concreteType="org.sagebionetworks.repo.model.TermsOfUseAccessRequirement", subjectIds=subjectIds, accessType="PARTICIPATE",
termsOfUse="<click wrap goes here>")
ar<-synRestPOST("/accessRequirement", ar)
Here's an example terms of use object that contains the DREAM 10 rules: (But, see SYNR-990)
ar <- list(
concreteType="org.sagebionetworks.repo.model.TermsOfUseAccessRequirement",
subjectIds=list(list(id="1234567", type="TEAM")),
accessType="PARTICIPATE",
termsOfUse='<div class="markdown" style="margin-left:20px; margin-right:20px;">\t\t<h3>Please read the full <a href="https://www.synapse.org/#!Synapse:syn4294018/wiki/" class="link" target="_blank">DREAM 10 Challenge Rules</a>.</h3>\t\t<h3 style="margin-bottom:10px">These rules cover:</h3>\t\t<div class="row" style="height: 90px;">\t\t\t<div class="col-xs-6">\t\t\t\t<img class="left" src="https://s3.amazonaws.com/static.synapse.org/images/person.png" /><h5 style="padding-top: 25px; text-align:left;">Who can participate</h5>\t\t\t</div>\t\t\t<div class="col-xs-6">\t\t\t\t<img class="left" src="https://s3.amazonaws.com/static.synapse.org/images/certificate.png" /><h5 style="padding-top: 5px; text-align:left;">What you need to submit to be evaluated as a best performer</h5>\t\t\t</div>\t\t</div>\t\t<div class="row" style="height: 110px;">\t\t\t<div class="col-xs-6">\t\t\t\t<img class="left" src="https://s3.amazonaws.com/static.synapse.org/images/people.png" /><h5 style="padding-top: 5px; text-align:left;">How to participate as an individual and on teams</h5>\t\t\t</div>\t\t\t<div class="col-xs-6">\t\t\t\t<img class="left" src="https://s3.amazonaws.com/static.synapse.org/images/anonymous-person.png" style="margin-bottom: 20px;" /><h5 style="text-align:left;">Your right to participate anonymously <small>To participate anonymously, you must update your Synapse username to a pseudonym.</small></h5>\t\t\t</div>\t\t</div>\t\t<div class="row" style="height: 80px;">\t\t\t<div class="col-xs-6">\t\t\t\t<img class="left" src="https://s3.amazonaws.com/static.synapse.org/images/unlock.png" /><h5 style="text-align:left;">That DREAM Challenges are IP-free and open source</h5>\t\t\t</div>\t\t\t<div class="col-xs-6">\t\t\t\t<img class="left" src="https://s3.amazonaws.com/static.synapse.org/images/handshake.png" style="margin-bottom: 20px;" /><h5 style="text-align:left;">How we will resolve disagreements with participants</h5>\t\t\t</div>\t\t</div>\t\t<h5>By clicking "Accept" you confirm that you have read and accept the DREAM 10 Challenge Rules</h5></div>')
ar<-synRestPOST("/accessRequirement", ar)
How to add a 'controlled' access requirement to a file:
# log in to Synapse
library(synapseClient)
synapseLogin(<your user name>, <your password>)
# use the synapse identifier for the data object to be put under access control
entityIds<-list("syn987654")
# # create the Tier Three access requirement
subjectIds<-lapply(entityIds, function(x){list(id=x,type="ENTITY")})
openJiraIssue<-TRUE # set to true or false to control whether the web portal allows the user to open a JIRA issue
actContactInfo<-"Please complete the following form and email it to the Access and Compliance Team at act@sagebase.org.<br/> 1) Your name:<br/> 2) Your organization: <br/> 3) Your IRB number: <br/> "
ar <- list(concreteType="org.sagebionetworks.repo.model.ACTAccessRequirement", subjectIds=subjectIds, accessType="DOWNLOAD", actContactInfo=actContactInfo, openJiraIssue=openJiraIssue)
ar<-synRestPOST("/accessRequirement", ar)
When someone tries to download the data from the Web interface, they will be presented with instructions for contacting the Access and Compliance Team to request download access.
How to link to an approval form to an access requirement:
In the actContactInfo, ensure that any anchor tags (<a>) include the attribute target="blank", e.g.:
actContactInfo<-"Please complete this form and send to act@sagebase.org:<br/><a href=\"https://staging.synapse.org/#!Synapse:syn2295117\" target=\"_blank\">https://staging.synapse.org/#!Synapse:syn2295117</a>"
This is necessary to make the browser open a new tab to show the approval form.
How to delete an existing access requirement:
If a requirement is deleted, all the approvals for said requirement are deleted as well. To delete from R:
requirementId<-"7"
synRestDELETE(paste("/accessRequirement/", requirementId, sep=""))
How to update an existing access requirement:
An access requirement may be changed after it is created. The main use for this is to add files to an existing requirement. You can also change the displayed text.
# Say there is currently an access restriction on just one file, syn2319165.
> entityIds<-list("syn2319165")
> currentRestrictions<-synRestGET(paste("/entity/", entityIds[[1]], "/accessRequirement", sep=""))
# Check that there is just one restriction
> currentRestrictions$totalNumberOfResults
[1] 1
# set 'ar' to be the restriction
> ar<-currentRestrictions$results[[1]]
# now verify what is the current entity list for the access restriction
> ar$subjectIds
[[1]]
id type
"syn2319165" "ENTITY"
# Now let's make the restriction apply to two files rather than one.
> entityIds<-list("syn2319165", "syn2341872")
> subjectIds<-lapply(entityIds, function(x){list(id=x,type="ENTITY")})
> ar$subjectIds<-subjectIds
#
#
#
# Optionally change the text of the access requirement
# If the Access Requirement is an "ACT" (Tier 3) Access Requirement, then the field is called "actContactInfo":
ar$actContactInfo<-"new ACT contact info"
#
# -- OR --
#
# If the Access Requirement is a "Terms of use" ("click wrap") Access Requirement, then the field is called "termsOfUse":
ar$termsOfUse<-"new click-wrap text"
#
#
# (The above two fields are the only mutable ones: the list of entities and the displayed text.)
#
# This command sends the modified restriction to Synapse.
> ar<-synRestPUT(paste("/accessRequirement/", ar$id, sep=""), ar)
> ar$subjectIds
[[1]]
id type
"syn2319165" "ENTITY"
[[2]]
id type
"syn2341872" "ENTITY"
How to list the access requirements for a data object:
Use the following function, which you can cut/paste into your R session:
listRequirementsForEntity<-function(entityId) {
ars<-synRestGET(sprintf("/entity/%s/accessRequirement", entityId))$results
for (ar in ars) {
if (ar$concreteType=="org.sagebionetworks.repo.model.ACTAccessRequirement") {
text<-ar$actContactInfo
} else {
text<-ar$termsOfUse
}
cat(sprintf("id=%s type=%s text=%s...\n", ar$id, substring(text=ar$concreteType, first=1+nchar("org.sagebionetworks.repo.model.")), substr(text, 1, 50)))
}
}
How to find out who has access to a data object:
Use the following function, which you can cut/paste into your R session:
whoHasAccess<-function(entityId) {
aas<-synRestGET(sprintf("/entity/%s/accessApproval", entityId))$results
ups<-list()
approvedUsers<-list()
cat(sprintf("There are %d access approvals for entity %s\n", length(aas), entityId))
for (aa in aas) {
# cat(sprintf("approval id=%s requirement id=%s, user id=%s\n", aa$id, aa$requirementId, aa$accessorId))
userId<-aa$accessorId
reqId<-aa$requirementId
if (!any(approvedUsers[[userId]]==reqId)) {
approvedUsers[[userId]]<-append(approvedUsers[[userId]], reqId)
}
if (is.null(ups[[userId]])) {
ups[[userId]]<-synRestGET(sprintf("/userProfile/%s", userId))
}
cat(".")
}
cat("\n")
for (userId in names(approvedUsers)) {
displayName<-ups[[userId]]$displayName
userName<-ups[[userId]]$userName
if (is.null(displayName)) {
cat(sprintf("%s (id=%s) is approved for access requirement(s) %s\n", userName, userId,
paste(approvedUsers[[userId]], collapse=",")))
} else {
cat(sprintf("%s (username=%s, id=%s) is approved for access requirement(s) %s\n", displayName, userName, userId,
paste(approvedUsers[[userId]], collapse=",")))
}
}
}
How to find out if a specific user has access to a data object:
Use the following function, which you can cut/paste into your R session:
doesUserHaveAccess<-function(entityId, principalId) {
aas<-synRestGET(sprintf("/entity/%s/accessApproval", entityId))$results
approvedRequirements<-list()
cat(sprintf("There are %d access approvals for entity %s\n", length(aas), entityId))
for (aa in aas) {
# cat(sprintf("approval id=%s requirement id=%s, user id=%s\n", aa$id, aa$requirementId, aa$accessorId))
userId<-aa$accessorId
reqId<-aa$requirementId
if (userId==principalId && !any(approvedRequirements==reqId)) {
approvedRequirements<-append(approvedRequirements, reqId)
cat(sprintf("\nuser %s was approved for requirement %s by %s on %s\n", principalId, reqId, aa$createdBy, aa$createdOn))
}
cat(".")
}
cat("\n")
allRequirements<-list()
ars<-synRestGET(sprintf("/entity/%s/accessRequirement", entityId))$results
for (ar in ars) {
allRequirements<-append(allRequirements, ar$id)
}
if (length(allRequirements)>0) allRequirements<-sort(unlist(allRequirements))
if (length(approvedRequirements)>0) approvedRequirements<-sort(unlist(approvedRequirements))
if (identical(allRequirements, approvedRequirements)) {
message(sprintf("User %s DOES have access approvals for ALL %d access requirement(s) on %s",
principalId, length(allRequirements), entityId))
} else {
if (length(approvedRequirements>0)) {
message(sprintf("User %s does NOT have access approvals for all the %d access requirement(s) on %s, only for %s",
principalId, length(allRequirements), entityId, paste(approvedRequirements, collapse=",")))
} else {
message(sprintf("User %s does NOT have access approvals for ANY of the %d access requirement(s) on %s",
principalId, length(allRequirements), entityId))
}
}
}
(TODO: We can also display the date/time when approval was granted.)
How to grant access for a specific access requirement:
Use the following function, which you can cut/paste into your R session:
grantAccess<-function(requirementId, principalId) {
actApproval <- list(concreteType="org.sagebionetworks.repo.model.ACTAccessApproval", requirementId=requirementId, accessorId=principalId, approvalStatus="APPROVED")
actApproval<-synRestPOST("/accessApproval", actApproval)
}
How to remove access to a data object:
The following deletes access approvals for all requirements found on the object, for the given user.
Use the following function, which you can cut/paste into your R session:
removeAccess<-function(entityId, principalId) {
aas<-synRestGET(sprintf("/entity/%s/accessApproval", entityId))$results
cat(sprintf("There are %d access approvals for entity %s\n", length(aas), entityId))
for (aa in aas) {
# cat(sprintf("approval id=%s requirement id=%s, user id=%s\n", aa$id, aa$requirementId, aa$accessorId))
userId<-aa$accessorId
reqId<-aa$requirementId
if (userId==principalId) {
synRestDELETE(sprintf("/accessApproval/%s", aa$id))
cat(sprintf("\nRemoved access approval for access requirement %s.\n", reqId))
}
cat(".")
}
cat("\n")
}
How to list the access requirements for a Team:
Use the following function, which you can cut/paste into your R session:
listRequirementsForTeam<-function(teamId) {
ars<-synRestGET(sprintf("/team/%s/accessRequirement", teamId))$results
for (ar in ars) {
if (ar$concreteType=="org.sagebionetworks.repo.model.ACTAccessRequirement") {
text<-ar$actContactInfo
} else {
text<-ar$termsOfUse
}
cat(sprintf("id=%s type=%s text=%s...\n", ar$id, substring(text=ar$concreteType, first=1+nchar("org.sagebionetworks.repo.model.")), substr(text, 1, 50)))
}
}
How to find out who was approved to join a Team:
Use the following function, which you can cut/paste into your R session:
whoHasAccessToTeam<-function(teamId) {
aas<-synRestGET(sprintf("/team/%s/accessApproval", teamId))$results
ups<-list()
approvedUsers<-list()
cat(sprintf("There are %d access approvals for Team %s\n", length(aas), teamId))
for (aa in aas) {
userId<-aa$accessorId
reqId<-aa$requirementId
if (!any(approvedUsers[[userId]]==reqId)) {
approvedUsers[[userId]]<-append(approvedUsers[[userId]], reqId)
}
if (is.null(ups[[userId]])) {
ups[[userId]]<-synRestGET(sprintf("/userProfile/%s", userId))
}
cat(".")
}
cat("\n")
for (userId in names(approvedUsers)) {
displayName<-ups[[userId]]$displayName
userName<-ups[[userId]]$userName
if (is.null(displayName)) {
cat(sprintf("%s (id=%s) is approved for access requirement(s) %s\n", userName, userId,
paste(approvedUsers[[userId]], collapse=",")))
} else {
cat(sprintf("%s (username=%s, id=%s) is approved for access requirement(s) %s\n", displayName, userName, userId,
paste(approvedUsers[[userId]], collapse=",")))
}
}
}
How to find out if a certain user was approved to join a Team:
Use the following function, which you can cut/paste into your R session:
isUserApprovedToJoinTeam<-function(teamId, principalId) {
aas<-synRestGET(sprintf("/team/%s/accessApproval", teamId))$results
approvedRequirements<-list()
cat(sprintf("There are %d access approvals for team %s\n", length(aas), teamId))
for (aa in aas) {
userId<-aa$accessorId
reqId<-aa$requirementId
if (userId==principalId && !any(approvedRequirements==reqId)) {
approvedRequirements<-append(approvedRequirements, reqId)
cat(sprintf("\nuser %s was approved for requirement %s by %s on %s\n", principalId, reqId, aa$createdBy, aa$createdOn))
}
cat(".")
}
cat("\n")
allRequirements<-list()
ars<-synRestGET(sprintf("/team/%s/accessRequirement", teamId))$results
for (ar in ars) {
allRequirements<-append(allRequirements, ar$id)
}
if (length(allRequirements)>0) allRequirements<-sort(unlist(allRequirements))
if (length(approvedRequirements)>0) approvedRequirements<-sort(unlist(approvedRequirements))
if (identical(allRequirements, approvedRequirements)) {
message(sprintf("User %s DOES have access approvals for ALL %d access requirement(s) on %s",
principalId, length(allRequirements), teamId))
} else {
if (length(approvedRequirements>0)) {
message(sprintf("User %s does NOT have access approvals for all the %d access requirement(s) on %s, only for %s",
principalId, length(allRequirements), teamId, paste(approvedRequirements, collapse=",")))
} else {
message(sprintf("User %s does NOT have access approvals for ANY of the %d access requirement(s) on %s",
principalId, length(allRequirements), teamId))
}
}
}
How to prevent download from a participant project in the ALS challenge
First, visit the project and verify that download access is unblocked at this time: Go to any file in the project in the Synapse portal, logged in with NON-ACT credentials, and check that you can download.
Now do the following in 'R':
library(synapseClient)
synapseLogin()
# the following line is an example. Customize as needed
projectsToBlock<-c("syn123456", "syn987654")
accessRequirement<-synRestGET("/accessRequirement/5360411")
projectsCurrentlyBlocked<-sapply(accessRequirement$subjectIds, function(x) x$id)
updatedProjectList<-c(projectsCurrentlyBlocked, projectsToBlock)
subjectIds<-lapply(updatedProjectList, function(x){list(id=x,type="ENTITY")})
accessRequirement$subjectIds<-subjectIds
accessRequirement<-synRestPUT("/accessRequirement/5360411", accessRequirement)
# now get the updated list of restricted projects
projectsCurrentlyBlocked<-sapply(accessRequirement$subjectIds, function(x) x$id)
projectsCurrentlyBlocked
Finally, return to the project(s) which you have 'blocked' in the Synapse web portal and check that the download restriction is in place: Visit any file in the project. You will see the "Controlled Use" notice and if you click "show" there is a message saying that download is suspended.
How to 'unblock' download from a participant project in the ALS challenge
First, visit the project and verify that download access is prevented: Go to any file in the project in the Synapse portal. You will see the "Controlled Use" notice and if you click "show" there is a message saying that download is suspended. (Note: Download restrictions do not apply to ACT members, so if you are logged in to Synapse with an ACT account you will not be prevented from downloading.)
Now do the following in 'R':
library(synapseClient)
synapseLogin()
# the following line is an example. Customize as needed
projectsToUnblock<-c("syn123456", "syn987654")
accessRequirement<-synRestGET("/accessRequirement/5360411")
projectsCurrentlyBlocked<-sapply(accessRequirement$subjectIds, function(x) x$id)
updatedProjectList<-projectsCurrentlyBlocked[sapply(projectsCurrentlyBlocked, function(x,y) all(x!=y), projectsToUnblock)]
subjectIds<-lapply(updatedProjectList, function(x){list(id=x,type="ENTITY")})
accessRequirement$subjectIds<-subjectIds
accessRequirement<-synRestPUT("/accessRequirement/5360411", accessRequirement)
# now get the updated list of restricted projects
projectsCurrentlyBlocked<-sapply(accessRequirement$subjectIds, function(x) x$id)
projectsCurrentlyBlocked
Finally, return to the project(s) which you have 'unblocked' in the Synapse web portal and check that the download restriction is gone.
Entity ID List:
MetabricEntityID<-"syn2133321" OslovalEntityID<-"syn1710393"
IRB/Ethics Board Approval Requirements ID List:
MetabricrequirementId <- "2161694" OslovalrequirementId <- "2227942"
General Notes:
EntityIDs and RequirementsIDs have syn before them and use quotation marks.
PrincipalIDs do not have syn before them and do not use quotation marks.
Incomplete Code to Complete Tasks:
For non-programming types - Often the code provided above does not include a final line to run it.
For example, when removing access you need to a line that :
## after running the code above for removing a person's access to an entity, type this line with the correct entityID and principalID
removeAccess("syn123456","789012")
Using Python instead of R:
The beginnings of an equivalent set of instructions using the Python client can be found at syn5040514. Please add to these!