Data Access Control

Background

Update:  The API has been extended to apply to Evaluations as well as Entities.

Update:  The API has been extended to apply to Teams.  See Teams

Update:  We allow updating of Access Requirements by the ACT.  We allow Access Requirements to apply to Folders as well as Files.   Entities inherit any access restrictions applies to Folders which contain them.

Update:  We add inheritance of access requirements:  Each entity inherits the access requirements of its ancestors (e.g. the Folder(s) containing a given File).

IRB-approved versions of all of the 'data access documents' :

Summary:

- Data layer access in Synapse requires one or more approval steps.  
- In Synapse granting data access is synonymous with providing the URL to the stored data.
    (This URL may have an embedded access token.)
- Prior to this work, the back end has a representation of EULAs and of Agreements (i.e. that a particular user agrees to a EULA)
- The work flow logic for creating the agreement is embedded in the Web client, so other clients would have to maintain duplicate logic. Specifically, the web client has the following logic:
1) When a user tries to download a layer, the Web client checks whether the parent dataset has an associate EULA;
2) If there is an EULA, the web client checks whether there is an Agreement, owned by the User and referencing the dataset and EULA;
3) If there is a EULA but no Agreement, the web client prompts the User to sign the EULA, creates the Agreement, then allows the download.

- There is no provision in our permissions scheme for an "ACT role" which can grant or revoke 'download permission' to a user.

- If the approval process changes, a user who has already been approved needs to go through the approval process again.

- Currently we've identified three tiers of access restriction/approval:

Tier 1: User agrees to a generic EULA that applies to all data layers available through Synapse.

Tier 2: (Tier 1) + User agrees to a second EULA specific to certain data layers.

Tier 3:  (Tier 1) + (Tier 2) + User access must be requested/approved through an Access and Compliance Team (ACT).

Design

Database

We have tables for Requirements and Approvals: ACCESS_REQUIREMENT contains the requirement information, while NODE_ACCESS_REQUIREMENT 'joins'  it to the JDONODE table, saying what entities are affected by the requirement.  ACCESS_APPROVAL has foreign keys to ACCESS_REQUIREMENT and principal.  The first imposes a requirement for access to the node.  The second fulfills the requirement, for a given principal. The requirement and approval tables have an "ENTITY_TYPE" field (populated from an ENUM) and a "SERIALIZED_ENTITY" field, a BLOB, allowing them to have variable content, so they can be used for Tier 2 or Tier 3 requirements.

Below we omit the primary key and foreign key constraints for simplicity.

CREATE TABLE `ACCESS_REQUIREMENT` (
  `ID` bigint(20) NOT NULL AUTO_INCREMENT,
  `ETAG` bigint(20) NOT NULL,
  `CREATED_BY` bigint(20) NOT NULL,
  `CREATED_ON` bigint(20) NOT NULL,
  `MODIFIED_BY` bigint(20) NOT NULL,
  `MODIFIED_ON` bigint(20) NOT NULL,
  `NODE_ID` bigint(20) NOT NULL,
  `ACCESS_TYPE` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
  `ENTITY_TYPE` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
  `SERIALIZED_ENTITY` mediumblob
 )

CREATE TABLE `NODE_ACCESS_REQUIREMENT` (
  `NODE_ID` bigint(20) NOT NULL,
  `REQUIREMENT_ID` bigint(20) NOT NULL,
  PRIMARY KEY (`NODE_ID`, `REQUIREMENT_ID`)
) 

CREATE TABLE `EVALAUTION_ACCESS_REQUIREMENT` (
  `EVALUATION_ID` bigint(20) NOT NULL,
  `REQUIREMENT_ID` bigint(20) NOT NULL,
  PRIMARY KEY (`NODE_ID`, `REQUIREMENT_ID`)
)

CREATE TABLE `ACCESS_APPROVAL` (
  `ID` bigint(20) NOT NULL AUTO_INCREMENT,
  `ETAG` bigint(20) NOT NULL,
  `CREATED_BY` bigint(20) NOT NULL,
  `CREATED_ON` bigint(20) NOT NULL,
  `MODIFIED_BY` bigint(20) NOT NULL,
  `MODIFIED_ON` bigint(20) NOT NULL,
  `REQUIREMENT_ID` bigint(20) NOT NULL,
  `ACCESSOR_ID`  bigint(20) NOT NULL,
  `ENTITY_TYPE` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
  `SERIALIZED_ENTITY` mediumblob
)

Data Access Object (DAO)

We have DAOs for AccessRequirement and AccessApproval objects.  In addition to basic CRUD operations we have:

• AccessRequirementDAO.getForNode(), which returns all the AccessRequirements associated with a Node/Entity
• AccessApprovalDAO.getForNode(), which returns all the AccessApprovals associatd with a Node/Entity.  This allows us to find all the users who have been given access to an entity via a single database query.
• AccessApprovalDAO.getForAccessRequirementsAndPrincipals() which returns the AccessApprovals for a given list of AccessRequirements and Principals.  This method allows us to look up the approval of all the access requirements for a given node (focussing on user of interest) with a single database query.

JSON Schema

We introduce JSON schemas for the generic interfaces AccessApproval and AccessRequirement, and schemas for specific types, TermsOfUseAccessRequirement, TermsOfUseAccessApproval (for tier 2 data), ACTAccessRequirement, and ACTAccessApproval (for tier 3 data).

Services

Web UI

When a user clicks Download on the page for a Data object having a Terms of Use access requirement, they are presented with a dialog showing the text from the access requirement, as shown below.  If they accept the terms, then an access approval is created and the Download link is presented.

When a user clicks Download on the page for a Data object having an ACT access requirement, they are presented with a dialog showing the text from the access requirement, as shown below.  Once they contact the ACT team, someone from the team may create the approval object on their behalf, after which they may download the Data without encountering the dialog.