COVID Recovery Corps integration design

In this document we’ll describe the integration and interaction between the systems that will be used to implement the first phase, with an eye toward evolving the system in the future.

Our external partner for this work is Columbia University Irving Medical Center (CUIMC) OHC. Their system is a part of the overall workflow we are supporting for this project. They have requested that we use HTTP Basic Authentication to authenticate their calls. We will need to create endpoints where we validate their credentials and perform some work. All the functionality exists in Bridge, it would just need to be orchestrated differently.

Questions for CUIMC: is there any rate-limiting? (They call us now in all cases, so this hasn’t been an issue.)

Workflow State

The Bridge Account object should be tagged with one (and only one) of the following data groups:

State	Description	Transitions to

State	Description	Transitions to
<no state>	Accounts should be created with no state. The account exists, but is not consented to participate in the study.	enrolled
enrolled	When the participant consents to the default consent, we add the `enrolled` data group. They are available for researchers to examine in order to determine if they should be asked for further lab tests.	selected, declined
declined	Researchers have decided this person should not be asked to have lab work done
selected	Researchers have indicated this person should get lab work done.	tests_requested
tests_requested	The worker process or API call to Bridge has created a lab order with the lab, which triggers a message from the lab to go schedule an appointment.	tests_scheduled
tests_scheduled	CUIMC has verified that the participant has scheduled to have their lab work done. Save this information as a participant report (“appointment” at 1970-01-01) and update the participant to `tests_collected`. We can show this appointment information to the user in our app, and to researchers. Hopefully, we can link back to CUIMC’s system to cancel or reschedule.	tests_collected
tests_cancelled	CUIMC has sent an appointment record with a status field of “cancelled.” (The other known state is “booked” for when an appointment has been scheduled).	Possibly back to tests_scheduled if user reschedules
tests_collected	CUIMC has verified that the participant has had lab specimens collected, and they are currently being processed. Save this information as a participant report (“collection” at 1970-01-01) and update the participant to `tests_available`.	tests_available
tests_available	CUIMC has verified that tests results are available by submitting them to Bridge, where they have been saved as health data. Bridge will be able to show these results through the patient portal using access to health data raw data API being developed. We also update the participant to `tests_notified` (the CUIMC system will notify the user that results are available).	tests_notified
tests_available_type_unknown	CUIMC has sent us observation JSON, but it’s for a test of a type we don’t know and didn’t expect to receive.
tests_notified	The UI can look at this state and adjust the UI to highlight test results, as this is one of the main benefits for participants (the test results).

There’s a state_change_timestamp attribute in each participant, indicating the last time that either Synapse or CUIMC have updated the participant record through the CRC specific endpoints. The state transitions occur through the following agents:

State	Agent & API

State	Agent & API
<no state>	The participant completes the consent, and the `enrolled` data group is added to the account. These are the accounts that will be selected for review by the research team. The participant is asked to answer some surveys and upload any evidence they have of their recovered status (e.g. a photo of a lab report). API: POST /v3/healthdata [survey answers] API: POST /v3/uploads [binaries of lab tests]
enrolled	The research team signs into a Synapse UI and reviews all the `enrolled` accounts, looking at their file uploads and survey answers, and marks them as `declined` or `selected`. API: POST /v3/participants/{userId} API for user to retrieve raw data of health records API for researchers to retrieve raw data of health records
declined	<end state>
selected	A script written by Megha periodically depends this in Synapse and calls to update the Bridge participant by calling the /v1/cuimc/participants/{userId}/laborders endpoint. This marks the Bridge account as `selected`. Periodically, CUIMC polls the Bridge server to search for `selected` accounts, and sends these accounts invitations to The worker process finds `selected` accounts and files lab orders. They clear the `selected` data group and send an email to the user requesting that they schedule an appointment, and set the Bridge participant to `tests_requested`. API: POST /v3/participants/{userId} [worker process is a researcher + worker account]
tests_requested	The user must then schedule for lab work. When this is completed, the CUIMC system will call back Bridge with an FHIR `Appointment` record, and we’ll store it in a participant report under the “appointment” key on the date 1970-01-01. The participant will also be changed to the `tests_scheduled` state. This call is idempotent. If CUIMC calls it again, we’ll just update the appointment information. API: POST /v1/cumic/appointments - change user state to scheduled unless the status of the appointment is “cancelled.”
tests_cancelled	If the user has scheduled for lab work, and later cancels, we’ll get an updated FHIR `Appointment` record with the status of “cancelled.” In this case, the `StudyParticipant` will be marked with the `tests_cancelled` data group. The appointment report remains under the same key.
tests_scheduled	The user goes to the lab to get specimens collected. CUIMC calls Bridge to notify us that the specimens have been collected and are being processed with a `ProcedureRequest` payload. This will be stored as a participant report under the “procedure request” key on the date 1970-01-01. The participant will also be changed to the `tests_collected` state. Since this is just an update on the test collection, calling this API multiple times would just update the `state_change_timestamp`. API: POST /v1/cumic/proceduralrequests - change user state to collected
tests_collected	When the lab is done processing the specimens, CUIMC will send back an `Observation` payload to the results to Bridge via an API that will submit the data as health data, and then mark the participant as `tests_available`. We will also save the results as a participant report under the “observation” key on the date 1970-01-01. CUIMC will also send the user a message that they can go look at the results. API: POST /v1/cuimc/observations - change user state to scheduled
tests_available	This state ensures the participant will not be examined again by our worker. The UI can use this data group to highlight testing since it’s available and should be stored in the StudyParticipant `clientData`. (Alternatively, we could use reports for the appointment and observation information.

Data Persistence

All the data being submitted for this project needs to be in Synapse, and it needs to be accessible to the participants and researchers. Normally research data is not accessible to the Bridge system. The issue with this is that the calls are not atomic... they can succeed in one call but fail in another, leaving the system in an inconsistent state.

Here is how I currently believe we're going to address this:

Enrollment survey answers. These are sent to the reports API until completed, and then sent to the health data API. This means the reports API data will be complete when the health data API succeeds, should be ok. Some of this data is also going to be saved on the participant record itself (if it's PHI).
Labs confirming prior covid infection. The user will upload this file through the uploads API, which Dwayne is adjusting so it will accept unencrypted, uncompressed files. He’s also going to introduce a new API for researchers to retrieve HealthDataRecords, including this file and surveys.
Wendy examining new consented participants. if 1 and 2 are done correctly, a UI can be created for Wendy’s team to review participants and to accept or reject them for further testings.
Test results from Columbia lab. I believe this result will be JSON and again they will be able to submit it via the /v3/uploads API. We believe the lab will send a message to users, telling them they can sign in to our portal to view the results.

So we will need the following the APIs:

upload JSON as exportable health data (already exists)
upload binary as health data (sorta exists--we need to add the flags for non-encrypted, non-zipped data)
participant API to retrieve health data by date range (already exists)
researcher API to retrieve health data by participant and date range (TODO)
participant API to retrieve the raw data of their own health data record (TODO)
researcher API to retrieve the raw data of any participant in their app (TODO)
worker API to update a participant (TODO, our worker could also be a researcher to avoid creating this)

We also need the following external integration endpoints to work with our partner at Columbia. These will take an API key or just credentials for a specific account and will not require prior authorization (ie. authorization will happen in the controller):

API to submit appointment information for a user that will move the state of the user to TESTS_SCHEDULED;
API to submit the fact that the labs have been collected, which moves the state of the user to TESTS_COLLECTED;
API to submit the tests results of the labs that submits the data to our health data API and moves the state of the user to TESTS_AVAILABLE.

Worker Process

Synapse has a client that is calling an endpoint to update Bridge records. CUIMC is using a normal Bridge account to sign in and access participant records.