Initially we referred to this as compliance, but because “compliance” also has meaning in a governance context, we’re using “adherence” to describe the measurement of the extent to which a participant performs all the assessments they are asked to perform as part of their participation in a study member participation in the study.
| Table of Contents | ||||
|---|---|---|---|---|
|
...
The schedule itself, which is finite in length and described by a timeline;
The set of adherence records for the participant which describe what the participant has done in that timeline (we only work with the session records when calculating adherence);
A set of event timestamps that tell us where the participant is at in the timeline (ie what the user should have done and what they should currently be doing in the timelineschedule).
So far the Bridge server provides this information through separate APIs and does not keep track of the time of the participants. For the sake of efficiency, I think we want the Bridge server to combine this information and provide reports on the status of the account, which will be less resource intensive that serving out all the information to be processed by a client or worker process. The issues I see with this:
For the first time the server will need to have knowledge of the participant’s time and time zone(?);
Because this information depends on the time of the request, it is not very cacheable;
The reports are (relatively) expensive to calculate and they are write seldom, read frequently in nature, so they will need to be cached.
API (Event Day Adherence Report)
Given the designs I have seen so far, I created the “event day” report which would be available via a single API:
...
Method
...
Path
...
Description
...
GET
probably update infrequently compared to the amount they will be read (the exact amount depends on many factors). We will have to devise caching strategies to make the system performant when looking at many participants.
Persistent time windows will be excluded from all adherence reports. Completing assessments that are part of a persistent window do not count for or against adherence.
All of these views operate on the most recent timestamps for all events. Building schedules that rely on a mutable event changing, and triggering a new timeline of sessions to perform, will not work with these adherence APIs. That would be events like “do X when session type Y has been completed.” Since it will show compliance with the most event time stream, it may be sufficient anyway. Past time streams are no longer actionable.
APIs
Method | Path (Under /v5/studies/{studyId} | Description |
|---|---|---|
GET | /participants/{userId |
}/adherence/ |
The JSON looks as follows (note that we can add whatever further information we need based on the UI, but the existing designs are minimal, with all windows just being represented by a symbol colored to show its current adherence state):
...
| language | json |
|---|
...
eventstream | EventStreamAdherenceReport for one user. This view includes scheduling based on events the user does not have, and is a detailed view of the entire schedule for one user. | |
GET | /participants/{userId}/adherence/weekly | A WeeklyAdherenceReport for one user. This calculates the “current week” of each event timestamp and aligns it to a seven day schedule. |
GET | /participants/adherence/weekly | A paged list where each record is a WeeklyAdherenceReport for one user. If no call has generated this report, it is not generated as part of this call, so a worker process will need to periodically generate these out-of-request. |
GET | /v1/apps/{appId}/studies/{studyId} | Worker endpoint to create and persist the weekly report for users, to enable the paged view across all accounts. |
Each report builds on the latter report.
EventStream Report (all adherence data for one participant)
This report is intended to support the following type of adherence chart:
...
The report is calculated against the caller’s clientTimeZone value (if they have a value set), or else it is calculated against the timeZone value of the Study (to determine the current date). This is what the JSON would like like:
| Code Block | ||
|---|---|---|
| ||
{ "timestamp": "2021-11-23T22:00:31.699Z", "clientTimeZone": "America/Los_Angeles", "adherencePercent": 25, "streams": [ { "startEventId": "custom:event1", "eventTimestamp": "2021-11-21T20:00:00.000Z", "sessionInstanceGuiddaysSinceEvent":"ePcCf6VmfIiVuU0ckdBeRw" 2, "byDayEntries": { "timeWindowGuid0":"sUaNAasy_LiT3_IYa1Fx_dSv", [ { "state "sessionGuid": "noteRLgI5gfe1kef_yet_availableXRZDfdFU9I", "typesessionLabel": "EventDayWindowSession #2", } "sessionSymbol": "2", { "startDay": 0, "sessionInstanceGuidstartDate": "DB13D4mO72j6S2021-11-g7PIkI2Q21", "timeWindowGuidtimeWindows":"Bw6rAAeG6zotqes4cLSgKjh5", [ { "state":"not_yet_available", "typesessionInstanceGuid": "EventDayWindowbDBVV02XrjrOG8NgBYSYFg", } "timeWindowGuid": "KZ1piANVdeD-r8PCHL2bviLh", ], "typestate": "EventDay" completed", } ], "2endDay":[ 0, { "sessionGuidendDate":"vZBHBVv_H2_1TBbELF48czjS "2021-11-21", "label": "Session #1", "symboltype": "circleEventStreamWindow", "timeWindows":[ } { ], "sessionInstanceGuidtype": "wvEV4fJZQ0nfgY-TN2LekAEventStreamDay", } "timeWindowGuid":"sUaNAasy_LiT3_IYa1Fx_dSv", ], "state1":"not_yet_available", [ { "type":"EventDayWindow" }"sessionGuid": "eRLgI5gfe1kef_XRZDfdFU9I", { "sessionLabel": "Session #2", "sessionInstanceGuidsessionSymbol": "IHDTSoj552vGDv1Qt7nXkg2", "timeWindowGuidstartDay":"Bw6rAAeG6zotqes4cLSgKjh5" 1, "statestartDate":"not_yet_available "2021-11-22", "type"timeWindows":"EventDayWindow" [ } { ], "typesessionInstanceGuid": "EventDayu-cc8Ou4xqUPVnht4oh-bw", } ] "timeWindowGuid": "KZ1piANVdeD-r8PCHL2bviLh", }, "type":"EventDayAdherenceReport" }, { "startEventIdstate":"study_burst:ClinicVisit:02 "expired", "eventTimestamp":"2021-11-16T19:00:00.000Z", "entries":{ "1endDay":[ 1, { "sessionGuidendDate":"vZBHBVv_H2_1TBbELF48czjS", "2021-11-22", "label": "Session #1", "symboltype": "circleEventStreamWindow", "timeWindows":[ } { ], "sessionInstanceGuidtype": "zk7X4dQCy7Nvnuo2PcnSCAEventStreamDay", } "timeWindowGuid":"sUaNAasy_LiT3_IYa1Fx_dSv", ] }, "statetype": "not_yet_availableEventStream", }, { "typestartEventId": "EventDayWindow"custom:event2", "eventTimestamp": "2021-11-15T20:00:00.000Z", "daysSinceEvent": }8, "byDayEntries": { { "0": [ "sessionInstanceGuid":"rMRne-cbwIN5mkGZLymxzg", { "timeWindowGuidsessionGuid": "Bw6rAAeG6zotqes4cLSgKjh5z_jb4p2Lr9Q56z8AwiYNieqw", "statesessionLabel":"not_yet_available "Session #3", "typesessionSymbol": "EventDayWindow3", } "startDay": 0, ], "typestartDate": "EventDay"2021-11-15", } "timeWindows": [ ], "2":[ { { "sessionGuidsessionInstanceGuid":"vZBHBVv_H2_1TBbELF48czjS "KAxvwhsX6jSVl89a3-gdKw", "label "timeWindowGuid": "Session #1gF6hy-UiipJLXqe7F_yK-wQc", "symbolstate": "circleexpired", "endDay": 2, "timeWindows":[ "endDate": "2021-11-17", { "sessionInstanceGuidtype": "QXM1cO6yb0gSPWzRwRD8eAEventStreamWindow", "timeWindowGuid":"sUaNAasy_LiT3_IYa1Fx_dSv",} "state":"not_yet_available"], "type": "EventDayWindowEventStreamDay" }, ], { "3": [ "sessionInstanceGuid":"hCXFevxbBnpaUYjH212dsQ", { "timeWindowGuidsessionGuid": "Bw6rAAeG6zotqes4cLSgKjh5z_jb4p2Lr9Q56z8AwiYNieqw", "statesessionLabel":"not_yet_available "Session #3", "typesessionSymbol": "EventDayWindow3", "startDay": 3, } ]"startDate": "2021-11-18", "typetimeWindows":"EventDay" [ }{ ] }, "typesessionInstanceGuid": "EventDayAdherenceReport"nQnWs_4ECvLtY4K1gYk67g", } ] |
All sessions in the timeline are combined in this view so that each entry is for one event (one entry for every event and only one entry), and then a list by day of the activities that the participant should perform as a result of that event. All potential events in the schedule are included in this report whether they exist for the user or not. Then the actual “days since each event” are calculated to determine what the state of these assessments are:
...
State
...
Description
...
Adherence
...
not_yet_available
...
Participant should not have seen or started this assessment.
...
N/A
...
unstarted
...
Participant should see the assessment (they are being asked to do it now), but they have not started it.
...
unknown
...
started
...
Participant has started the assessment.
...
unknown
...
completed
...
Participant has completed the assessment before it expired and was removed from the UI.
...
compliant
...
abandoned
...
Participant started the assessment but it expired before they finished it.
...
noncompliant
...
ignored
...
Participant did not start the assessment; it is now expired.
...
noncompliant
I would propose that a participant’s noncompliance percentage is equal to noncompliant / (compliant + noncompliant + unknown). We can then set a threshold at which we would want to intervene (from 0% — any failure gets reported — to something less stringent).
Adherence Event Day Report
Basic structure of an adherence record (per participant) is:
durationInDays (the highest endDay value calculated by the timeline, which can be lower that the schedule duration, but the schedule duration will cap any open-ended sequences);
Enum studyAdherence (compliant, not in compliance, some measure of this);
Map<Stream,List<SessionAdherence>> adherenceTimeline;
There’s nothing in this about event sequences that haven’t happened because they haven’t been triggered by an event. We might need to encode, in a schedule, what is desired in terms of event generation.
There’s nothing about individual sessions or assessments being optional. There’s no easy way to determine adherence for persistent time windows, so we’ll assume that performing such a window one time is compliant…maybe.
Stream
The timeline describes parts of scheduling that can be repeated, if the triggering events can change over time. Study bursts are also an example of repeating parts of scheduling. Each of these things represents a separate stream.
label (the session name, probably)
startEventId
eventTimestamp
session window GUID (each time window generates one stream)
SessionAdherence (might be possible to edit the state if coordinator knows state better). In visual designs, all time windows are grouped under each session (in our server model, each window creates a separate stream of activities…but we would want a set of tests). So we might have a set of WindowAdherence records, as below, in one session adherence container (not sure there’s much in the container).
state
- not yet available NOT_YET_AVAILABLE
- available but unstarted UNSTARTED
- started STARTED
- successfully completed COMPLETED
- expired unfinished ABANDONED
- expired never started IGNORED/NONCOMPLIANTinstanceGuid
sessionGuid
startDay
endDay
startTime
expiration
SessionAdherence needs to boil down the completed state of the session based on the timestamps, I think. User would be out of compliance if some of their states are expired. Should probably be possible to calculate these records and then store them so Bridge can deliver them. So a summary of the state of the record should probably also be saved for efficient querying.
Caching individual participant adherence reports
There are options:
Cache them with Redis (probably my least-preferred option since domain logic will be based on these reports);
Write them to a participant report either when requested, or on a nightly basis, and expose only reports through the APIs (treating reports as a caching mechanism);
Write the report back to the database (actually more performant than calculating in memory? Not proven). If we want paginated views of individual performance status, then we’ll need to define a schema for the base information and store it back to a table.
The reports only need to change when the user submits an event or an adherence record (or when the timeline changes, which would invalidate quite a lot). Typically we delete the cached item on these writes and let the next read re-create the cached item.
Protocol adherence
There is definitely a lot of possible ways we could measure adherence. However it can be measured as a proportion of compliant to non-compliant scheduled assessment states. We could then set a threshold for this (ie. warn administrators if there are any non-compliant assessments, vs. setting a proportion threshold that would need to be hit before notifying administrators). This can be calculated as part of the event day adherence report.
Arm and study summaries
Not sure what the requirements are for these, but worker processes would periodically (or when prompted by a message) calculate and store these. The logical place would be study reports (but again a SQL table is an option).
Messaging APIs
One UI design I have seen shows a list of notifications in an administrative UI, indicating specific participants that are out-of-adherence with the study protocol. This hints at a very large amount of functionality.
We have had requirements to message both participants and study administrators over the years. For example, very early on we embedded a blog in an app for this purpose. Then we implemented support for remote notifications and topics (rarely used). We often message participants through local notifications based on scheduling information. Now we are talking about showing notifications to administrators about participants being out-of-adherence in an administrative UI.
I would like to add a simple but complete messaging system to Bridge which could be used to persist, deliver, and record the consumption of messages. Features would include:
API would be push-pull (write a message, request a list of active messages for a recipient; push-push is a lot more complicated);
recipients could include individuals or organizations (and other groups once we have them);
each recipient could mark the message as “read” to remove it from their UIs, separate from others, or “resolved” to remove it from everyone’s UIs. They would not be deleted so they are auditable;
messages could be set to expire;
Messages could indicate if they can be read, resolved, or if they expire. For example, an adherence message might only allow you to mark it “resolved” since you don’t want multiple study coordinators all following up with a participant.
Messages could be assigned (change recipients), and indicate a status like “busy” along with the user currently working on the message. Now you have a very simple way to hand off work between team members.
Once pull messaging is working, we could support “push” messaging (where we send a message via email or SMS, or possibly through remote or local push notifications). This introduced a lot of additional complexity, however:
...
It needs to be integrated with Amazon’s messaging APIs, where we are recording “sent” status. The advantage is that we’d have a record of such messaging, which is part of the study protocol and something that we’ve put together one-off solutions for in the past for scientists;
...
"timeWindowGuid": "gF6hy-UiipJLXqe7F_yK-wQc",
"state": "expired",
"endDay": 5,
"endDate": "2021-11-20",
"type": "EventStreamWindow"
}
],
"type": "EventStreamDay"
}
]
},
"type": "EventStream"
},
{
"startEventId": "study_burst:main-sequence:01",
"eventTimestamp": "2021-11-21T20:00:00.000Z",
"daysSinceEvent": 2,
"studyBurstId": "main-sequence",
"studyBurstNum": 1,
"byDayEntries": {
"0": [
{
"sessionGuid": "LcWpQFKaGY5FSQ0LT4tnvdO7",
"sessionLabel": "Session #1",
"sessionSymbol": "1",
"startDay": 0,
"startDate": "2021-11-21",
"timeWindows": [
{
"sessionInstanceGuid": "1aCbUaFYkixIsIJBf9WGpg",
"timeWindowGuid": "GNp94CnfTTtR-s0OzrFeftrh",
"state": "expired",
"endDay": 0,
"endDate": "2021-11-21",
"type": "EventStreamWindow"
},
{
"sessionInstanceGuid": "yQnubrShfYMY9ZzOE3zw3Q",
"timeWindowGuid": "aRaHNKIY0yKgOl5CLuA3ZDHJ",
"state": "expired",
"endDay": 0,
"endDate": "2021-11-21",
"type": "EventStreamWindow"
}
],
"type": "EventStreamDay"
}
]
},
"type": "EventStream"
}
],
"type": "EventStreamAdherenceReport"
} |
EventStreamWindow includes the state of that session (which is the basis for adherence calculation). This adherence report could be displayed on a calendar since it is specific to a user and the server must know all the calendrical values to calculate it (it's the only report that can take a timestamp to indicate the time at which the report should be calculated, and it is calculated in the user’s local time if possible).
The completion states are:
State | Description | Adherence |
|---|---|---|
not_applicable | Participant does not have this event in their events, so these sessions will not currently ever be shown to the participant. | N/A |
not_yet_available | Participant should not have seen or started this session. It’s in the future. | N/A |
unstarted | Participant should see the session (they are being asked to do it now), but they have not started it. | unknown |
started | Participant has started the session. | unknown |
completed | Participant completed the session before it expired. | compliant |
abandoned | Participant started or finished at least one assessment in the session, but there was more work to do and it expired before they finished it. | noncompliant |
expired | Participant did not start the session and it is now expired. | noncompliant |
We can calculate a compliance percentage from these values across a participant’s entire participation in the study. In the weekly report below, we can calculate adherence for that week alone.
Weekly Adherence Report
This report always returns seven days of adherence records for a given user. The report calculates this information by finding the “week since event N” for every event and every session (it will be the week that has the day that falls on “today” as measured in the user’s local time zone). For performance reasons, we will not calculate this week outside of the “current week” so it’s not possible to move forward or backward from this view (at least initially).
Note that the the individual sessions listed in this report are not lined up by calendar date (unless they were triggered by the same event). The structure of this report would be as follows:
| Code Block | ||
|---|---|---|
| ||
{
"participant": {
"firstName": "A-chan",
"email": "alx.dark+achan@sagebase.org",
"externalId": "asdfasdf",
"identifier": "GqYpNUWolebxS2eQudF1hc-a",
"type": "AccountRef"
},
"requestTimestamp": "2021-11-23T21:03:21.356Z",
"clientTimeZone": "America/Los_Angeles",
"createdOn": "2021-11-23T22:03:21.356Z",
"weeklyAdherencePercent": 33,
// nextActivity, if byDayEntries is empty
"byDayEntries": {
"0": [
{
"sessionGuid": "eRLgI5gfe1kef_XRZDfdFU9I",
"sessionLabel": "Session #2",
"sessionSymbol": "2",
"week": 1,
"startDate": "2021-11-21",
"timeWindows": [
{
"sessionInstanceGuid": "bDBVV02XrjrOG8NgBYSYFg",
"timeWindowGuid": "KZ1piANVdeD-r8PCHL2bviLh",
"state": "completed",
"endDate": "2021-11-21",
"type": "EventStreamWindow"
}
],
"type": "EventStreamDay"
},
{
"sessionGuid": "LcWpQFKaGY5FSQ0LT4tnvdO7",
"sessionLabel": "Session #1",
"sessionSymbol": "1",
"week": 1,
"studyBurstId": "main-sequence",
"studyBurstNum": 1,
"startDate": "2021-11-21",
"timeWindows": [
{
"sessionInstanceGuid": "1aCbUaFYkixIsIJBf9WGpg",
"timeWindowGuid": "GNp94CnfTTtR-s0OzrFeftrh",
"state": "expired",
"endDate": "2021-11-21",
"type": "EventStreamWindow"
},
{
"sessionInstanceGuid": "yQnubrShfYMY9ZzOE3zw3Q",
"timeWindowGuid": "aRaHNKIY0yKgOl5CLuA3ZDHJ",
"state": "expired",
"endDate": "2021-11-21",
"type": "EventStreamWindow"
}
],
"type": "EventStreamDay"
}
],
"1": [
{
"sessionGuid": "eRLgI5gfe1kef_XRZDfdFU9I",
"sessionLabel": "Session #2",
"sessionSymbol": "2",
"week": 1,
"startDate": "2021-11-22",
"timeWindows": [
{
"sessionInstanceGuid": "u-cc8Ou4xqUPVnht4oh-bw",
"timeWindowGuid": "KZ1piANVdeD-r8PCHL2bviLh",
"state": "expired",
"endDate": "2021-11-22",
"type": "EventStreamWindow"
}
],
"type": "EventStreamDay"
},
{
"sessionGuid": "LcWpQFKaGY5FSQ0LT4tnvdO7",
"sessionLabel": "Session #1",
"sessionSymbol": "1",
"week": 1,
"studyBurstId": "main-sequence",
"studyBurstNum": 1,
"startDate": "2021-11-22",
"timeWindows": [
{
"sessionInstanceGuid": "7HaIdOehYJrJk3VZcGeNxg",
"timeWindowGuid": "GNp94CnfTTtR-s0OzrFeftrh",
"state": "completed",
"endDate": "2021-11-22",
"type": "EventStreamWindow"
},
{
"sessionInstanceGuid": "R0I7n1fEeIR88fYv6iVPTg",
"timeWindowGuid": "aRaHNKIY0yKgOl5CLuA3ZDHJ",
"state": "expired",
"endDate": "2021-11-22",
"type": "EventStreamWindow"
}
],
"type": "EventStreamDay"
}
],
"2": [
{
"sessionGuid": "eRLgI5gfe1kef_XRZDfdFU9I",
"sessionLabel": "Session #2",
"sessionSymbol": "2",
"week": 1,
"startDate": "2021-11-23",
"timeWindows": [
{
"sessionInstanceGuid": "s7HuTxm4E-ffhfiddVFkXQ",
"timeWindowGuid": "KZ1piANVdeD-r8PCHL2bviLh",
"state": "started",
"endDate": "2021-11-23",
"type": "EventStreamWindow"
}
],
"type": "EventStreamDay"
},
{
"sessionGuid": "z_jb4p2Lr9Q56z8AwiYNieqw",
"sessionLabel": "Session #3",
"sessionSymbol": "3",
"week": 2,
"startDate": "2021-11-24",
"timeWindows": [
{
"sessionInstanceGuid": "WHE_gHE71tk8qFERatcruA",
"timeWindowGuid": "gF6hy-UiipJLXqe7F_yK-wQc",
"state": "not_yet_available",
"endDate": "2021-11-26",
"type": "EventStreamWindow"
}
],
"type": "EventStreamDay"
},
{
"sessionGuid": "LcWpQFKaGY5FSQ0LT4tnvdO7",
"sessionLabel": "Session #1",
"sessionSymbol": "1",
"week": 1,
"studyBurstId": "main-sequence",
"studyBurstNum": 1,
"startDate": "2021-11-23",
"timeWindows": [
{
"sessionInstanceGuid": "SgTohgnPkGjB9eE73QRXEA",
"timeWindowGuid": "GNp94CnfTTtR-s0OzrFeftrh",
"state": "completed",
"endDate": "2021-11-23",
"type": "EventStreamWindow"
},
{
"sessionInstanceGuid": "ezwrtaRiJd5Bfa3w5Xxffw",
"timeWindowGuid": "aRaHNKIY0yKgOl5CLuA3ZDHJ",
"state": "unstarted",
"endDate": "2021-11-23",
"type": "EventStreamWindow"
}
],
"type": "EventStreamDay"
}
],
"3": [
{
"sessionGuid": "eRLgI5gfe1kef_XRZDfdFU9I",
"sessionLabel": "Session #2",
"sessionSymbol": "2",
"week": 1,
"startDate": "2021-11-24",
"timeWindows": [
{
"sessionInstanceGuid": "u78_-kFdyNnKt6wbUmSXcA",
"timeWindowGuid": "KZ1piANVdeD-r8PCHL2bviLh",
"state": "not_yet_available",
"endDate": "2021-11-24",
"type": "EventStreamWindow"
}
],
"type": "EventStreamDay"
},
{
"sessionGuid": "LcWpQFKaGY5FSQ0LT4tnvdO7",
"sessionLabel": "Session #1",
"sessionSymbol": "1",
"week": 1,
"studyBurstId": "main-sequence",
"studyBurstNum": 1,
"startDate": "2021-11-24",
"timeWindows": [
{
"sessionInstanceGuid": "mcpv_-sW1vXfk4Vyi5dwgg",
"timeWindowGuid": "GNp94CnfTTtR-s0OzrFeftrh",
"state": "not_yet_available",
"endDate": "2021-11-24",
"type": "EventStreamWindow"
},
{
"sessionInstanceGuid": "t4AEs7UzOjXYACSOO1REPA",
"timeWindowGuid": "aRaHNKIY0yKgOl5CLuA3ZDHJ",
"state": "not_yet_available",
"endDate": "2021-11-24",
"type": "EventStreamWindow"
}
],
"type": "EventStreamDay"
}
],
"4": [
{
"sessionGuid": "eRLgI5gfe1kef_XRZDfdFU9I",
"sessionLabel": "Session #2",
"sessionSymbol": "2",
"week": 1,
"startDate": "2021-11-25",
"timeWindows": [
{
"sessionInstanceGuid": "tJxcgcQK40X6jL71MORb3g",
"timeWindowGuid": "KZ1piANVdeD-r8PCHL2bviLh",
"state": "not_yet_available",
"endDate": "2021-11-25",
"type": "EventStreamWindow"
}
],
"type": "EventStreamDay"
},
{
"sessionGuid": "LcWpQFKaGY5FSQ0LT4tnvdO7",
"sessionLabel": "Session #1",
"sessionSymbol": "1",
"week": 1,
"studyBurstId": "main-sequence",
"studyBurstNum": 1,
"startDate": "2021-11-25",
"timeWindows": [
{
"sessionInstanceGuid": "22nOvDMhYIE7btX6X8zlHw",
"timeWindowGuid": "GNp94CnfTTtR-s0OzrFeftrh",
"state": "not_yet_available",
"endDate": "2021-11-25",
"type": "EventStreamWindow"
},
{
"sessionInstanceGuid": "H2EG1SV-tF_s8dmKWwvtOQ",
"timeWindowGuid": "aRaHNKIY0yKgOl5CLuA3ZDHJ",
"state": "not_yet_available",
"endDate": "2021-11-25",
"type": "EventStreamWindow"
}
],
"type": "EventStreamDay"
}
],
"5": [
{
"sessionGuid": "eRLgI5gfe1kef_XRZDfdFU9I",
"sessionLabel": "Session #2",
"sessionSymbol": "2",
"week": 1,
"startDate": "2021-11-26",
"timeWindows": [
{
"sessionInstanceGuid": "-OoNi3vhifRUqN6X9WTMCg",
"timeWindowGuid": "KZ1piANVdeD-r8PCHL2bviLh",
"state": "not_yet_available",
"endDate": "2021-11-26",
"type": "EventStreamWindow"
}
],
"type": "EventStreamDay"
},
{
"sessionGuid": "z_jb4p2Lr9Q56z8AwiYNieqw",
"sessionLabel": "Session #3",
"sessionSymbol": "3",
"week": 2,
"startDate": "2021-11-27",
"timeWindows": [
{
"sessionInstanceGuid": "2O-jPnpWOYZLx0VjBvvO8g",
"timeWindowGuid": "gF6hy-UiipJLXqe7F_yK-wQc",
"state": "not_yet_available",
"endDate": "2021-11-29",
"type": "EventStreamWindow"
}
],
"type": "EventStreamDay"
},
{
"sessionGuid": "LcWpQFKaGY5FSQ0LT4tnvdO7",
"sessionLabel": "Session #1",
"sessionSymbol": "1",
"week": 1,
"studyBurstId": "main-sequence",
"studyBurstNum": 1,
"startDate": "2021-11-26",
"timeWindows": [
{
"sessionInstanceGuid": "JfyiAbHZ-nlmTZui_N19Tg",
"timeWindowGuid": "GNp94CnfTTtR-s0OzrFeftrh",
"state": "not_yet_available",
"endDate": "2021-11-26",
"type": "EventStreamWindow"
},
{
"sessionInstanceGuid": "g6kRO-JZgt0lGghDT8v37A",
"timeWindowGuid": "aRaHNKIY0yKgOl5CLuA3ZDHJ",
"state": "not_yet_available",
"endDate": "2021-11-26",
"type": "EventStreamWindow"
}
],
"type": "EventStreamDay"
}
],
"6": [
{
"sessionGuid": "eRLgI5gfe1kef_XRZDfdFU9I",
"sessionLabel": "Session #2",
"sessionSymbol": "2",
"week": 1,
"startDate": "2021-11-27",
"timeWindows": [
{
"sessionInstanceGuid": "_-1AlFrrN9IfA5tK-OYvag",
"timeWindowGuid": "KZ1piANVdeD-r8PCHL2bviLh",
"state": "not_yet_available",
"endDate": "2021-11-27",
"type": "EventStreamWindow"
}
],
"type": "EventStreamDay"
},
{
"sessionGuid": "LcWpQFKaGY5FSQ0LT4tnvdO7",
"sessionLabel": "Session #1",
"sessionSymbol": "1",
"week": 1,
"studyBurstId": "main-sequence",
"studyBurstNum": 1,
"startDate": "2021-11-27",
"timeWindows": [
{
"sessionInstanceGuid": "IUBXsO4ioFgJfi8GA9NSxw",
"timeWindowGuid": "GNp94CnfTTtR-s0OzrFeftrh",
"state": "not_yet_available",
"endDate": "2021-11-27",
"type": "EventStreamWindow"
},
{
"sessionInstanceGuid": "WdRjHkDWoqmQx4vxRuCVeg",
"timeWindowGuid": "aRaHNKIY0yKgOl5CLuA3ZDHJ",
"state": "not_yet_available",
"endDate": "2021-11-27",
"type": "EventStreamWindow"
}
],
"type": "EventStreamDay"
}
]
},
"type": "WeeklyAdherenceReport"
} |
This API would calculate and persist this report before returning it in the call, so that this call can be used by a worker process to build a record for every participant. The call could also be used to retrieve up-to-date state information for a specific participant. If this call is not called for a user, it will not show up in the paginated view of these records. We may want to hide a record that hasn’t been updated in the last 24 hours although a worker process will hopefully force a refresh on all accounts.
The SQL table for this report will not attempt to store the report normalized:
| Code Block | ||
|---|---|---|
| ||
CREATE TABLE `WeeklyAdherenceReports` (
`appId` varchar(255) NOT NULL,
`studyId` varchar(255) NOT NULL,
`userId` varchar(255) NOT NULL,
`requestTimestamp` bigint(20) unsigned NOT NULL,
`createdOn` bigint(20) unsigned NOT NULL,
`labels` text NOT NULL,
`weeklyAdherencePercent` int(3) unsigned NOT NULL,
`reportData` mediumtext NOT NULL,
PRIMARY KEY (`appId`, `studyId`, `userId`),
CONSTRAINT `WeeklyAdherenceReports-Study-Constraint`
FOREIGN KEY (`studyId`, `appId`)
REFERENCES `Substudies` (`id`, `studyId`) ON DELETE CASCADE,
CONSTRAINT `WeeklyAdherenceReports-Account-Constraint`
FOREIGN KEY (`userId`)
REFERENCES `Accounts` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci; |
The “labels“ columns will pull out the labels from the JSON into a pipe-delimited string, to enable searching for records that match the label in the study-scoped API (searching by label and percentage of adherence are UI design requirements). Note that right now, this would mean the server would need to create the labels and the client would need to search with those server-supplied labels.
Weekly Adherence Reports for Study
This API is intended to support the following kind of adherence overview for the study:
...
The final API will allow an admin to retrieve a paged set of weekly adherence records for all participants in the study. This retrieves the records from the database and does not calculate them. The paging search criteria should be the same as the study participants search API. In addition to the search arguments in the /v5/studies/{studyId}/participants/search API, it should be possible to filter and or sort on the following fields:
The adherence percentage;
The label of individual windows (through a like query on the labels column).
I would like to reuse the search code for participants, but it may be necessary to join tables in a way where that can’t be done, while still maintaining the desired paging logic. To start, it’s probably possible to just search and sort by adherence and labels.
Protocol adherence and notifications (worker process)
Since all the reports give an adherence percentage (whole study or one week), we can use this to sort records by adherence in the weekly view, or to generate a message to administrators as part of the worker process that will be run nightly to update the weekly reports. Any necessary configuration for this can be added to the Study model (in particular, at what percentage of adherence should we notify administrators that a participant is out of adherence).
A worker process could run hourly, querying Bridge for any studies that had a time zone such that the local time of the study was at a specific time (e.g. 4am). If that study had a schedule, all the participants would be retrieved and the weekly adherence report would be generated for them. Since these reports are being generated on the server’s time, it’s possible in unusual scenarios that this time would give odd results for a user in a far off time zone…if so we can possibly run reports for a study more frequently than every 24 hours (but we needn’t start by implementing to prevent this).
If the worker hit certain criteria, it could also send a message warning about a participant being out of adherence.
Adherence Service
The following methods will be added to the AdherenceService:
| Code Block | ||
|---|---|---|
| ||
public class AdherenceService {
public EventStreamAdherenceReport getEventStreamAdherenceReport(
String appId, String studyId, String userId, DateTime now, boolean showActive) {
}
public WeeklyAdherenceReport getWeeklyAdherenceReport(
String appId, String studyId, String userId, DateTime now) {
}
public PagedResourceList<WeeklyAdherenceReport> getPagedWeeklyAdherenceReports(
AdherenceSearch search) {
}
}
public class AdherenceSearch {
int offsetBy;
int pageSize;
searches accounts that are enrolled in a specific study
String studyId;
String emailFilter;
String phoneFilter;
Set<String> allOfGroups;
Set<String> noneOfGroups;
String language;
DateTime startTime;
DateTime endTime;
String externalIdFilter;
AccountStatus status;
String attributeKey;
String attributeValueFilter;
String sessionLabel;
Integer weekNumber;
String studyBurstId;
Integer studyBurstNum;
Integer maxAdherencePercent;
SearchTermPredicate predicate;
StringSearchPosition stringSearchPosition;
} |
System Messaging APIs
We have had requirements to message both participants and study administrators over the years. For example, very early on we embedded a blog in an app for this purpose. Then we implemented support for remote push notifications and topics (rarely used). Instead in recent studies we have tried using local notifications based on scheduling information. Now we are talking about showing notifications to administrators about participants being out-of-adherence in an administrative UI. Because “notifications” is heavily overloaded for Bridge, I will call these “system messages.”
I would like to add a simple Ωmessaging system to Bridge which could be used to persist, deliver, and record the consumption of messages. Features would include:
API would be pub-sub and pull-based (write a message, request a list of active messages for a recipient; push-push via something like SMS or email is a lot more complicated);
Recipients should include individuals only, opt-in, to start;
Each recipient could mark the message as “read” to hide it from their UIs, or “resolved” to hide it for everyone. They would not be deleted so they are auditable. We should record who deleted or resolved the message;
Messages would have a type (“out of adherence,” “study joined,” “study completed”, etc.) and messages could be filtered on the type and/or study;
These should be app-scoped because it would be very annoying to have to go into every study to see what needed attention in that study.
This should meet the basics needs of our recent UI designs.
Method | Path | Description |
|---|---|---|
GET | /v1/messages | Get the messages (paged and filterable by type and whether message has been read) for the caller. |
POST | /v1/messages | Publish a message to all subscribers. |
DELETE | /v1/messages/{guid} | Mark a message as read or resolved (if resolved=true) |
GET | /v1/messages/subscriptions | Get the studies and types of messages you are subscribed to |
POST | /v1/messages/subscriptions/{studyId}/{type} | Subscribe to receive messages of a specific type in a specific study. |
DELETE | /v1/messages/subscriptions/{studyId}/{type} | Unsubscribe from receiving messages of a given type in a given study. |
The objects (subscriptions and message):
| Code Block | ||
|---|---|---|
| ||
public class Subscription {
String appId;
String studyId;
MessageType messageType;
String recipientId;
}
// As submitted
public class Message {
String studyId;
MessageType messageType;
String text;
}
// As persisted, one record per subscription
public class Message {
String guid;
String appId;
String studyId;
String userId;
MessageType messageType;
String text;
DateTime createdOn;
boolean read;
String readBy;
} |
Additional proposed work items
Allow study burst and custom event IDs to have spaces, essentially allowing them to be human-readable labels like “Study Burst” or “Clinic Visit.” Right now we prevent the obvious labels from being used but this becomes difficult when we’re going to allow people to search by labels that include these identifiers.
Study should add a
studyTimeZonefield. We may also want to add anadherenceThresholdPercentin anticipation of messaging.