Participant adherence APIs

Initially we referred to this as compliance, but because “compliance” also has meaning in a governance context, so we’re using “adherence” to describe the measurement of study member participation in the study.

In the v2 science APIs, Bridge has three APIs that are involved in schedule adherence:

The schedule itself, which is finite in length and described by a timeline;
The set of adherence records for the participant which describe what the participant has done in that timeline;
A set of event timestamps that tell us where the participant is at in the timeline (ie what the user should have done and what they should currently be doing in the timeline).

So far the Bridge server provides this information through separate APIs and does not keep track of the time of the participants. For the sake of efficiency, I think we want the Bridge server to combine this information and provide reports on the status of the account, which will be less resource intensive that serving out all the information to be processed by a client or worker process. The issues I see with this:

For the first time the server will need to have knowledge of the participant’s time and time zone;
Because this information depends on the time of the request, it is not very cacheable;
Nevertheless, the reports probably update infrequently (the exact amount depends on many factors), while they may be read frequently in a couple of different formats.

APIs

Method	Path (Under /v5/studies/{studyId}	Description
GET	/participants/{userId}/adherence/eventstream	List of SessionStream reports for one user. The only view that shows scheduling for events the user does not have, this is a detailed view for one user of the whole schedule.
GET	/participants/{userId}/adherence/weekly	A single WeeklyAdherenceReport for one users. This API would generate and persist this report before returning it in the call, so that this call can be used by a worker process to build a record for every participant, but the call could also be called for up-to-date state information on a specific participant.
GET	/participants/adherence/weekly	A paged list where each record is a WeeklyAdherenceReport for one user. If no call has generated this report, it is not generated as part of this call, so a worker process will need to periodically generate these out-of-request.

Each report builds on the latter report.

EventStream Report (all adherence data for one participant)

The windows include the state of that session (which is the basis for adherence calculation). The states are:

State	Description	Adherence
not_applicable	Participant does not have this event in their events, so these sessions will not currently ever be shown to the participant.	N/A
not_yet_available	Participant should not have seen or started this session. It’s in the future.	N/A
unstarted	Participant should see the session (they are being asked to do it now), but they have not started it.	unknown
started	Participant has started the session.	unknown
completed	Participant completed the session before it expired.	compliant
abandoned	Participant started or finished at least one assessment in the session, but there was more work to do and it expired before they finished it.	noncompliant
expired	Participant did not start the session and it is now expired.	noncompliant

We can calculate a compliance percentage from these values at the session level, for the week, in the next report.

Weekly Adherence Report

This report always returns seven days of adherence records for a given user. The report calculates this information by finding the “week since event N” for every event and every session that includes the date for “today” (I don’t propose this timestamp will be adjustable since every timestamp would require a recalculation of all these reports). This means that the individual sessions listed in this report are not on the same calendar date. The structure of this report would be as follows:

{
  "participant": {
    "firstName": "A-chan",
    "email": "alx.dark+achan@sagebase.org",
    "externalId": "asdfasdf",
    "identifier": "GqYpNUWolebxS2eQudF1hc-a",
    "type": "AccountRef"
  },
  "createdOn": "2021-11-22T21:02:24.052Z",
  "weeklyAdherencePercent": 10,
  "dailyReports": [
    {
      "day": 0,
      "sessions": [
        {
          "sessionGuid": "LcWpQFKaGY5FSQ0LT4tnvdO7",
          "label": "Session #1",
          "symbol": "1",
          "startDay": 7,
          "startDate": "2021-11-16",
          "timeWindows": [
            {
              "sessionInstanceGuid": "jX9oQDeCJhGvI5Ee76obsQ",
              "timeWindowGuid": "GNp94CnfTTtR-s0OzrFeftrh",
              "state": "expired",
              "endDay": 7,
              "endDate": "2021-11-16",
              "type": "EventStreamWindow"
            },
            {
              "sessionInstanceGuid": "QIi64sd_gZYbA5fgLau8ig",
              "timeWindowGuid": "aRaHNKIY0yKgOl5CLuA3ZDHJ",
              "state": "expired",
              "endDay": 7,
              "endDate": "2021-11-16",
              "type": "EventStreamWindow"
            }
          ],
          "type": "EventStreamDay"
        },
        {
          "sessionGuid": "eRLgI5gfe1kef_XRZDfdFU9I",
          "label": "Session #2",
          "symbol": "2",
          "startDay": 7,
          "startDate": "2021-11-16",
          "timeWindows": [
            {
              "sessionInstanceGuid": "MzqzKXnkfIrcSiwkH7Yjig",
              "timeWindowGuid": "KZ1piANVdeD-r8PCHL2bviLh",
              "state": "expired",
              "endDay": 7,
              "endDate": "2021-11-16",
              "type": "EventStreamWindow"
            }
          ],
          "type": "EventStreamDay"
        },
        {
          "sessionGuid": "z_jb4p2Lr9Q56z8AwiYNieqw",
          "label": "Session #3",
          "symbol": "3",
          "startDay": 9,
          "startDate": "2021-11-24",
          "timeWindows": [
            {
              "sessionInstanceGuid": "WHE_gHE71tk8qFERatcruA",
              "timeWindowGuid": "gF6hy-UiipJLXqe7F_yK-wQc",
              "state": "started",
              "endDay": 9,
              "endDate": "2021-11-24",
              "type": "EventStreamWindow"
            }
          ],
          "type": "EventStreamDay"
        }
      ],
      "type": "DailyAdherenceReport"
    },
    {
      "day": 1,
      "sessions": [
        {
          "sessionGuid": "LcWpQFKaGY5FSQ0LT4tnvdO7",
          "label": "Session #1",
          "symbol": "1",
          "startDay": 8,
          "startDate": "2021-11-17",
          "timeWindows": [
            {
              "sessionInstanceGuid": "CS8QpTlrTSJdOE39U7SfKQ",
              "timeWindowGuid": "GNp94CnfTTtR-s0OzrFeftrh",
              "state": "expired",
              "endDay": 8,
              "endDate": "2021-11-17",
              "type": "EventStreamWindow"
            },
            {
              "sessionInstanceGuid": "aeVdiKislHPpC2w0VBn5fg",
              "timeWindowGuid": "aRaHNKIY0yKgOl5CLuA3ZDHJ",
              "state": "expired",
              "endDay": 8,
              "endDate": "2021-11-17",
              "type": "EventStreamWindow"
            }
          ],
          "type": "EventStreamDay"
        },
        {
          "sessionGuid": "eRLgI5gfe1kef_XRZDfdFU9I",
          "label": "Session #2",
          "symbol": "2",
          "startDay": 8,
          "startDate": "2021-11-17",
          "timeWindows": [
            {
              "sessionInstanceGuid": "K-P_nDWic33zjrxtlABE6Q",
              "timeWindowGuid": "KZ1piANVdeD-r8PCHL2bviLh",
              "state": "expired",
              "endDay": 8,
              "endDate": "2021-11-17",
              "type": "EventStreamWindow"
            }
          ],
          "type": "EventStreamDay"
        },
        {
          "sessionGuid": "z_jb4p2Lr9Q56z8AwiYNieqw",
          "label": "Session #3",
          "symbol": "3",
          "startDay": 12,
          "startDate": "2021-11-27",
          "timeWindows": [
            {
              "sessionInstanceGuid": "2O-jPnpWOYZLx0VjBvvO8g",
              "timeWindowGuid": "gF6hy-UiipJLXqe7F_yK-wQc",
              "state": "not_yet_available",
              "endDay": 12,
              "endDate": "2021-11-27",
              "type": "EventStreamWindow"
            }
          ],
          "type": "EventStreamDay"
        }
      ],
      "type": "DailyAdherenceReport"
    },
    // etc.
    {
      "day": 6,
      "sessions": [
        {
          "sessionGuid": "LcWpQFKaGY5FSQ0LT4tnvdO7",
          "label": "Session #1",
          "symbol": "1",
          "startDay": 13,
          "startDate": "2021-11-22",
          "timeWindows": [
            {
              "sessionInstanceGuid": "FHBhnKvSxoNXk9ubiWJVSw",
              "timeWindowGuid": "GNp94CnfTTtR-s0OzrFeftrh",
              "state": "unstarted",
              "endDay": 13,
              "endDate": "2021-11-22",
              "type": "EventStreamWindow"
            },
            {
              "sessionInstanceGuid": "PzplumYKek685k_DUU1lqg",
              "timeWindowGuid": "aRaHNKIY0yKgOl5CLuA3ZDHJ",
              "state": "completed",
              "endDay": 13,
              "endDate": "2021-11-22",
              "type": "EventStreamWindow"
            }
          ],
          "type": "EventStreamDay"
        },
        {
          "sessionGuid": "eRLgI5gfe1kef_XRZDfdFU9I",
          "label": "Session #2",
          "symbol": "2",
          "startDay": 13,
          "startDate": "2021-11-22",
          "timeWindows": [
            {
              "sessionInstanceGuid": "Kz1NQRlXiDnW_viqoGYfJA",
              "timeWindowGuid": "KZ1piANVdeD-r8PCHL2bviLh",
              "state": "unstarted",
              "endDay": 13,
              "endDate": "2021-11-22",
              "type": "EventStreamWindow"
            }
          ],
          "type": "EventStreamDay"
        }
      ],
      "type": "DailyAdherenceReport"
    }
  ],
  "type": "WeeklyAdherenceReport"
}

Persistent time windows will be excluded from all adherence reports. Completing assessments that are part of a persistent window do not count for or against adherence.

All of these views operate on the most recent timestamps for all events. Building schedules that rely on a mutable event changing, and triggering a new timeline of sessions to perform, will not work with these adherence APIs. That would be events like “session type X has been completed.” OTOH, it will show compliance with the most recent time stream, and that might be all that matters. Past time streams are no longer actionable.

The SQL table for this report will not attempt to store the report normalized.

CREATE TABLE `WeeklyAdherenceReports` (
  `appId` varchar(255) NOT NULL,
  `studyId` varchar(255) NOT NULL,
  `userId` varchar(255) NOT NULL,
  `createdOn` bigint(20) unsigned NOT NULL,
  `weeklyAdherencePercent` int(3) unsigned NOT NULL,
  `json` mediumtext NOT NULL,
  PRIMARY KEY (`appId`, `studyId`, `userId`),
  CONSTRAINT `WeeklyAdherenceReports-Study-Constraint` 
      FOREIGN KEY (`studyId`, `appId`) 
      REFERENCES `Substudies` (`id`, `studyId`) ON DELETE CASCADE,
  CONSTRAINT `WeeklyAdherenceReports-Account-Constraint` 
      FOREIGN KEY (`userId`) 
      REFERENCES `Accounts` (`id`) ON DELETE CASCADE,
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;

Weekly Adherence Reports for Study

The weekly adherence API for a user will persist the report

[TODO]

Protocol adherence and notifications

There is definitely a lot of possible ways we could measure adherence and I have some questions I have asked above about this. We can probably be pretty flexible on this since at some point, we have to load the whole state of the user’s participation to generate this report. We could then set a threshold for this (ie. warn administrators if there are any non-compliant assessments, vs. setting a proportion threshold that would need to be hit before notifying administrators). This warning apparently applies to specific participants, so it would be generated as part of the worker process described above.

Messaging APIs

One UI design I have seen shows a list of notifications in an administrative UI, indicating specific participants that are out-of-adherence with the study protocol. This hints at a very large amount of functionality.

We have had requirements to message both participants and study administrators over the years. For example, very early on we embedded a blog in an app for this purpose. Then we implemented support for remote notifications and topics (rarely used). We often message participants through local notifications based on scheduling information. Now we are talking about showing notifications to administrators about participants being out-of-adherence in an administrative UI.

I would like to add a simple but complete messaging system to Bridge which could be used to persist, deliver, and record the consumption of messages. Features would include:

API would be push-pull (write a message, request a list of active messages for a recipient; push-push is a lot more complicated);
recipients could include individuals or organizations (and other groups once we have them);
each recipient could mark the message as “read” to remove it from their UIs, separate from others, or “resolved” to remove it from everyone’s UIs. They would not be deleted so they are auditable;
messages could be set to expire;
Messages could indicate if they can be read, resolved, or if they expire. For example, an adherence message might only allow you to mark it “resolved” since you don’t want multiple study coordinators all following up with a participant.
Messages could be assigned (change recipients), and indicate a status like “busy” along with the user currently working on the message. Now you have a very simple way to hand off work between team members.

Once pull messaging is working, we could support “push” messaging (where we send a message via email or SMS, or possibly through remote or local push notifications). This introduced a lot of additional complexity, however:

It needs to be integrated with Amazon’s messaging APIs, where we are recording “sent” status. The advantage is that we’d have a record of such messaging, which is part of the study protocol and something that we’ve put together one-off solutions for in the past for scientists;
We’d want to implement messaging preferences, per user/organization and possibly by message type.