Versions Compared

Version Old Version 37 New Version 38
Changes made by

Alx Dark

Alx Dark

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

EventStream Report (all adherence data for one participant)

Weekly Adherence Report

This report always returns seven days of adherence records for a given user. The report calculates this information by finding the “week since event N” for every event and every session that includes the date for “today” (I don’t propose this timestamp will be adjustable since every timestamp would require a recalculation of all these reports). This means that the individual sessions listed in this report are not on the same calendar date. The structure of this report would be as follows:

...

languagejson

...

The windows include the state of that session (which is the basis for adherence calculation). The states are:

State

Description

Adherence

not_applicable

Participant does not have this event in their events, so these sessions will not currently ever be shown to the participant.

N/A

not_yet_available

Participant should not have seen or started this session. It’s in the future.

N/A

unstarted

Participant should see the session (they are being asked to do it now), but they have not started it.

unknown

started

Participant has started the session.

unknown

completed

Participant completed the session before it expired.

compliant

abandoned

Participant started or finished at least one assessment in the session, but there was more work to do and it expired before they finished it.

noncompliant

expired

Participant did not start the session and it is now expired.

noncompliant

We can calculate a compliance percentage from these values at the session level, for the week, in the next report.

Weekly Adherence Report

This report always returns seven days of adherence records for a given user. The report calculates this information by finding the “week since event N” for every event and every session that includes the date for “today” (I don’t propose this timestamp will be adjustable since every timestamp would require a recalculation of all these reports). This means that the individual sessions listed in this report are not on the same calendar date. The structure of this report would be as follows:

Code Block
languagejson
{
  "participant": {
    "firstName": "A-chan",
    "email": "alx.dark+achan@sagebase.org",
    "externalId": "asdfasdf",
    "identifier": "GqYpNUWolebxS2eQudF1hc-a",
    "type": "AccountRef"
  },
  "createdOn": "2021-11-22T21:02:24.052Z",
  "weeklyAdherencePercent": 10,
  "dailyReports": [
    {
      "day": 0,
{      "sessions": [
       "sessionInstanceGuid": "jX9oQDeCJhGvI5Ee76obsQ", {
              "timeWindowGuidsessionGuid": "GNp94CnfTTtR-s0OzrFeftrhLcWpQFKaGY5FSQ0LT4tnvdO7",
 
            "statelabel": "expiredSession #1",
 
            "endDaysymbol": 7"1",
 
            "endDatestartDay": "2021-11-16",
  7,
           "typestartDate": "EventStreamWindow2021-11-16",
          "timeWindows": [
},             {
              "sessionInstanceGuid": "QIi64sd_gZYbA5fgLau8igjX9oQDeCJhGvI5Ee76obsQ",
              "timeWindowGuid": "aRaHNKIY0yKgOl5CLuA3ZDHJGNp94CnfTTtR-s0OzrFeftrh",
              "state": "expired",
              "endDay": 7,
              "endDate": "2021-11-16",
              "type": "EventStreamWindow"
            },
          ],  {
        "type": "EventStreamDay"         }"sessionInstanceGuid": "QIi64sd_gZYbA5fgLau8ig",
        {
          "sessionGuidtimeWindowGuid": "eRLgI5gfe1kef_XRZDfdFU9IaRaHNKIY0yKgOl5CLuA3ZDHJ",
              "labelstate": "Session #2expired",
       
  "symbol": "2",           "startDayendDay": 7,
              "startDateendDate": "2021-11-16",
              "timeWindowstype": ["EventStreamWindow"
            {}
          ],
   "sessionInstanceGuid       "type": "MzqzKXnkfIrcSiwkH7Yjig"EventStreamDay"
        },
        {
          "timeWindowGuidsessionGuid": "KZ1piANVdeD-r8PCHL2bviLheRLgI5gfe1kef_XRZDfdFU9I",
          "label": "Session #2",
          "statesymbol": "expired2",
   
          "endDaystartDay": 7,
   
          "endDatestartDate": "2021-11-16",
  
           "typetimeWindows": "EventStreamWindow"[
            }{
          ],    "sessionInstanceGuid": "MzqzKXnkfIrcSiwkH7Yjig",
              "typetimeWindowGuid": "EventStreamDay"KZ1piANVdeD-r8PCHL2bviLh",
          },    "state": "expired",
   {           "sessionGuidendDay": "z_jb4p2Lr9Q56z8AwiYNieqw",7,
              "labelendDate": "Session #32021-11-16",
              "symboltype": "3EventStreamWindow",
          "startDay": 9, }
         "startDate": "2021-11-24" ],
          "timeWindowstype": ["EventStreamDay"
        },
   {     {
          "sessionInstanceGuidsessionGuid": "WHEz_gHE71tk8qFERatcruAjb4p2Lr9Q56z8AwiYNieqw",
   
          "timeWindowGuidlabel": "gF6hy-UiipJLXqe7F_yK-wQcSession #3",
              "statesymbol": "started3",
   
          "endDaystartDay": 9,
   
          "endDatestartDate": "2021-11-24",
  
           "typetimeWindows": "EventStreamWindow"[
            }{
          ],           "type": "EventStreamDay"
  "sessionInstanceGuid": "WHE_gHE71tk8qFERatcruA",
      }       ],       "type"timeWindowGuid": "DailyAdherenceReport"gF6hy-UiipJLXqe7F_yK-wQc",
    },     {       "daystate": 1"started",
      "sessions": [        "endDay": 9,
   {           "sessionGuidendDate": "LcWpQFKaGY5FSQ0LT4tnvdO72021-11-24",
              "labeltype": "Session #1EventStreamWindow",
          "symbol": "1", }
         "startDay": 8],
          "startDatetype": "2021-11-17EventStreamDay",
        }
 "timeWindows": [    ],
      "type": "DailyAdherenceReport"
{    },
    {
      "sessionInstanceGuidday": "CS8QpTlrTSJdOE39U7SfKQ"1,
         "sessions": [
    "timeWindowGuid": "GNp94CnfTTtR-s0OzrFeftrh",    {
          "statesessionGuid": "expiredLcWpQFKaGY5FSQ0LT4tnvdO7",
   
          "endDaylabel": 8"Session #1",
              "endDatesymbol": "2021-11-171",
          "startDay": 8,
          "typestartDate": "EventStreamWindow2021-11-17",
            },"timeWindows": [
            {
              "sessionInstanceGuid": "aeVdiKislHPpC2w0VBn5fgCS8QpTlrTSJdOE39U7SfKQ",
              "timeWindowGuid": "aRaHNKIY0yKgOl5CLuA3ZDHJGNp94CnfTTtR-s0OzrFeftrh",
              "state": "expired",
              "endDay": 8,
              "endDate": "2021-11-17",
              "type": "EventStreamWindow"
            },
            {
   ],           "typesessionInstanceGuid": "EventStreamDayaeVdiKislHPpC2w0VBn5fg",
        },      "timeWindowGuid": "aRaHNKIY0yKgOl5CLuA3ZDHJ",
   {           "sessionGuidstate": "eRLgI5gfe1kef_XRZDfdFU9Iexpired",
              "labelendDay": 8,
  "Session #2",           "symbolendDate": "22021-11-17",
 
        "startDay": 8,           "startDatetype": "2021-11-17EventStreamWindow",
          "timeWindows": [ }
           {
   ],
          "sessionInstanceGuidtype": "K-P_nDWic33zjrxtlABE6Q",EventStreamDay"
        },
      "timeWindowGuid  {
          "sessionGuid": "KZ1piANVdeD-r8PCHL2bviLheRLgI5gfe1kef_XRZDfdFU9I",
          "label": "Session #2",
          "statesymbol": "expired2",
  
           "endDaystartDay": 8,
 
            "endDatestartDate": "2021-11-17",

             "typetimeWindows": "EventStreamWindow"[
            }
   {
      ],           "typesessionInstanceGuid": "EventStreamDay"
        },
  K-P_nDWic33zjrxtlABE6Q",
     {           "sessionGuidtimeWindowGuid": "z_jb4p2Lr9Q56z8AwiYNieqwKZ1piANVdeD-r8PCHL2bviLh",
              "labelstate": "Session #3expired",
              "symbolendDay": "3",
       8,
  "startDay": 12,           "startDateendDate": "2021-11-2717",
              "timeWindowstype": "EventStreamWindow"
[            }
{          ],
    "sessionInstanceGuid      "type": "2O-jPnpWOYZLx0VjBvvO8g",EventStreamDay"
        },
        {
          "timeWindowGuidsessionGuid": "gF6hy-UiipJLXqe7F_yK-wQcz_jb4p2Lr9Q56z8AwiYNieqw",
          "label": "Session #3",
          "statesymbol": "not_yet_available3",
  
           "endDaystartDay": 12,
 
            "endDatestartDate": "2021-11-27",
          "timeWindows": [
  "type": "EventStreamWindow"         {
   }           ]"sessionInstanceGuid": "2O-jPnpWOYZLx0VjBvvO8g",
              "typetimeWindowGuid": "EventStreamDay"gF6hy-UiipJLXqe7F_yK-wQc",
        }       ]"state": "not_yet_available",
      "type": "DailyAdherenceReport"     },  "endDay": 12,
 // etc.     {       "dayendDate": 6"2021-11-27",
      "sessions": [        "type": "EventStreamWindow"
  {          }
"sessionGuid": "LcWpQFKaGY5FSQ0LT4tnvdO7",           "label": "Session #1"],
          "symboltype": "1EventStreamDay",
        }
 "startDay": 13,    ],
      "startDatetype": "2021-11-22DailyAdherenceReport",
    },
     "timeWindows": [// etc.
    {
      "day": 6,
{      "sessions": [
       "sessionInstanceGuid {
          "sessionGuid": "FHBhnKvSxoNXk9ubiWJVSwLcWpQFKaGY5FSQ0LT4tnvdO7",
              "timeWindowGuidlabel": "GNp94CnfTTtR-s0OzrFeftrhSession #1",
  
           "statesymbol": "unstarted1",
              "endDaystartDay": 13,
   
          "endDatestartDate": "2021-11-22",
  
           "typetimeWindows": "EventStreamWindow"
            },[
             {
              "sessionInstanceGuid": "PzplumYKek685k_DUU1lqgFHBhnKvSxoNXk9ubiWJVSw",
              "timeWindowGuid": "aRaHNKIY0yKgOl5CLuA3ZDHJGNp94CnfTTtR-s0OzrFeftrh",
              "state": "completedunstarted",
              "endDay": 13,
              "endDate": "2021-11-22",
              "type": "EventStreamWindow"
            },
          ],  {
        "type": "EventStreamDay"         },
    "sessionInstanceGuid": "PzplumYKek685k_DUU1lqg",
   {           "sessionGuidtimeWindowGuid": "eRLgI5gfe1kef_XRZDfdFU9IaRaHNKIY0yKgOl5CLuA3ZDHJ",
              "labelstate": "Session #2completed",
       
  "symbol": "2",           "startDayendDay": 13,
              "startDateendDate": "2021-11-22",
          "timeWindows": [   "type": "EventStreamWindow"
        {    }
          "sessionInstanceGuid": "Kz1NQRlXiDnW_viqoGYfJA"],
              "timeWindowGuidtype": "KZ1piANVdeD-r8PCHL2bviLhEventStreamDay",
          },
   "state": "unstarted",    {
          "endDaysessionGuid": 13,
  "eRLgI5gfe1kef_XRZDfdFU9I",
           "endDatelabel": "2021-11-22Session #2",
   
          "typesymbol": "EventStreamWindow2",
          "startDay": 13,
}           ]"startDate": "2021-11-22",
          "typetimeWindows": "EventStreamDay"[
        }    {
  ],
      "type": "DailyAdherenceReport"     }   ],   "type": "WeeklyAdherenceReport"
}

Weekly Adherence Reports for Study

SessionStream Report

Given the designs I have seen so far, I would suggest a set of records that show the sessions to be performed N days after each event, available as a single report that’ll be similar in size to the Timeline.

All sessions in the timeline are grouped in this view by the event that triggers them, and then the number of days since that event. All potential events in the schedule are included in this report whether they exist for the user or not. Then the actual “days since each event” are calculated to determine what the state of each time window is (states are described below the JSON sample):

Code Block
languagejson
[
  {
    "startEventId":"study_burst:ClinicVisit:01",
    "eventTimestamp":"2021-10-27T19:00:00.000Z",
    "entries":{
      "1":[
        {
          "sessionGuid":"vZBHBVv_H2_1TBbELF48czjS",
          "label": "Session #1",
          "symbol": "circle",
          "timeWindows":[
            {
              "sessionInstanceGuid":"ePcCf6VmfIiVuU0ckdBeRw",
              "timeWindowGuid":"sUaNAasy_LiT3_IYa1Fx_dSv",
              "state":"not_yet_available",
              "type":"SessionStreamWindow"
            },
            {
              "sessionInstanceGuid":"DB13D4mO72j6S-g7PIkI2Q",
              "timeWindowGuid":"Bw6rAAeG6zotqes4cLSgKjh5",
              "state":"not_yet_available",
              "type":"SessionStreamWindow"
            }
          ],
          "type":"SessionStream"
        }
      ],
      "2":[
        {
          "sessionGuid":"vZBHBVv_H2_1TBbELF48czjS",
          "label": "Session #1",
          "symbol": "circle",
          "timeWindows":[
            {
              "sessionInstanceGuid":"wvEV4fJZQ0nfgY-TN2LekA",
              "timeWindowGuid":"sUaNAasy_LiT3_IYa1Fx_dSv",
              "state":"not_yet_available",
              "type":"SessionStreamWindow"
            },
            {
              "sessionInstanceGuid":"IHDTSoj552vGDv1Qt7nXkg",
              "timeWindowGuid":"Bw6rAAeG6zotqes4cLSgKjh5",
              "state":"not_yet_available",
              "type":"SessionStreamWindow"
            }
          ],
          "type":"SessionStream"
        }
      ]
    },
    "type":"SessionStreamReport"
  },
  {
    "startEventId":"study_burst:ClinicVisit:02",
    "eventTimestamp":"2021-11-16T19:00:00.000Z",
    "entries":{
      "1":[
        {
          "sessionGuid":"vZBHBVv_H2_1TBbELF48czjS",
          "label": "Session #1",
          "symbol": "circle",
          "timeWindows":[
            {
              "sessionInstanceGuid":"zk7X4dQCy7Nvnuo2PcnSCA",
              "timeWindowGuid":"sUaNAasy_LiT3_IYa1Fx_dSv",
              "state":"not_yet_available",
              "type":"SessionStreamWindow"
            },
            {
              "sessionInstanceGuid":"rMRne-cbwIN5mkGZLymxzg",
              "timeWindowGuid":"Bw6rAAeG6zotqes4cLSgKjh5",
              "state":"not_yet_available",
              "type":"SessionStreamWindow"
            }
          ],
          "type":"SessionStream"
        }
      ],
      "2":[
        {
          "sessionGuid":"vZBHBVv_H2_1TBbELF48czjS",
          "label": "Session #1",
          "symbol": "circle",          
          "timeWindows":[
            {
              "sessionInstanceGuid":"QXM1cO6yb0gSPWzRwRD8eA",
              "timeWindowGuid":"sUaNAasy_LiT3_IYa1Fx_dSv",
              "state":"not_yet_available",
              "type":"SessionStreamWindow"
            },
            {
              "sessionInstanceGuid":"hCXFevxbBnpaUYjH212dsQ",
              "timeWindowGuid":"Bw6rAAeG6zotqes4cLSgKjh5",
              "state":"not_yet_available",
              "type":"SessionStreamWindow"
            }
          ],
          "type":"SessionStream"
        }
      ]
    },
    "type":"SessionStreamReport"
  }
]

The states are:

...

State

...

Description

...

Adherence

...

not_applicable

...

Participant does not have this event in their events, so these sessions will not currently ever be shown to the participant.

...

N/A

...

not_yet_available

...

Participant should not have seen or started this session. It’s in the future.

...

N/A

...

unstarted

...

Participant should see the session (they are being asked to do it now), but they have not started it.

...

unknown

...

started

...

Participant has started the session.

...

unknown

...

completed

...

Participant completed the session before it expired.

...

compliant

...

abandoned

...

Participant started or finished at least one assessment in the session, but there was more work to do and it expired before they finished it.

...

noncompliant

...

expired

...

Participant did not start the session and it is now expired.

...

noncompliant

I would propose that a participant’s noncompliance percentage is equal to noncompliant / (compliant + noncompliant + unknown). We can then set a threshold at which we would want to intervene (from 0% — any failure gets reported — to something less stringent).

Persistent time windows will be excluded from all adherence reports. Completing assessments that are part of a persistent window do not count for or against adherence.

All of these views operate on the most recent timestamps for all events. Building schedules that rely on a mutable event changing, and triggering a new timeline of sessions to perform, will not work with these adherence APIs. That would be events like “session type X has been completed.” OTOH, it will show compliance with the most recent time stream, and that might be all that matters. Past time streams are no longer actionable.

Weekly Adherence Report

Given an input date, we can take each stream, divide it into weeks, and return the week that has sessions which overlap with the input date. These seven records can generate a weekly report for the user (still a set of streams, one per event that is relevant to the schedule).

This report is probably not useful for a single individual, but when cached, it can serve as the basis for the study-wide weekly adherence reports below.

This API would take a local date and return a list of the following objects (one record per event ID), with each object representing one week where one of the days of that week (when measured for this user based on their timestamp for this event ID) falls on that date:

Code Block
public class WeeklyAdherence {
  // Key is userId + studyId + eventId
  String userId;
  String studyId
  String eventId;
  // could record information about burst, if relevant
  int weekNumber;
  DateTime createdOn;
  // Always 7 keys ("0" through "6")
  
  Map<String, List<SessionStreamWindow>> adherence;
}

This could be a straightforward Hibernate object with a child collection converted to a map. We’re generating one per user per study per app every evening, so we may need to use similar batch update tricks as were used to improve the performance of TimelineMetadata records.

Study SessionsOnDate API

Finally, the records above can be returned for a set of users. This is a very large amount of information (if you are looking for one week and are showing 25 users, it’s 25x7 pages of information). This is the visual design of the report this supports:

...

The information on the left-hand side about week and study burst is not easily inferred or stored for every user, but once implemented we can think about the cost of capturing this information. I am convinced the only way this view can be assembled is by aligning each user according to date (that is, there is no query to produce this that doesn’t relate it to calendar time, since “this week” is implicitly calendrical and this chart makes no sense if we’re not looking at a specific point in time, since each user is at their own point in the study).

...

sessionInstanceGuid": "Kz1NQRlXiDnW_viqoGYfJA",
              "timeWindowGuid": "KZ1piANVdeD-r8PCHL2bviLh",
              "state": "unstarted",
              "endDay": 13,
              "endDate": "2021-11-22",
              "type": "EventStreamWindow"
            }
          ],
          "type": "EventStreamDay"
        }
      ],
      "type": "DailyAdherenceReport"
    }
  ],
  "type": "WeeklyAdherenceReport"
}

Persistent time windows will be excluded from all adherence reports. Completing assessments that are part of a persistent window do not count for or against adherence.

All of these views operate on the most recent timestamps for all events. Building schedules that rely on a mutable event changing, and triggering a new timeline of sessions to perform, will not work with these adherence APIs. That would be events like “session type X has been completed.” OTOH, it will show compliance with the most recent time stream, and that might be all that matters. Past time streams are no longer actionable.

The SQL table for this report will not attempt to store the report normalized.

Code Block
languagesql
CREATE TABLE `WeeklyAdherenceReports` (
  `appId` varchar(255) NOT NULL,
  `studyId` varchar(255) NOT NULL,
  `userId` varchar(255) NOT NULL,
  `createdOn` bigint(20) unsigned NOT NULL,
  `weeklyAdherencePercent` int(3) unsigned NOT NULL,
  `json` mediumtext NOT NULL,
  PRIMARY KEY (`appId`, `studyId`, `userId`),
  CONSTRAINT `WeeklyAdherenceReports-Study-Constraint` 
      FOREIGN KEY (`studyId`, `appId`) 
      REFERENCES `Substudies` (`id`, `studyId`) ON DELETE CASCADE,
  CONSTRAINT `WeeklyAdherenceReports-Account-Constraint` 
      FOREIGN KEY (`userId`) 
      REFERENCES `Accounts` (`id`) ON DELETE CASCADE,
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;

Weekly Adherence Reports for Study

The weekly adherence API for a user will persist the report

[TODO]

Protocol adherence and notifications

...