Content Comparison

compute scores for DCCS (shape-color sorting) based on inferred Response Time for Android users where direct collection of RT is missing. This is to recover from a bug where RT was not collected from Android users prior to build 26.

User facing

• DCCS scores updated in daily .csv provided to collaborators

• item-level response times provided to NU for QA

• provide item-level response times persistent for users, prior to public release of data

• maintenance of computation in future scoring code

Detailed adherence report visible in the Researcher Portal UI that shows each assessment completion status, skipped status, and completion timestamp

User facing

In Researcher Portal UI, a coordinator can click into an individual participant and see the detailed completion status at the assessment level. Accessible via staging.studies.mobiletoolbox.org

• provide status of data as ‘in progress’ or ‘ready for analysis’

• persist scores completion state for Bridge to consume

Finalize reqs and implementation plan for schema refresh to support upcoming ICAR measure validation studies and update of existing measures.

Planning

• ETL design complete

• Implementation Q&A complete

• release coordination plan complete

• implementation work

• QA and data validation testing

• scoring code update and versioning plan

Publication of internal viz tools (Jupyter notebook) for data monitoring

Infrastructure

• Jupyter notebook published for viewing and code access

• daily completion record available on Synapse

• respond to feedback

• closeout anomalies

Updates to batch processing for 24hr scoring

Infrastructure

move to AWS batch jobs to address 10min timeout limitation

automate addition of new study IDs

Add data pipeline changes to release notes

Process

Monthly release notes

include downstream changes from ETL and scoring

respond to feedback

Jira Service Desk

Process

Mechanism for internal and external collaborators to file bugs in a template that propagates through to staging for triage

• respond to feedback

• setup cloning to Jira project space

Data re-processing of data ‘stuck’ in parquet queue

Issue resolution

• reprocess of missing UCSF from Jan-Mar due to broken array crawler

• root cause of UCSF missing data from Nov/Dec 2022

• identify remaining ~1-2% missing parquet

• resolution of UCSF missing data from Nov/Dec 2022

• disposition of remaining 1-2% missing parquet

Resolve PSM restart issues

Issue resolution

• Fix crash on Android

• Fix touching edge of screen causes PSM to restart on some iOS devices

monitor in production

Jira Service Desk Metrics

Process

Define and agree on metrics for bug response time

implement through Jira Service Desk

Login/logout capability

Allow participant and preview accounts to logout and login so that data state is retained and cache i s properly cleared without deleting and reinstalling the app

high

1

TBD

Turn off background recorders

part 1: turn off capability for researchers to optionally turn on recorders and default all study configs to OFF

part 2: excise the implementation of the recorders on both Android and iOS

2

TBD

Editable schedules

Allow changes to a schedule after a study has been launched

high

2

TBD. New studies can currently be manually constructed to have more flexibility. Existing studies will not be editable. New work will ensure that ALL new studies will automatically have more flexibility.

Spanish language

Provide app and assessments to run studies ALL in english or ALL in spanish

medium

2

earliest Fall 2023

Self-service account setup

Allow public users to onboard to Mobile Toolbox to explore, and to launch studies with a verified identity

high

1

target late summer 2023

Self-service access controls

Allow Study Admins to invite users and have granular control over what team members can see and do with PII and study data

high

1

target late summer 2023

Custom Survey Authoring

Researchers can collaborate and construct custom survey questions that can be delivered to participants as part of their schedule of activities

high

2

Deprioritized. Target beta in Q3/Q4

Detailed Adherence completion reporting

Study Coordinators can see the details of completion status and date for every assessment administered

medium

1

Q1 April 2023

Adherence Alerts page

Study Coordinators can view alerts when participants fall below adherence thresholds or other events

medium

2

Q2 2023

Schema refresh

Update the schemas of assessments to optimize data processing

medium

2

pending implementation plan for schema refresh. Ask is to complete for May/June start of clinical study

Automated scoring

Scores are automatically processed on-device or on a daily basis to keep up with ongoing data collection

medium

1

V1 complete

Public website updates

Include suggested language for grant submissions, sample data, FAQs, About Us bios, more detailed assessment information

medium

2

Q3 2023

Customer support helpdesk

Ticketing and tracking system and knowledge base to scale up responses to issues reported by coordinators

high

2

Q3 2023

Product documentation

Update and publish product documentation

lowmedium

3

May 2023

Crashlytics tuning

Improve crash and log collection from the mobile App to more quickly characterize and respond to bugs.

medium

2

Q2 2023

Data samples

Provide data samples so potential users can demo the app and explore data output without having to launch a study

low

3

Q2 2023

Demographics upload UI

Study Coordinators can upload demographics metadata so it can be automatically incorporated into data processing and analysis

medium

3

TBD