Mobile Toolbox Status Summary - March/April 2023
- Ann Campton
- Sonia Carlson
Headlines
Our goal for the Mobile Toolbox Platform is to provide a complete end-to-end solution for the use of remotely-administered cognitive and other tests by researcher teams, without the need for any study-specific technology development. A complete solution must cover scenarios across the complete lifecycle of a study, and support research teams in:
Delivering timely, reliable, high-fidelity data to researchers
Providing well documented validated measures
Learning about available mobile cognitive measures and designing appropriate digital research protocols
Managing the collection of data from study participants, including support for enrollment, aggregate/high-level adherence tracking, and participant communications
Providing secure transfer of de-identified data to analysts
Disseminating data, results, and other findings from completed studies for forms that encourage reuse by other research teams, subject to FAIR principles.
Highlights - March 2023
Met criteria for Go-ahead to restart MGH study. Data collection has started and is at 100% fidelity
Fixed in production edge scenarios with schedule delivery and sign-in for mobile clients
Crashlytics monitoring deployed on iOS
Progress on defining weekly release cadence and criteria. Will finalize process over the next two sprint cycles
New devops viz tool identified an issue with Parquet exporter which has reduced ‘stuck’ data.
Baseline QA metrics defined. Next step to socialize, signoff and build dashboard
Centralized issue tracker up and in-use for active triage. Still needs refinement.
Lowlights - March 2023
Reports of PSM issues
Need further triage of remaining ‘stuck’ data
Risks
Need to close on background recorder decisions
manual onboarding process is documented
have received new binaries from Monash group
met to provide feedback, Monash has some additional work to do
server work is complete to support growth of assessment library
upcoming bug bash of researcher UI portal
Monthly summary
Where can I find information about…
*reach out to Sonia or Ann if you can’t access links
Issue Triage and Prioritization Scale
Data completion downloadable .csv
Jira Service Desk bug template - in progress
Jira Metrics - in progress
Jira Incident Manager - not started
Jira projects, backlogs, boards
ETL / Data Processing
Synapse Mobile Toolbox parent project
What are we working on? April 2023 our focus is on:
Description | Type | Expected output by end of month | Future work |
|---|---|---|---|
compute scores for DCCS (shape-color sorting) based on inferred Response Time for Android users where direct collection of RT is missing. This is to recover from a bug where RT was not collected from Android users prior to build 26. | User facing |
|
|
Detailed adherence report visible in the Researcher Portal UI that shows each assessment completion status, skipped status, and completion timestamp | User facing | In Researcher Portal UI, a coordinator can click into an individual participant and see the detailed completion status at the assessment level. Accessible via staging.studies.mobiletoolbox.org |
|
Finalize reqs and implementation plan for schema refresh to support upcoming ICAR measure validation studies and update of existing measures. | Planning |
|
|
Publication of internal viz tools (Jupyter notebook) for data monitoring | Infrastructure |
|
|
Updates to batch processing for 24hr scoring | Infrastructure | move to AWS batch jobs to address 10min timeout limitation | automate addition of new study IDs |
Add data pipeline changes to release notes | Process | Monthly release notes include downstream changes from ETL and scoring | respond to feedback |
Jira Service Desk | Process | Mechanism for internal and external collaborators to file bugs in a template that propagates through to staging for triage |
|
Data re-processing of data ‘stuck’ in parquet queue | Issue resolution |
|
|
Resolve PSM restart issues | Issue resolution |
| monitor in production |
Jira Service Desk Metrics | Process | Define and agree on metrics for bug response time | implement through Jira Service Desk |
Communications
UCSF response and instructions for PSM and for some duplicate data collections
User facing feature requests
Feature | Description | Complexity | Priority | Delivery forecast |
|---|---|---|---|---|
Login/logout capability | Allow participant and preview accounts to logout and login so that data state is retained and cache i s properly cleared without deleting and reinstalling the app | high | 1 | TBD |
Turn off background recorders | part 1: turn off capability for researchers to optionally turn on recorders and default all study configs to OFF part 2: excise the implementation of the recorders on both Android and iOS |
| 2 | TBD |
Editable schedules | Allow changes to a schedule after a study has been launched | high | 2 | TBD. New studies can currently be manually constructed to have more flexibility. Existing studies will not be editable. New work will ensure that ALL new studies will automatically have more flexibility. |
Spanish language | Provide app and assessments to run studies ALL in english or ALL in spanish | medium | 2 | earliest Fall 2023 |
Self-service account setup | Allow public users to onboard to Mobile Toolbox to explore, and to launch studies with a verified identity | high | 1 | target late summer 2023 |
Self-service access controls | Allow Study Admins to invite users and have granular control over what team members can see and do with PII and study data | high | 1 | target late summer 2023 |
Custom Survey Authoring | Researchers can collaborate and construct custom survey questions that can be delivered to participants as part of their schedule of activities | high | 2 | Deprioritized. Target beta in Q3/Q4 |
Detailed Adherence completion reporting | Study Coordinators can see the details of completion status and date for every assessment administered | medium | 1 | April 2023 |
Adherence Alerts page | Study Coordinators can view alerts when participants fall below adherence thresholds or other events | medium | 2 | Q2 2023 |
Schema refresh | Update the schemas of assessments to optimize data processing | medium | 2 | pending implementation plan for schema refresh. Ask is to complete for May/June start of clinical study |
Automated scoring | Scores are automatically processed on-device or on a daily basis to keep up with ongoing data collection | medium | 1 | V1 complete |
Public website updates | Include suggested language for grant submissions, sample data, FAQs, About Us bios, more detailed assessment information | medium | 2 | Q3 2023 |
Customer support helpdesk | Ticketing and tracking system and knowledge base to scale up responses to issues reported by coordinators | high | 2 | Q3 2023 |
Product documentation | Update and publish product documentation | medium | 3 | May 2023 |
Crashlytics tuning | Improve crash and log collection from the mobile App to more quickly characterize and respond to bugs. | medium | 2 | Q2 2023 |
Data samples | Provide data samples so potential users can demo the app and explore data output without having to launch a study | low | 3 | Q2 2023 |
Demographics upload UI | Study Coordinators can upload demographics metadata so it can be automatically incorporated into data processing and analysis | medium | 3 | TBD |
OKRs and supporting initiatives - Work in progress….
<Objective 1>
on track / at risk / OFF Track
Key results |
|---|
<Objective 2>
on track / at risk / OFF Track
Key results |
|---|
<Objective 3>
on track / at risk / OFF Track
Key results |
Other updates
Engineering/Operational Health |
|---|
|
Customer insights |
|---|
|
Marketing Highlights |
|---|
|
Competitor insights |
|---|
|