...

JSON Validation

New Design

JSON Schemas will be used to validate the structure of the JSON data and type correctness. In Java, we can use https://github.com/networknt/json-schema-validator (see also https://mvnrepository.com/artifact/com.networknt/json-schema-validator). This library was chosen because it’s compatible with both Jackson and Schema Draft 7.

JSON Schemas will be loaded from the mapping at file manifest in each upload’s metadata.json. See https://github.com/BridgeDigitalHealth/mobile-client-json/blob/main/archive-map.jsonpull/17 for documentation on the file format. For each file listed in the uploaded zip file manifest, we will use the filename to grab the correct schema, and then download the JSON Schema and use that to validate the file against that schema.

Previous experience tells us that researchers still want data even if it fails validation. However, we will log a message to the logs if it fails validation. We will use a Log Scan to add this to our dashboards so we can monitor and detect invalid uploads and incorrect schemas and fix app issues and systemic issues.

Open Questions:

• The mapping looks very sparse. The only file that has a schema is metadata.json. Everything else appears to be specific to the MTB app or assessments. Do we plan to update this mapping?

• From our previous meeting, it sounded like we settled on this design because this was what D&T wanted. If we need to write a new schemas anyway, we’re not actually saving that much work by re-using that design decision. (And we aren’t planning on using BridgeDownstream anymore anyway.) At that point, wouldn’t it make more sense to put them into AssessmentResources. This way, we don’t have to write code to download a GitHub URL and parse a bespoke JSON file.

• Are we only going to use the filename mapping? Or are we also going to use assessment and app mapping?

Old Design

JSON Schemas can be added as an Assessment Resource. Pre-req

JSON-to-Table-Row (see

Table Column Validation

A list of table columns will be added as an Assessment Resource. The format will be a simple JSON Array of strings, where the strings are the names of the expected table columns (excluding common metadata columns).

Converting from JSON to table columns is done server-side, so we only need to write the code once (on the server) instead of twice (once on iOS, once on Android). This means that we wouldn’t have JSON when we have table columns, so a JSON schema wouldn’t be usable here. A list of column names is the simplest solution that doesn’t require us to build a new typing system from scratch.

If JSON-to-Table-Row generates columns not in this list, we will log a message. We will use Log Scans to add this to our dashboard, similar to above. (This table column will still be included in the CSV, because researchers usually want these columns anyway.)