...
A hierarchical structure of typed nodes. The assessment model in Bridge 2 is a hierarchical tree structure where each node in the assessment is typed using a meta typing system (not the Java types of the server’s implementation, and not types as defined by Swagger, but a meta model of type information that can be associated with any node in an assessment structure). This would allow us to ship some common assessment node types like survey or string question, with associated metadata about how the node should behave, but it would also be extensible by client developers without server-side development work.
Example. If client developers wanted to define a new kind of question for a survey, they could do so without server-side development work. If this definition became generally useful, the type model could be copied as a default to all studies in the system.
Every node in an assessment can be referenced in new an Assessment or a subtype. Some common subtypes would be our existing server objects like Survey or StringQuestion (collapsing SurveyElement and the constraints for a string question). Some of these subtypes may simply be markers and some may contain additional fields (all question types would include additional fields, in favor of the constraints in our current implementation model).
Limited ability to extend the node type system. Assessments can be assigned a “type” field by authors and they can take a map of metadata as part of the instance, so that assessment authors can take an existing node type and give it additional information. Bridge does not consider this a new type but a UI or authoring tool may use this information to treat it as such, with the understanding that successful experiments in defining new node types will be incorporated into later versions of the Bridge domain model.
Every assessment node can be treated as a first order assessment. My simply making an assessment node as a “root,” it will show up in our assessment APIs as an assessment and can be made available for scheduling.
Every child node in an assessment tree can be included in other assessments. A given node can have many parents, and even nodes that are only defined within an assessment could be promoted to be “root” assessments in their own right, or referenced as part of another assessment. (This has consequences for how these assessments are represented, queried, and retrieved).
This approach has pros and cons.
Pros:
Client developers currently maintain a lot of information about the internal structure and behavior of assessments that could be partially moved to the server (when it is independent of any specific platform implementation);
This approach requires less server-side development work to maintain evolution of our assessments;
Structures that we have desired in Bridge, such as survey sections, or screens with multiple form inputs, are trivially added to the system with this implementation (the types need to be defined and authoring tools have to be augmented to construct these new assessment structures);
Any assessment can be reconstructed into a new assessment (compound assessments are the default in this design structure);
Cons:
...
structures may raise some performance issues when we want to query them. However, apps could bundle all these structures when they are released to the App Store; we do not necessarily need to query the server over and over for them, since we want to move away from supporting in flight changes to running studies.
...
Data boundaries and data validation become more difficult, though we are deprecating our current support for validating uploaded data.
...
allowing for the composition of new assessments.
This model provides three important benefits over the existing model: 1) it unifies all assessments under a common type model with future extensions available to all forms of assessments; 2) it is composable into new parts dynamically; and 3) it allows for the nesting of assessment elements (survey groups, forms with multiple controls on one screen, etc.)
The type system is not fully dynamic. I toyed with this. For example, an assessment of a given type could include a JSON schema of additional information that could be defined for that type. Or we could develop our own typing system, as we did with upload schemas. There are a few problems with this:
it would encourage unplanned extension of the type system to include UI and implementation-specific details that are not supportable by the Bridge server and not used across client platforms;
it would be difficult to use in environments where code generation is being used based on our API definitions to create client libraries (the libraries, like the type system, would be completely generic, providing little support for the author in terms of what they can expect to be supported by actual app libraries);
validation errors based on schema validation are notoriously opaque and difficult for API consumers to understand and fix, particularly if the authors of the app submitting the data are not the authors of the assessment type system.
Assessment Nodes (/v1/assessments/*)
Colloquially we can refer to an assessment as any assessment node that has been marked as a “root” node (ie it is visible in the API as an assessment, and is a thing that would be directly scheduled for participants). All other nodes in an assessment can be referred to as assessment nodes, although the root is a similar kind of recordAll assessments are also nodes, but assessments that are marked as “roots” will appear in the assessments API and can be scheduled with other APIs. If an assessment node is treated as a root, it can just be referred to as an assessment if context warrants.
The data in an assessment node would be as follows:
Field | Data type | Notes |
|---|---|---|
studyId | String | Like all models, these are scoped and can vary between studies, although each study should be populated with a set of default assessment node types |
internalLabel | String | The label of this assessment when shown to study designers and implementers. |
internalDescription | String | The description of this assessment when show to study designers and implementers. It might initially be copied from module information, but could then be changed. |
createdOn, modifiedOn | DateTime, DateTime | |
moduleId, moduleVersion | String, Integer | References to a shared module from which this assessment tree was copied into a study. (Some metadata about the assessment should probably be retrieved from this module.) |
deleted | boolean | Assessments can be logically deleted if they are not referenced in any other assessment |
guid | String | |
root | boolean | Should this assessment node appear in lists of assessments as presented to study designers? (This isn’t determinable from having no parents.) |
label | String | A descriptor of the assessment |
labelDetail | String | A longer description of the assessment |
prompt | String | |
promptDetail | String | Probably a “learn more” feature |
image | Image | The metadata to load an image via HTTP |
beforeRules, afterRules | AssessmentRule | Similar to rules currently defined in survey elements, rules for navigating an assessment tree can be defined on any node in the tree |
children | List<Assessment> | An ordered list of child assessment nodes |
copyrightNotice | String | |
version | Long | optimistic locking version |
typeId | String | The type of this assessment node (see below) |
typeMetadatametadata | Map<String,Object> | Metadata that can be defined for this node based on its type (see below) |
Assessment Node Types (/v1/assessments/types/*)
An assessment node type defines a string identifier of the node’s type, and the metadata that can be collected for any node of that type.
Example. The string question node type has certain defined properties. It can only be included as a child in certain other node types (a survey or a form). It can collect minLength, maxLength, pattern, patternErrorMessage and patternPlaceholder metadata that can be used to validate data in the UI. If you were to add this type to a node that was not in a survey or form, or included a key in the metadata map that was not listed in the metadata for this type, or failed to include a required key in the map, the server would throw a validation error.
This type system does not support inheritance and does not support cascading from parent to child nodes (e.g. setting a type on the root does not make that type’s metadata fields available to sub-nodes). This is simpler to implement and understand, but if a new property needs to be made globally available across all types, it is also a significant limitation.
...
Field
...
Data Type
...
Notes
...
identifier
...
String
...
A primary key for this type within a study
...
studyId
...
String
...
The study this type is defined in
...
label
...
String
...
A label for the type
...
definitions
...
List<AssessmentTypeMetadataEntryDefinition>
...
A list of definitions guiding validation of metadata for this node type (see below)
Assessment Type Metadata Entry Definitions
To provide validation of assessment authoring, we can provide constraints for the data that is allowed in a metadata map in a given node, once it is set to a given type:
Field | Data Type | Notes |
|---|---|---|
label | String | Human readable explanation of the field’s value |
identifier | String | The key value to use when storing this metadata in the metadata map |
dataType | Enum (String, Number, Boolean, Object) | The type of the value that can be entered under this key. If the data type is “object”, then the value must be another metadata entry definitions object describing that object (and this can recurse to any depth that is needed to describe the metadata). |
array | Boolean | false if this is a single value, or true if the type is an array of elements of the specified dataType. |
required | boolean | Are assessment authors required to supply this value if the node is typed with the given type? |
The most complex node type I can identify on the server is the MultiValueConstraints object. It has a list of SurveyQuestionOption objects which themselves reference an Image object. Obviously, we don’t want design tools to attempt a generic assessment node editor, but rather we want to tailor the authoring experience to known types. For types unknown to editing tools, the typeMetada field could be exposed as JSON. The server can still validate the content is valid.
...
by assessment authors (has no defined meaning for Bridge and is not validated) |
Here is a very partial class hierarchy based on Bridge’s current domain support (note that Assessment is not an abstract class and can be the root node without further sub-typing):
...
Data boundaries and export
I would propose that all the data collected by an assessment (as a reminder, the root assessment node of an assessment definition) should be exported as a single dataset.
...