This document was last Last updated on 2023-09-1415
It This page is intended to describe the workflow required to build, edit, and update the data models model for MODEL-AD. Please ask questions to help clarification.
Data Modeling at Sage requires using two in-house tools: Schematic and the Data Curator App (DCA).
...
| Code Block |
|---|
pyenv install 3.10.1110 pyenv virtualenv 3.10.1110 schematicpy_3_10_1110 pyenv activate schematicpy_3_10_11 pyenv -m pip install schematic_3_10_11 pip install schematicpy |
Edit Configuration
...
/wiki/spaces/SCHEM/pages/2473623559
...
The data model is defined in a table, then stored (i.e. serialized) in a JSON-LD schema.
The JSON-LD schema follows the specifications from Schema.org for attributes.
Convert data model from CSV to JSONLD
schematic schema convert input.csv output.jsonld
...
Create Data Model
https://sagebionetworks.jira.com/wiki/spaces/SCHEM/pages/
...
2967568387/Guide+How+to+use+Schematic+for+Data+Model+
...
Development#Create-
...
a-
...
Data-
...
Schematic DB is a package used to ingress the manifests created by Schematic into a database.
Schematic DB will use any of these validation rules:
strfloatnumintdateIf the attribute has none of the above rules it use a string type
the attribute datatype will be determined based on the rule
Model
The data model is defined in a table, then stored (i.e. serialized) in a JSON-LD schema which specifies attributes as suggested by Schema.org.
/wiki/spaces/SCHEM/pages/2473623559
Build a Data Model
AD Data Models https://github.com/adknowledgeportal/data-models
...
/wiki/spaces/SCHEM/pages/2473623559
Recommendations
...
Draw a diagram. A diagram is a useful reference when developing the model.
Start small with a basic skeleton and then build.
Use schematic in dev mode to convert model to JSON-LD regularly to check for errors
...
The data model requires these columns:
AttributeDescriptionValidValuesDependsOnrequiredsourceparentpropertiesdependsOnComponent
Convert Data Model
https://sagebionetworks.jira.com/wiki/spaces/SCHEM/pages/2967568387/Guide+How+to+use+Schematic+for+Data+Model+Development#Convert-Data-Model
schematic schema convert model.csv
Schematic DB
Schematic DB is a package used to ingress the manifests created by Schematic into a database.
Schematic DB will use any of these validation rules:
str, float, num, int, dateIf no rule provided, defaults to a string type
the attribute datatype is based on the rule
Example Model
Github: https://github.com/Sage-Bionetworks/schematic/blob/develop/tests/data/example.model.csv
Formatted for readability:
Lref gdrive file url https://docs.google.com/spreadsheets/d/1Wde5YBFtEa4GhO-smXgbVApGioBGNnc-95n4LY8YB_E/edit#gid=925738608
...