This vignette will combine concepts from Annotations and Queries, Views, Uploading and Downloading Data in Bulk in order to create a manifest velociraptor_manifest.txt, upload 100 files and edit annotations . You will learn how to:
Create a manifest
Upload 100 files
Edit annotations on these files using the Synapse programmatic clients
...
Annotation dictionaries
The Sage Bionetworks Systems Biology and Computational Oncology research teams maintain annotation dictionaries in GitHub. You can use the terms found in the synapseAnnotations GitHub repo in this repository as a starting point for , or you can create your own annotationsannotation dictionary.
Create a Manifest and Batch Upload the Files with Annotations
To batch upload files, create a tab-delimited manifest which contains, at minimum, the columns path and parent. You can also add additional annotations as columns in your manifest. For example, your manifest might have the following headers: path, parent, specimenID, assay, species, platform, sex, and fileFormat. See Creating a Manifest in Uploading and Downloading Data in Bulk for additional details.
path:
...
the local path to your file
parent:
...
the Synapse ID (in the format syn123456)
...
of the Folder or Project where your files will be uploaded
...
specimenID:
...
the unique identifier for each of your specimens
assay:
...
the technology used to generate the data in this file (
...
for example, RNASeq, ChIPSeq, wholeGenomeSeq)
species:
...
the species of your sample (
...
for example, Mouse, Rat, Human, Triceratops)
platform:
...
the hardware used to generate the data (
...
for example, HiSeq2500, Affy6.0, HoodDNASequencer)
sex:
...
a label assigned at birth based on biological attributes (for example, male or female)
fileFormat: is the type of file (e.g. fastq, R script)
...
path | parent | specimenID | assay | species | platform | sex | fileFormat |
|---|---|---|---|---|---|---|---|
/local/path/to/velociraptor_b.fastq | syn123 | blue_1 | wholeGenomeSeq | Velociraptor mongoliensis | HoodDNASequencer | female | fastq |
/local/path/to/velociraptor_d.fastq | syn123 | delta_1 | wholeGenomeSeq | Velociraptor mongoliensis | HoodDNASequencer | female | fastq |
See Creating a Manifest in Uploading and Downloading Data in Bulk for additional details.
Save this file in a tab-delimited format called velociraptor_manifest.tsv.
Files can be uploaded in one go all at once with a manifest fileFile. If you would like to do a “dry run” validation of the file File before uploading, you can add the parameter dryRun = True to the function syncToSynapse. Please note Note that the dryRun feature checks everything, but does not upload the files.
validate the manifest and upload files in the Python client.
validate the manifest and upload files in the R client.
...
Create a File View (Web)
Since Once the files Files have been uploaded with annotations, you can use a file File View allows users to query, facet, and bulk manipulate the files Files and metadataMetadata.
To create your File View:
...
An annotation for a single file File can be modified in the Web web client View in the case that . For example, you can updatespecimenID:delta_1 needs to be updated to specimenID:echo_1.
Select Tables Tools and Edit Query Results.
Find the From your View, select the pencil icon to edit query results.
In the pop-up window, find the single value you want to change or delete in the pop-up and edit the field.
Scroll down to the bottom and click Save to update the file View.
...
Perform a Bulk Annotation Update or Deletion
A bulk annotation update is required in the case that species:Velociraptor mongoliensis should be modified to Utahraptor ostrommaysorum in all 100 files.
Web
To download the annotation values from the Web web client:
Navigate to the file your View of choice.
Click the Download Options button to the right of the query bar of the file File View.
Click Export Table to download the file View. Be sure to have
Include row metadata (Row Id and Row Version)selected when downloading.
Now that you have the file File View downloaded, you can edit the values using your tool of preference, whether that is preferred tool (Python, R, Excel, LibreOffice, etcor other).
With the changes saved, go back to the file File View in your browser.
Click View Tools located at the upper right of the file File View
Select Upload Data to View from the dropdown.
Browse and choose the edited file.
Click Next to preview the uploaded file.
Click Update Table to update the file File View and populate the changes to all the files in Synapse.
Programmatic Clients
Alternatively, download the File View with the R or Python client, then:
Query for the View with
synTableQuery()orsyn.tableQuery(). To To delete all the annotations of a key, you have to keep the column in the file view File View but remove the values.Update and then store the annotations in the R client or Python client.
...