Data in Synapse can be downloaded using the our programmatic clients (Python, R, and command line) as well as the web client, or /wiki/spaces/DOCS/pages/2004254837. In this guide, you will learn the basic commands to download data programmatically.
Downloading Files
Before you begin, it is important to understand that most items in Synapse have a unique identifier associated with them. This identifier is called a Synapse ID, or a synID. The synID format is the prefix “syn” followed by 8 numbers (for example, syn12345678). Items that have unique synIDs in Synapse are: files, folders, projects, tables, views, wikis, links, and Docker repositories. You can use synIDs to refer to specific items when working with Synapse programmatically.
...
| Code Block |
|---|
filepath <- entity$path |
Downloading a Specific File Version
If there are multiple versions of a file, a specific version can be downloaded by passing the version parameter.
...
| Code Block |
|---|
entity <- synGet("syn3260973", version=1)
|
See Versioning for more details.
Downloading Linked Data
When you click on a link on the Synapse website, it will redirect you to the linked entity. The followLink parameter will have to be specified when using the programmatic clients or you will only retrieve the link itself without downloading the linked entity.
...
| Code Block |
|---|
library(synapser)
synLogin()
linkEnt = synGet("syn1234")
entity = synGet("syn1234", followLink=TRUE)
|
...
Downloading Location
To override the default download location, you can specify the downloadLocation parameter.
...
| Code Block |
|---|
entity <- synGet("syn00123", downloadLocation="/path/to/folder")
|
Finding and Downloading Files via Annotations
Files can be annotated/wiki/spaces/DOCS/pages/2667708522 in Synapse to help organize your data and make files findable. In order to search the annotations, a file view must be created you must create a /wiki/spaces/DOCS/pages/2011070739 first.
For example, the PCBC Project has a table listing sequencing data files that are annotated. To find all mRNA fastq files originating from CD34+ cells in the we can query by:
...
| Code Block |
|---|
results <- synTableQuery('"select * from syn7511263 where dataType="'mRNA"' AND fileType="'fastq"' AND Cell_Type_of_Origin="'CD34+ cells'"') df <- as.data.frame(results) |
...
| Code Block |
|---|
results <- synTableQuery('"select * from syn7511263 where dataType="'mRNA"' AND fileType="'fastq"' AND Cell_Type_of_Origin="'CD34+ cells'"') df <- as.data.frame(results) entity <- lapply(df$file.id, function(x) synGet(x)) |
Recursive
...
Downloading
The folder structure that is present on Synapse can be maintained by recursive downloading.
...
| Code Block |
|---|
# Unfortunately, this feature is not available in the R client |
...
Downloading Wikis
The structure of a wiki page can be extracted through the R and Python clients. The ID, title, and parent wiki page of each sub-wiki page is also determined through the same method.
...
| Code Block |
|---|
entity <- synGet("syn00123")
wiki <- synGetWiki(entity, 12345)
|
Downloading in Bulk
Files can be downloaded in bulk using the syncFromSynapse function found in the synapseutils helper package. This function crawls all the subfolders of the project or folder that you specify and retrieves all the files that have not been downloaded. By default, the files will be downloaded into your synapseCache, but a different download location can be specified with the path parameter. If you do download to a location out side of synapseCache, this function will also create a tab-delimited manifest of all the files along with their metadata (path, provenance, annotations, etc).
...