Data in Synapse can be downloaded using the programmatic clients (Python, R, and command line) as well as the web client. In this guide, you will learn the basic commands to download data programmatically. For instructions on how to download data from the web, see Downloading Filesfiles in our quick start guide.
Downloading a
...
file
Before you begin, it is important to understand that most items in Synapse have a unique identifier associated with them. This identifier is called a Synapse ID, or a synID. The synID format is the prefix “syn” followed by 8 numbers (for example, syn12345678). Items that have unique synIDs in Synapse are: Filesfiles, Foldersfolders, Projectsprojects, Tablestables, Viewsviews, Wikiswikis, Linkslinks, and Docker Repositoriesrepositories. You can use synIDs to refer to specific items when working with Synapse programmatically.
When using the Python, R, or command line clients, Files can files can be downloaded by using the get command. Downloaded Files files are stored and/or registered in a cache. By default, the cache location is in your home directory in a hidden folder named .synapseCache. Whenever the get function is invoked, the cache is checked to see if the same file is already present by checking its MD5 checksum. If it already exists, the file will not be downloaded again. In other words, if the current version of a File file has already been downloaded, Synapse will not re-download the same Filefile.
For the Python and R clients, the default download location is the Synapse cache. The command line client downloads to your current working directory. On the web, your own browser settings determine the download location for files. The Synapse cache is not updated to reflect downloads through a web browser. In all cases you can specify the directory in which to download the file.
...
| Code Block | ||
|---|---|---|
| ||
library(synapser)
synLogin()
entity <- synGet("syn3158111")
|
Once a File has a file has been downloaded, you can find the file path using the following:
...
| Code Block | ||
|---|---|---|
| ||
filepath <- entity$path |
Downloading a
...
specific version
If there are multiple versions of a Filea file, a specific version can be downloaded by passing the version parameter.
...
| Code Block | ||
|---|---|---|
| ||
entity <- synGet("syn3260973", version=1)
|
See versioning/wiki/spaces/DOCS/pages/2668134540 for more details.
Links
When you click on a Link link on the Synapse website, it will redirect you to the linked entity. The followLink parameter will have to be specified when using the programmatic clients or you will only retrieve the link itself without downloading the linked entity.
...
| Code Block |
|---|
library(synapser)
synLogin()
linkEnt = synGet("syn1234")
entity = synGet("syn1234", followLink=TRUE)
|
Download
...
location
To override the default download location, you can specify the downloadLocation parameter.
...
| Code Block |
|---|
entity <- synGet("syn00123", downloadLocation="/path/to/folder")
|
Finding and
...
downloading files
Files can be annotated in Synapse to help organize your data and make Files files findable. In order to search the annotations, a File View/wiki/spaces/DOCS/pages/2011070739 must be created first.
For example, the PCBC Project has a table listing sequencing data files that are annotated. To find all mRNA fastq files originating from CD34+ cells in the we can query by:
...
| Code Block | ||
|---|---|---|
| ||
results <- synTableQuery('select * from syn7511263 where dataType="mRNA" AND fileType="fastq" AND Cell_Type_of_Origin="CD34+ cells"')
df <- as.data.frame(results)
entity <- lapply(df$file.id, function(x) synGet(x))
|
Recursive
...
downloads
The folder structure that is present on Synapse can be maintained by recursive downloading.
...
| Code Block |
|---|
# Unfortunately, this feature is not available in the R client |
Download
...
wikis
The structure of a Wiki page a wiki page can be extracted through the R and Python clients. The ID, title, and parent Wiki page parent wiki page of each sub-Wiki page wiki page is also determined through the same method.
...
The Markdown content within a Wiki page wiki page can be downloaded if you know the synID and page ID for the Wikiwiki. The Wiki page The wiki page ID can either be obtained through the above method or can be found in the URL. For example, in the URL www.synapse.org/#!Synapse:syn00123/wiki/123456, the last 6 digits of the URL path is the Wiki page the wiki page ID (123456).
Python
| Code Block | ||
|---|---|---|
| ||
wiki = syn.getWiki("syn00123", 12345)
|
...
| Code Block |
|---|
entity <- synGet("syn00123")
wiki <- synGetWiki(entity, 12345)
|
Downloading in
...
bulk
Files can be downloaded in bulk using the syncFromSynapse function found in the synapseutils helper package. This function crawls all the subfolders of the Project project or Folder folder that you specify and retrieves all the files that have not been downloaded. By default, the files will be downloaded into your synapseCache, but a different download location can be specified with the path parameter. If you do download to a location out side of synapseCache, this function will also create a tab-delimited manifest of all the files along with their metadata (path, provenance, annotations, etc).
...