Document toolboxDocument toolbox

Uploading and Organizing Data Into Projects, Files, and Folders

Owned by Stacey Taylor (Unlicensed)
Last updated: Dec 15, 2023 by Bryan FaubleVersion comment
6 min read

Synapse helps you upload and organize your data in an efficient way through the use of projects, files, and folders.

Projects

Projects in Synapse are “containers” that group relevant content and people together. All data must be uploaded into a project. Projects can be private so only you can see the contents, they can be shared with your collaborators, or they can be made public so anyone on the web can view your research.

Projects help you to:

  • Organize your work: With Synapse, you can organize the parts in your workflow (data, code, etc) into a hierarchy like a file system. With the tabs across the top of each project, you can easily navigate to your wikis, files and folders, SQL-based tables, and even conduct conversations using discussion forums.

  • Store data, code, and results: You can upload your data, code, and results to Synapse, or store a reference to their location in your local computer system or on the web. Everything can be stored as files hosted by Synapse, in your own external cloud storage, or using Docker containers.

  • Control data access or release it publicly: You have complete control over how users and groups can interact with your work. Work privately, openly, or somewhere in between.

  • Link and share content with others: Just as you can control the access to your work, projects can serve as a platform for linking and sharing your work with others.

  • Custom, searchable annotations: Assign any key/value pair you want. Those values become searchable and available to those granted access.

  • Attach figures and documents: Upload documents and images via the website or programmatically.

Creating a Project

To create a new project:

  1. Click the Projects icon in the left-hand toolbar, and click the plus sign (+) icon next to Projects

    • (Alternatively, you can go to your dashboard and click Create a New Project

  2. Enter a unique name for your Project and click OK

Files

Synapse files can be created by uploading content from your local computer or linking to digital files on the web. You can annotate files with custom metadata, embed files into Synapse wiki pages, or associate them with a DOI. 

Files in Synapse always have a “parent”, which could be a project or a folder. You can organize collections of files into folders and sub-folders, just as you would on your local computer.

Synapse files (as well as folders and projects) are identified by a unique identifier called a Synapse ID, or synID. The synID is represented by the prefix “syn” followed by numbers (for example, syn12345678). This identifier can be used to refer to a specific file on the web and through the programmatic clients.

You can control who has access to files that you upload in two ways. First, you can apply sharing settings, which control who can view, edit, download, or delete a file. Second, you can also apply conditions for use, which are additional requirements that Synapse users must meet before accessing your file. By default, files inherit the conditions for use of the Synapse folder where they are uploaded. You can also add additional conditions for use on specific files within a folder.

To read more about how to use sharing settings and conditions for use together, see the full article on Sharing Settings, Permissions, and Conditions for Use.

Folders

Folders offer an additional way to organize your data. Instead of uploading a bunch of single files into your project, you can create folders to separate your data in a systematic way. You can create a folder within a folder within a folder, and so on.

Creating a Folder

To create a folder:

  1. Within your Synapse project, click the Files tab

  2. To create a folder at this level*, click the Files Tools menu, followed by Add New Folder

  3. Enter a folder name and click Save

*To create a folder within a folder, click on the folder name within the project, and click the Folder Tools menu, followed by Add New Folder, and proceed to step 3.

Uploading a File via the Synapse UI

To upload a file:

  1. Within your Synapse project, click the Files tab

  2. Click the Files Tools menu (or, to upload a file to a folder, then first go to the folder, and click the Folder Tools menu), followed by Upload or Link to a File

  3. Click Browse to select the file, or drag and drop it to upload, or click Link to URL to add a link to an external storage space

  4. Click Save

Uploading a File Programmatically

You can annotate your data during the upload process or after. Find more information here.

Command line

The command line has the sub-command store, which can be used to upload and add a file to Synapse.

# Add a local file to an existing project (syn12345) on Synapse synapse store raw_data.txt --parentId syn12345

Python

The main function for uploading or updating a file in the Python client is through the store function. For more information about this function, see the Python Docs.

import synapseclient from synapseclient import File syn = synapseclient.login() # Add a local file to an existing project (syn12345) on Synapse file = File(path='/path/to/raw_data.txt', parent='syn12345') file = syn.store(file)

R

The main function for uploading or updating a file in the Python client is through the synStore function. For more information about this function, see the R Docs.

library(synapser) synLogin() # Add a local file to an existing project (syn12345) on Synapse file <- File(path='/path/to/raw_data.txt', parentId='syn12345') file <- synStore(file)

Moving a File or Folder via the Synapse UI

Using the Synapse UI, you can move single files one at a time, or you can move an entire folder and all of its file contents. You cannot move more than one folder at once. For this reason, if you are primarily using the web interface to manage your files, you should consider your folder structure carefully when you create your initial project.

To move a file or folder:

  1. Within the file or folder, click File Tools or Folder Tools, followed by Move File or Move Folder

  2. In the resulting pop-up window, browse for the destination folder or project or use the search bar to enter the Synapse ID of the new location

Moving a File or Folder Programmatically

Command line

The command line client has a sub-command mv which can be used to move files and folders.

# move a file or folder (syn123) to a different folder/project (syn456) synapse mv --id syn123 --parentId syn456

Python

You can change the file or folder location in the Python client by fetching the file information, changing the parent property, and storing the change.

import synapseclient syn = synapseclient.login() # fetch the file/folder to move (syn123 in this example) # note the downloadFile=False parameter to fetch only the file's metadata and not the entire file foo = syn.get('syn123', downloadFile=False) # change the parentId to the new location, can be a folder or project (syn456 in this example) foo.properties.parentId = 'syn456' # store the file/folder to move it syn.store(foo)

R

You can change the file or folder location in the R client by fetching the file information, changing the parent property, and storing the change.

library(synapser) synLogin() # fetch the file/folder to move (syn123 in this example) # note the downloadFile=False parameter to fetch only the file's metadata and not the entire file foo <- synGet('syn123', downloadFile = FALSE) # change the parentId to the new location, can be a folder or project (syn456 in this example) foo$properties$parentId <- 'syn10056031' # store the file/folder to move it synStore(foo)

Deleting a File or Folder via the Synapse UI

Deleting a file will permanently remove it from Synapse. The synID that was associated with the deleted file will not be re-used. To delete a file:

  1. Within the project, click the Files tab

  2. Click the file or folder you want to delete (or the file/folder within a folder)

  3. Click File Tools or Folder Tools, followed by Delete File or Delete Folder

Deleting a File or Folder Programmatically

Deleting a file will permanently remove it from Synapse. The synID that was associated with the deleted file will not be re-used.

Command line

synapse delete syn56789

Python

import synapseclient syn = synapseclient.login() entity = syn.delete("syn56789")

R

library(synapser) synLogin() entity <- synDelete("syn56789")

File Previews

Synapse has a viewer for certain file types so that you can preview them before you download. Synapse previews will show the contents of the following file types:

  • PDFs

  • CSV (may show a portion of the table only)

  • Videos (.mp4, .webm, .ogg)

  • Jupyter notebooks (.ipybn)

  • Images (.jpeg, .png, .gif, .svg)

  • HTML (limited rendering only)

  • Markdown (.md or .rmd)

To see a file preview, navigate to the Files tab of a project and click on the file name. If a file preview is available, it will be visible in the left window.

Deleting a Project

Deleting a project will permanently remove it from Synapse. The synID that was associated with the deleted project will not be re-used. To delete a project:

  1. Navigate to the project you want to delete

  2. Within the project, click the Project Tools button

  3. Click Delete Project

  4. In the dialog box, click the red Delete button to permanently delete the project, or click Cancel if you do not want to delete the project