Page Comparison

This guide helps organizers create a space within Synapse One of the features of the Synapse.org platform is the ability to host a crowd-sourced Challenge. Challenges on Synapse are meant to be community competitions, designed to crowd-source new computational methods for fundamental questions in systems biology and translational medicine.

...

A Challenge space provides participants with a Synapse project to Running a Challenge on Synapse will require creating a Challenge space for Participants to learn about the Challenge, join the Challenge community, submit entries, track progress, and view results. This article is aimed at Challenge Organizers, and will focus on:

• Challenge Setting up the infrastructure setup

• Launching and updating the Challenge launch

• Submission Monitoring Challenge updatethe submissions

...

Before You Begin

...

Required Compute Power

At Sage Bionetworks, we generally provision an AWS EC2 Linux instance for to run infrastructure of a Challenge that leverages , leveraging the SynapseWorkflowOrchestrator to run CWL workflows.  These workflows will be responsible for evaluating and scoring submissions (see model-to-data-challenge-workflow GitHub for an example workflow).  If Sage Bionetworks .

• If Sage is responsible for providing the cloud compute services

...

• : we ask that you give us a general estimate of the computing power needed (e.g. memory, volume, GPU required?, etc)

...

By default, up to ten submissions can be evaluated concurrently, though this number can be increased or decreased accordingly within the orchestrator's .env file.  Generally, the more submissions you want to run concurrently, the more power will be required of the instance.

Using Sensitive Data

...

• .

Using Sensitive Data

Synapse has the ability to apply access restrictions to sensitive data (e.g. human data), so that legal requirements are met before participants Participants can access it. If human data are being used in the Challenge, or if you have any questions about the sensitivity of the Challenge data, contact the Synapse Access and Compliance Team (act@sagebase.org) for support with the necessary data access procedures.

...

• Support for Docker and, if possible, docker-compose

  • If Docker is not allowed, then support for Singularity and Java 8 is requireda must

• SynapseWorkflowOrchestrator repository

If Sage is not allowed access to the server, then it is the Note that it will be the data contributor’s responsibility to get the orchestrator running in whatever environment chosen.  If Docker is not supported by the system, please let us know as we do have solutions for workarounds (e.g. using Java to execute, etc.).set up the infrastructure if Sage employees are not allowed access to the remote server. However, we are happy to consult if needed.

...

Challenge Infrastructure Setup

...

• Synapse account

• Python 3.7+

• synapseclient

• challengeutils

• (for local testing) CWL runner of choice, e.g. cwltool

• Access to cloud compute services, e.g. AWS, GCP, etc.

Outcome

Once the infrastructure is set up, the selected Evaluation queue(s) will continuously be monitored for new submissions. Once a submission is received, it will undergo evaluation including validation and scoring. All submissions will be accessible to the Challenge Organizers, including the Docker image (if model-to-data Challenge) and/or prediction files. Participants may periodically receive email notifications about their submissions (e.g. status, scores), depending on the infrastructure configurations.

Steps

1. Create a workflow infrastructure GitHub repository for the Challenge. For the orchestrator to work, the repository must be public.

...

When using copyWiki, it is important to specify the destinationSubPageId parameter.  This ID can be found in the URL of the live site, where it is the integer following .../wiki/<some number>.

Once copyWiki has been used once, DO NOT USE IT AGAIN!!

Following this action, all changes to the live site should now be synced over with challengeutils' mirrow-wiki. More on updating the Wikis under the Update the Challenge section below.

...

...