At the end of every Challenge, we aim to collect writeups from all participating users/teams, in which they will provide information about their submission(s). Currently, writeups are collected as Synapse Projects (sample template here), and we ask the participants to include information such as their methodologies, external data sources (if any), and source code and/or Docker images. Typically, writeups are *required* in every Challenge, in order to be considered for “top performer” eligibility and other incentives, including byline authorship.
This guide will outline the steps to collect and validate writeups on Synapse.
...
Setup
Requirements
Synapse account
(for local testing) CWL runner of choice, e.g. cwltool
Access to cloud compute services, e.g. AWS, GCP, etc.
Steps
1. On the live version of the Challenge site, go to the Challenge tab and check whether there is already an Evaluation queue for collecting writeups (skip to Step 4 if so).
...
Line Number | TODO | Motivation |
|---|---|---|
31 | Update | This will identify the Synapse team that are the organizers for the Challenge. |
45 | Update | This will check that the participant/team did not submit the Challenge site as their writeup. |
47-50 | Uncomment. | Lines 49-50 checks that the writeup Project is AT LEAST accessible to the Organizers team (as defined by Line 31). Lines 47-48 checks the writeup is accessible to anyone on the web. |
Push the changes up to GitHub when done.
| Info |
|---|
The next couple of steps are also described in the Creating and Managing a Challenge guide. |
...
b) create a new instance (a t3.small would be sufficient enough) and setup Orchestrator on that machine. See Steps 7-9, 11 of Creating and Managing a Challenge for more details on how to set this up.
Submit Writeup Dry-runs
Use and/or create a new Synapse project and submit it to the writeup queue. (We recommend saving this Project for re-use of future writeup queues setup).
...
Test Case | Workflow Configurations | Expected Outcome |
|---|---|---|
Submitting the Challenge site |
| |
Submitting a private Synapse project | Lines 49-50 is used (writeup should be accessible to the Organizers team) |
|
Submitting a private Synapse project | Lines 47-48 is used (writeup should be publicly accessible) |
|
Submitting a Private Synapse project that is shared with the Organizers team | Lines 49-50 is used (writeup should be accessible to the Organizers team) |
|
Submitting a Private Synapse project that is shared with the Organizers team | Lines 47-48 is used (writeup should be publicly accessible) |
|
Submitting a public Synapse project | Lines 47-48 and/or lines 49-50 are used |
|
Once you are satisfied that the writeup workflow is to your expectations, remember to open the queue to the Challenge participants!
You can do so by updating its Sharing Settings so that the Participants team has Can submit permissions.