One of the features to the Synapse.org platform is the ability to host a crowd-sourced challenge. Hosting challenges are a great way to crowd-source new computational methods for fundamental questions in systems biology and translational medicine.
...
When using copyWiki, it is important to specify the destinationSubPageId parameter. This ID can be found in the URL of the live project, where it is the number following .../wiki/.
Once copyWiki has been used once, DO NOT RUN IT AGAIN!
Once the wiki has been copied over, all changes to the live project should now be synced with challengeutils' mirrow-wiki.
...
Column Name | Description | Facet values? |
|---|---|---|
| Evaluation ID (evaluation ID, but rendered as evaluation name) – recommended for SubmissionViews with multiple queues in scope |
|
| Submission ID | |
| Date and time of the submission (in Epoch, but rendered as | |
| User or team who submitted (user or team ID, but rendered as username or team name) |
|
| Docker image name – recommended for model-to-data challenges |
|
| Docker SHA digest – recommended for model-to-data challenges |
|
| Workflow status of the submission (one of [ |
|
| Evaluation status of the submission (one of [None, |
|
| (if any) Validation errors for the predictions file |
|
| synID to the submission’s logs folder |
|
| synID to the predictions file (if any) |
|
(any annotations related to scores) | Submission annotations - names used depends on what annotations were used in the scoring step of the workflow |
Recommended
Not recommended...
3. Click Save. A table of the submissions and their metadata will now be available for viewing and querying. Changes to the information displayed can be edited by clicking on the schema icon, followed by Edit Schema:
...
Update the Challenge
...
Updating Existing Wikis
1. Make whatever changes needed Go to the staging project.2. Use challengeutils' mirror-wiki to push the changes to the live project and navigate to the page(s) you wish to edit. Click on the pencil icon to Edit Project Wiki:
...
Make edits as needed, then click Save.
2. Use challengeutils' mirror-wiki to push the changes to the live project:
| Code Block |
|---|
challengeutils mirror-wiki staging_synid live_synid [--dryrun] |
Use --dryrun to optionally preview which pages will be updated prior to doing an official sync.
Adding a New Wiki Page
1. Go to the staging project, click on Wiki Tools > Add Wiki Subpage. Enter a page title, then click OK. A new page should now be available.
2. On the new page, click on the pencil icon to Edit Project Wiki:
...
Add the page content, then click Save.
3. Go to the live project and create a new wiki page with the same name as the new page in the staging project. mirror-wiki depends on the page titles to be the same for synchronization.
4. Use challengeutils' mirror-wiki to push the changes to the live project:
| Code Block |
|---|
challengeutils mirror-wiki staging_synid live_synid [--dryrun] |
...
It is not unheard of for there to be a change in the submission deadline. To change extend the submission deadline date/time, you can either:
edit the Round End of an existing round; or
add a new round that will immediately start after the current one (note that this approach will reset everyone’s submission limit)
Updating the Workflow
...
For any changes to the CWL scripts or run_docker.py, make edits as needed to the scripts, then push the changes. We highly recommend conducting dryruns whenever there is a change to the workflowimmediately after, so that errors are addressed in a timely manner.
...