The purpose here is to start a discussion and engage some design thinking about how to automate more of the process of creating and administrating a Challenge on Synapse.
...
- Work towards a more plug-n-play way to run scoring scripts
- Automate launch and provisioning of scoring machine AWS instance
- Schedule scoring script
- Define email templates (validation failure or reporting successful scoring) for feedback to participants
- Collect errors, invalid submissions, submission statistics, perhaps as an extension of the Challenge Statistics page.
Ideas that are really far-out and wacky
...