NOTE: The Competition services are still under development and are not yet available on Synapse.
Overview
The Competition API is designed to support open-access Synapse data analysis and modeling challenges. This framework provides tools for administrators to create new competitions in Synapse and manage the users and data objects associated with a competition. A competition owner can control access to competition resources and registration, as well as track user participation and evaluate submitted models. This API will support user-facing interfaces to monitor competitions including activity notifications and on-demand scoring and leaderboarding.
Data Models
Synapse Competitions are constructed around three primary objects: Competition, Participant, and Submission. These objects are defined by eponymous JSON schema files in the Synapse Repository Services.
Competition
Object representing a Synapse Competition
- ID - unique ID for the Competition
- eTag - optimistic concurrency flag; changes with each update
- ownerID - user ID of the owner of the Competition
- name
- description
- contentSource - ID of the data source (e.g. entity) for the Competition (string)
status - enum describing the state of the Competition, one of: PLANNED, OPEN, CLOSED, COMPLETED
Participant
Immutable object to represent a single user participating in a Competition. Competitions can have many users, and users can participate in many Competitions.
- ownerID - user ID of the Participant
- competitionID
- createdOn - join date
Submission
Immutable object to represent an entity submission to a Competition. A participant can create many Submissions for a given Competition.
- ID - unique ID for the Submission
- ownerID - user ID of the Submission owner
- competitionID
- entityID - ID of the submitted entity
- version - specific entity version
- createdOn - submission date
SubmissionStatus
Object to track the status and score of a specific Submission. This object is generated at the time of submission, and can be modified by Competition administrators.
- submissionID
- eTag - optimistic concurrency flag; changes with each update
- status - enum describing the state of the Submission, one of: OPEN, CLOSED, SCORED, INVALID
- score - numerical score for this Submission (long)
REST API
Competition
| URL | HTTP Type | Description |
|---|---|---|
| /competition | POST | Create a new Competition |
| /competition/{competitionId} | GET | Fetch a Competition |
| /competition/{competitionId} | PUT | Update a Competition |
| /competition/{competitionId} | DELETE | Delete a Competition |
Participant
| URL | HTTP Type | Description |
|---|---|---|
| /competition/{competitionId}/participant | POST | Join as a new Participant in the Competition specified by competitionId |
Submission
| URL | HTTP Type | Description |
|---|---|---|
| /competition/submission | POST | Create a new Submission (and a corresponding SubmissionStatus object) |
| /competition/submission/{submissionId} | GET | Fetch a Submission |
| /competition/submission/{submissionId}/status | GET | Get the status of a Submission |
| /competition/submission/{submissionId}/status | PUT | Update the status of a Submission |
Example Workflow
The following is a proposed workflow for interacting with the Competition Services. Please note that some components are still under development.
User Perspective
- Find a Synapse Competition
- Sign up as a Participant
- Download relevant Synapse Entities referenced in Competition
- Submit a Synapse entity to the Competition
- Track status of personal Submissions and view global competition status/leaderboard
- Earn achievements for participation / success
Owner/Admin Perspective
- Create a Synapse Competition
- Fill in details, attach relevant Synapse entities
- Specify access requirements
- Specify open/close dates
- Listen for Submissions (using messaging service)
- Download unscored Submissions
- Perform scoring
- Push scores back to each SubmissionStatus
- Track user participation and publish competition status/leaderboard
- Award achievements for user participation / success
Planned Additions
We plan to include the following features in the Competition Services, but these features have not yet been implemented.
- Banned user lists
Banned users should be prohibited from participating in one or more Competitions, and any Submissions by a banned user should be isolated from the general populations of Submissions. - Customizable Amazon Simple Notification Services messaging tools
Competition administrators should have tools to configure a selective Amazon instance to listen for messages pertaining to Competitions which they own. These messages should be sent to a queue dedicated to Competitions. - Immutable copies of the submitted Entities
Once an Entity is submitted to a Competition, a static copy of that Entity should become the property of the Competition administration. - Administrator lists for Competitions
Competitions should support administration by multiple users (rather than just the Competition owner). These admin lists should function as a lightweight ACL of sorts. - Multi-author Submissions
Competitions should support team Submissions, where more than one Synapse user claims authorship of a given Submission.