Users

• Challenge Organizers - A superset of roles that develop and administer individual challenges

  • Challenge Initiator - An individual who wants to create/host a challenge. This user is the person who will create the challenge page and write the challenge materials. They will also engage with the challenge community to generate interest and participation. This user is most likely the driving energy behind a challenge and may be a funder, data provider, scientific lead, or project manager.

  • Challenge Reviewer - This user is responsible for reviewing the draft challenge pages prior to deploying them to production. This may be the PI on the challenge project. They are one of the Challenge Organizers.

  • Challenge Project Manager - This user is responsible for coordinating between all the other Challenge Organizers.

  • Challenge Scientific Lead - This user is one of the Challenge Organizers and is responsible for developing the scientific question and evaluation methods for the challenge.

  • Challenge Infrastructure Lead - This user is responsible for building and managing the infrastructure that runs the challenge. This may be remote servers that run the orchestrator tool, or they may be AWS environments running the orchestrator. This may involved coordinating between multiple institutions to run multiple remote servers.

  • Challenge Data Provider - This user is the point of contact at the organization/lab providing data for the challenge.

  • Challenge Funder - This user is the organization The organization(s) funding the challenge. They are one of recognized by attaching their logo to the Challenge Banner and in the Challenge Organizers. (Should this be an organization or the user representing the organization?)

  Prospective Challenge Participant

  • description. Some members of the funding organization(s) often serve as Data Providers or Scientific Leads.

  • Challenge Governance Liaison - This user is a

  potential challenge participant who needs to be able to explore current, past, and future challenges. They would need to know what the different questions were, what data was available, what the relevant domain expertise of the challenge was

  • contact on the Sage governance team who is responsible for review the governance policies of the challenge. They will also be responsible for creating click wraps and gathering signatures for the different MOUs, DTAs, or DURs.

• Challenge Participant - This user is the researcher, student, or data scientist taking part in the challenge. This user needs to download data, create models, and submit either a prediction file, a docker image, or some other type of submission. This is a broad role with some special cases:

  • Benchmarking Participant - Much like a challenge participant, this user is the researcher, student, or data scientist taking part in a benchmarking project.

  • Prospective Challenge

  Archivist

  • Participant - This user is

  responsible for archiving the final challenge.

• Challenge Results Consumer - This is a challenge participant who is checking the results of the challenge. They will look at the leaderboard and check to see how their methods ranked in comparison to the other submissions.

• Challenge Governance Liaison - This user is a contact on the Sage governance team who is responsible for review the governance policies of the challenge. They will also be responsible for creating click wraps and gathering signatures for the different MOUs, DTAs, or DURs

  • a potential challenge participant who needs to be able to explore current, past, and future challenges. They would need to know what the different questions were, what data was available, what the relevant domain expertise of the challenge was.

  • Challenge Team Owner - A Challenge Participant with privileges over Team membership, or an individual (a team of one)

  • Challenge Team Member - A Challenge Participant who coordinates submissions with other members of a Team.

  • Challenge Team Observer - A Challenge Participant who can view the submissions and scores of a Team, but cannot make submissions on behalf of the Team. For example, a Teacher may be an observer for several Teams of students.

• Challenge Archivist - This user is responsible for archiving the final challenge, and ensuring that metadata is accessible.

• Organization Manager - This user is the manager for the organization’s account on the challenge platform.

• Challenge Observer - A user who wishes to browse challenges to understand current activity in the space. Challenge Observers may or may not have login credentials and may have no interest in participating in a challenge, only in reading the public information about challenges.

  • Challenge Results Consumer - This is a challenge participant or observer who is checking the results of the Challenge. They will view the results of a Challenge during open submissions (if permitted by the Challenge settings) or after a Challenge has completed.

Responsibilities

• Data quality control

• Evaluation metric design

• Discussion board monitoring

• Infrastructure dry runs

...