Page Comparison

Motivations:

Existing Evaluations have a very inflexible way to define rounds:

...

...

Source material:

...

Source material:

UI Design:
https://www.figma.com/file/E5zIAPsI2fagDc8btWGU06Ol/Challenge-Evaluation-Queue

Use case/desired behavior:
https://docs.google.com/document/d/19M9og2JW7CaG7YLNi8cCRhFuFwF-y6-nx4CJDAwr6qw/edit#heading=h.y0949lyt5na0

Motivations:

Existing Evaluations have a very inflexible way to define rounds:

{
	"name":"SubmissionQuota",
	"description":"Maximum submissions per team/participant per submission round.  If round information is omitted, then this indicates the overall submission limit per team/participant.",
	"properties":{
		"firstRoundStart": {
			"type": "string",
			"format": "date-time",
			"description": "The date/time at which the first round begins."
		},
		"roundDurationMillis":{
			"type": "integer",
			"description":"The duration of each round."
		},
		"numberOfRounds":{
			"type": "integer",
			"description":"The number of rounds, or null if there is no end."
		},
		"submissionLimit": {
			"type": "integer",
			"description":"the maximum number of submissions per team/participant per round."
		}
	}
}

...

• Store rounds List as JSON in a single database column.

  • This would require searching for the correct round based on start/end dates in memory

  • Evaluations object can be cleanly retrieved using query on single table

• Separate Table For Rounds (primary key is evaluationId , ID) index on roundStart and roundEnd

  • Allows us to pull out one specific limit

  • Separate queries

    • query to retrieve list of all rounds for that evaluation,

      • On REST API GET, we care about all rounds

    • query to retrieve list single evaluation id,

      • On challenge submissions during the submission limit enforcement, we only care about the specific round whose start/end interval encapsulate the current time

    • query to pull from the Evaluations table

  • Limits are enforced in Java code so we can store the them as JSON

  • Question: should ID be created by the id-generator? or just index in rounds list?

    • generated unique ID makes it easy to perform updates on evaluations

      • with current API setup a list of round are submitted, making this pointless

    • index in round list as ID

      • this allows easier tagging of submission

      • updating rounds involves row deletion and addition if added

        • hash metdata to avoid pointless delete/add?

    • If we use roundStart, roundEnd to identify the round config.

      • any change to start,roundend would mean an insertion/deletion

 {
         "id":"2",
         "roundStart":1231412213213123123,
         "roundEnd":"2020-08-11T16:45:10−08:00",
         "submissionLimit":{
            "totalSubmissionLimit":40,
            "dailySubmissionLimit":8,
            "weeklySubmissionLimit":12,
            "monthlySubmissionLimit":{
               "dayOfMonth":31,
               "limit":20
            }
         }
      }

...

"limit":20
            }
         }
      }

Evaluation_maintenance table

Submission table

Add EVALUATION_ROUND, which is the round number during which it was submitted. This will be replicated into Submission Views.

...