...
On the surface it would make sense to go for this re-factoring as this would automatically inherit the entity query engine. While this sounds easy in theory, in practice refactoring the system this way would most likely introduce more issues than it solves. Among others the :
The ACL scheme adopted by the evaluation API is completely different (for good reasons)
...
. Different access levels are required for submissions vs evaluations and in general the submission do not inherit from the evaluation.
The life cycle of the evaluations objects is different than the one of entities
...
: for example submissions are immutable and cannot be deleted (but from the evaluation manager) and their management is driven by the evaluation they are associated with.
Additional and dedicated APIs are needed for evaluations, for example to inspect submission eligibility. While their basic CRUD operations could be managed through the entity APIs we risk having dedicated APIs on top that work on evaluations which might be confusing.
It would also introduce several complexities around the the user experience, how do we represent these objects? Do we need to filter them in a project? Do we put them along side the project itself (e.g. See docker repository)?
Documentation: If would be hard from an API perspective to document their usage and constraints
Some advantages if we model evaluation and submissions as entities:
We can automatically index annotations using a common and known API
Views could be created both on top of evaluations and submissions
We wouldn’t have to (heavily) alter the view engine to support external objects (we would still need to introduce add in the replication index system properties that are tied to evaluations and submissions)
Proposed solution: decouple entities and annotations
...
The rest of the table APIs should largely remain the same with minor additions to accommodate the new view type. For example we might want to introduce a new ColumnType to render submission ids or evaluation ids.
Submissions Schemas
In the future with the introduction of schemas for objects we could introduce binding of a schema to the annotations on submissions. In particular we could bind a schema to an evaluation so that the submissions annotations would abide by the rules set on the schema. At the same time creating a view on submissions could be guided by the same schema.