Enhanced Form Submission Workflow

1 Introduction
2 Current Challenges
3 Proposed Solution
- 3.1 REOPENED State
- 3.2 ARCHIVED State
- 3.3 allowEditingRejectedForms Configuration
4 Proposed API Changes:
- 4.1 New Endpoints
- 4.2 Schema Updates
  - 4.2.1 FormReopenRequest (new)
  - 4.2.2 SubmissionStatus (update)
  - 4.2.3 StateEnum (update)
  - 4.2.4 FormGroupConfig (new)
  - 4.2.5 FormGroup (update)

Introduction

Reference JIRA:

PLFM-8583 - Getting issue details... STATUS
PLFM-8672 - Getting issue details... STATUS

The current form services API adopts a state machine workflow to handle its form submission status:

WAITING_FOR_SUBMISSION: The initial state where the user can freely edit the form.
On submit → Transitions to SUBMITTED_WAITING_FOR_REVIEW.
SUBMITTED_WAITING_FOR_REVIEW: The form is locked for the user and visible to reviewers.

On accept → Transitions to ACCEPTED.

On reject → Transitions to REJECTED.

ACCEPTED: A terminal state indicating that the submission was approved and is now immutable
REJECTED: A state indicating that the submission was not approved (with a rejection message from the reviewer). The user that created the form is allowed to edit a rejected form. The update transitions the form back to WAITING_FOR_SUBMISSION.

Currently, if a reviewer wants to ask for changes, their only option is to reject the form. The submitter can then edit the rejected form, which transitions it back to the WAITING_FOR_SUBMISSION state, effectively starting a new submission (since the form is not shown in the reviewer dashboard).

Current Challenges

Based on the feedback from

PLFM-8583 - Getting issue details... STATUS

the current workflow presents two main challenges:

The "reject-to-edit" workflow does not seem to be very intuitive. It forces the reviewers to transition submissions to a (seemingly) terminal REJECTED state. Additionally, the submission disappears from the reviewer's dashboard once the user updates the form.
Cluttered dashboard: There is no mechanism for reviewers to hide or archive old, irrelevant, or test submissions. This can lead to a cluttered interface, making it difficult to manage reviews.

Proposed Solution

To address these challenges, we propose to introduce two new states (REOPENED, ARCHIVED), two new form endpoints (/reopen, /archive) and a new configuration option for a FormGroup.

`REOPENED` State

We can introduce a new status (REOPENED) to create a controlled feedback loop:

A reviewer can transition a form from SUBMITTED_WAITING_FOR_REVIEW to REOPENED using a new /reopen endpoint. The reviewer will provide a feedback message during this operation.
The operation allows the submitter to update their form
The form will remain visible to the reviewers
Once the form is updated the user can re-submit the form using the existing API

`ARCHIVED` State

To solve the clutter problem, we will introduce an ARCHIVED state:

A reviewer can move any form that is ACCEPTED or REJECTED to ARCHIVED using a new /archive endpoint.
This is a non-destructive action that preserves the form data but hides it from the default list view in reviewer applications (The UI can exclude by default the ARCHIVED forms from the list API call)
The UI can provide an option to "Show Archived" forms if needed.

`allowEditingRejectedForms` Configuration

To create a more consistent and predictable workflow, we also propose adding a configuration option, allowEditingRejectedForms for a FormGroup:

Group form administrator can decide if a REJECTED form is final and immutable or if the user is allowed to make edits.
We can default to allowing editing rejected forms to maintain backward compatibility.
This allows for better control of the feedback loop without the REJECTED ambiguity. Additionally, it removes the unexpected behavior where a rejected form can disappear from the reviewer dashboard.
Notice that the current form group API is idempotent and does not allow “updating” a form group, therefore we might need a new API for the form group configuration.

The new state machine diagram would look like the following:

Proposed API Changes:

New Endpoints

API	Request Body	Response Body	Description

API

Request Body

Response Body

Description

PUT /repo/v1/form/data/{id}/reopen

FormReopenRequest

FormData

Allows reviewers with the READ_PRIVATE_SUBMISSION permission on the form group to transition a submitted form to the REOPENED state, enabling the original submitter to make edits based on reviewer feedback. Only forms in the SUBMITTED_WAITING_FOR_REVIEW state can be reopened.

PUT /repo/v1/form/data/{id}/archive

N/A

FormData

Allows reviewers with the READ_PRIVATE_SUBMISSION permission on the form group to transition a form to the ARCHIVED state. Only forms in the ACCEPTED or REJECTED states can be archived.

PUT /repo/v1/form/group/{id}/config

FormGroupConfig

Allows to set configuration options on an existing FormGroup. Only users with UPDATE permission of the form group can change its configuration (existing form groups will need to be backfilled).

The configuration will be returned as part of the FormGroup.

Schema Updates

FormReopenRequest (new)

Contains a message with the reviewer feedback when reopening a submission.

{
    "concreteType": "org.sagebionetworks.repo.model.form.FormReopenRequest",
    "description": "Request to reopen a submitted form for editing that includes feedback for the submitter."
    "properties": {
      "message": {
        "type": "string",
        "description": "Feedback message to provide to the submitter explaining why the form was reopened for revision. Limit 1000 characters or less."
      }
    }
}

SubmissionStatus (update)

Stores the reopenedMessage with the reviewer feedback.

{
	"description": "The status of a a submitted FormData object.",
	"properties": {
		"submittedOn": {
			"type": "string",
			"format": "date-time",
			"description": "The date when the object was submitted."
		},
		"reviewedOn": {
			"type": "string",
			"format": "date-time",
			"description": "The date when this submission was reviewed."
		},
		"reviewedBy": {
			"type": "string",
			"description": "The id of the service user that reviewed the submission."
		},
		"state": {
			"description": "The current state of this submission.",
			"$ref": "org.sagebionetworks.repo.model.form.StateEnum"
		},
		"rejectionMessage": {
			"type": "string",
			"description": "The message provided by the reviewer when a submission is rejected."
		},
        "reopenedMessage": {
            "type": "string",
            "description": "The message provided by the reviewer when a submission is reopened."
        }
	}
}

StateEnum (update)

Additional REOPENED and ARCHIVED states

{
	"description": "The enumeration of possible FormData submission states.",
	"name": "StateEnum",
	"type": "string",
	"enum": [
		{
			"name": "WAITING_FOR_SUBMISSION",
			"description": "Indicates that the FormData is waiting for the creator to submit it."
		},
		{
			"name": "SUBMITTED_WAITING_FOR_REVIEW",
			"description": "Indicates the FormData has been submitted and is now awaiting review."
		},
		{
			"name": "ACCEPTED",
			"description": "The submitted FormData has been reviewed and accepted."
		},
		{
			"name": "REJECTED",
			"description": "The submitted FormData has been reviewed but was not accepted.  See the rejectedReason for more details."
		},
        {
			"name": "REOPENED",
			"description": "The submitted FormData has been reviewed but was reopened for additional input.  See the reopenedReason for more details."
		},
        {
			"name": "ARCHIVED",
			"description": "The FormData has been archived by a group admin.  See the archivedReason for more details."
		}
	]
}

FormGroupConfig (new)

Allows the group administrator to decide if the REJECTEDforms can be edited by the submitter.

{
    "description": "Configuration settings for a FormGroup.",
    "properties": {
        "allowEditingRejectedForms": {
            "type": "boolean",
            "description": "When true, users can edit a form that is in the REJECTED state. When false, REJECTED is a terminal state and the user cannot perform and changes to the form.",
            "default": true
        }
    }
}

FormGroup (update)

Add the config property that contains the current FormGroupConfig for the form group.

{
	"description": "All form data belongs to a form group.  Access to the group is managed via the ACL of the group.",
	"properties": {
		"groupId": {
			"type": "string",
			"description": "Unique identifier provided by the system."
		},
		"name": {
			"type": "string",
			"description": "Unique name for the group provided by the caller."
		},
		"createdBy": {
			"type": "string",
			"description": "Id of the user that created this group"
		},
		"createdOn": {
			"type": "string",
			"format":"date-time",
			"description": "The date this object was originally created."
		},
        "config": {
            "$ref": "org.sagebionetworks.repo.model.form.FormGroupConfig"
            "description": "The current configuration for the form group."
        }
	}
}