OAuth Client Verification

1 Verification Procedure
2 Initial Implementation
3 Proposed Approach for Synapse

With the implementation of

PLFM-4585 - Getting issue details... STATUS synapse can now act as an OAuth 2.0 Provider. In order for application clients (See roles in https://tools.ietf.org/html/rfc6749#section-1.1) to use Synapse as a resource server our implementation of the authorization server requires the client application to be registered before issuing authorization codes and access tokens.

Once a client is registered it can perform scoped operations on behalf of the user, at the time of writing we support the openid scope for the Open ID Connect /userInfo service. In the future we will extend the scope (and claims) that would allow access to data with the user consent.

One of the issues in the OAuth architecture is that while authentication and authorization is decoupled from a client application in a secure manner, establishing trust on the client application is not defined by the specification.

Since the data that synapse users can access might potentially be sensitive according to the scopes and claims, we need a way to establish trust with a client application by having a process in place to verify that the application will not misuse and/or abuse the system. There is not a standard (automated or manual) procedure for verifying a client in the industry and different companies take different approaches (or choose not to have a verification process).

Google and Facebook have both an automated and manual verification and review process, according to the type of client, scope and claims the client has access to. (See https://developers.google.com/apps-script/guides/client-verification and https://support.google.com/cloud/answer/9110914?hl=en).

Verification Procedure

As of November 2019 the OAuth Clients needs to be verified in order to be usable. A dedicated OAuth Verification Job in the ops build system has been created to perform verification of a specific client. The following procedure can be used in order to verify a client:

Once a request is received and the client needs to be verified, create a new ticket in JIRA with the client id
Launch the OAuth Verification Job filling out the required parameters (Note: you will need a Jenkins account):
1. SESSION_TOKEN: A valid session token for an admin user
2. CLIENT_ID: the id of the client to verify
3. ETAG: The etag of the client to verify (this is to ensure that the client didn’t change after the client details were read)
4. VERIFY_STATUS: Leave enabled (deselecting would un-verify a client)
Resolve the related JIRA issue

Initial Implementation

As from the design review meeting held on the 11th of November 2019 the synapse team decided that due to the number of expected use cases a complete verification process is not needed and the initial implementation will be based on white listing on a case by case the oauth clients:

Client will not be usable after creation until verified. If a client is being (e.g. to get an authorization code or access token) used while not verified an error message should include a the email to contact in order to verify the client.
In the Web Client a form to submit a Jira issue to the ACT team to request the client verification will be shown (as part of

SWC-4957 - Getting issue details... STATUS

)
A user should be blocked in the web client from using a non verified client (

SWC-5043 - Getting issue details... STATUS

)
An administrative API (Only ACT and admins can invoke this endpoint) will be added to verify a client:
PUT /admin/oauth2/client/{id}/verified?status=<boolean>

The initial proposed design below is still valid and can be taken into account in the future when/if the verification will be extended.

Proposed Approach for Synapse

We propose to introduce a (manual) verification step for application clients that are registered in synapse which is similar to the user verification (https://sagebionetworks.jira.com/wiki/spaces/PLFM/pages/82935813/Verified+User). The manual verification will be implemented on top of an automatic validation of ownership of the domain in the redirect uri(s) submitted by the client. We allow registration and usage without verification but the web client should show a banner if a user consenting to an application that has not been verified. The banner should clearly state that the application is not verified and discourage the user to consent.

Allowing the usage of a client which is not verified can be necessary for testing (note that we could think of a sandbox/public environment for clients but for the time being we want to keep the process as simple as possible).

When a client register an application in synapse currently the following required information is supplied:

Name of the client
Redirect URIs + (Optional) Sector Identifier URI (See https://openid.net/specs/openid-connect-registration-1_0.html#ClientMetadata)

We do not require the response_type as we default to "code" nor the the application_type (which defaults to "web").

Additionally the following optional information can be supplied:

client_uri: URL of the home page of the Client
policy_uri: URL that the Relying Party Client provides to the End-User to read about the how the profile data will be used
tos_uri: URL that the Relying Party Client provides to the End-User to read about the Relying Party's terms of service

The information above can all be useful to establish trust in a client. We propose the following pre-conditions for submitting a verification:

The client, policy and tos uris should be filled out
The client, policy and tos uris domain should match at least one of the domains in the list of redirect uris (As suggested in https://openid.net/specs/openid-connect-registration-1_0.html#Impersonation). This should be validated at registration time.
The redirect uris scheme should be https and the host should not be localhost or loopback (127.0.0.1, ::1). Note that a client can still be registered with a localhost or loopback for testing, but verification would not be available.
A natural language description of the application must be included in the submission

The verification of a client will be performed manually and approved or rejected by a member of a designated team (ACT?). The user that submitted the verification should be able to check the verification status at any time.

The designated team will be able to access the list of submissions along with the domain verification information. A submission shall not be approved if the domain verification is not complete.

In case of rejection a reason must be included explaining why. A user should be able to update the client and perform another submission after being rejected.

Domain validation

In addition to the manual approval or rejection, we should establish ownership of the domain referred to by the redirect uri (s) in the client as an additional trust layer. Note that while the specification does not enforce the scheme for redirect uris for web application types (See application_type in https://openid.net/specs/openid-connect-registration-1_0.html#ClientMetadata), it allows custom schemes for "native" applications. Since we do not support native applications and the specification does not explicitly forbid it, we can enforce the redirect uri scheme to be https. In this way we can assume that we should always be able to verify the domain ownership.

An exception applies if the host in the redirect uri is the loopback, this might be used for two reasons:

Testing
Native Applications

We currently do not support 2. but we probably should in the future to allow our own clients to authenticate using OAuth 2.0 (With PKCE https://tools.ietf.org/html/rfc7636). For a native client the the loopback might be used as the redirect uri: the reason is that the user authentication should be executed through a third party user-agent (such as the browser) and not the application itself (See https://tools.ietf.org/html/rfc8252 for best practices), since the application that launches the browser to authenticate the user needs to receive a callback the idea is to start a temporary server on the device listening on an available port and wait for the callback from the browser. It's also worth mentioning that there are other implementations that achieve the same goal without the loopback "workaround". An example is the heroku cli which releases a temporary authorization code and the cli performs a long polling operation waiting for the callback from the server itself with the verified code when the user performed the login (See https://github.com/heroku/heroku-cli-command/blob/master/src/login.ts for its the implementation). This same mechanism could be adopted for our clients. For native clients (if supported) we might decide not to validate the redirect uri.

For the time being we can enforce the scheme for a redirect uri to be https at registration time or http for the loopback (for testing). For the domain validation we can adopt a common solution that involves generating a unique validation code that needs to be placed on the target server and verified by our backend. We will not support DNS record based validation for the time being.

There exists a standard HTTP challenge used by lets encrypt (https://letsencrypt.org/docs/challenge-types) for automatically certify domain ownership when in the context of automated certificate management, this would not be practical in our case as it requires a complex configuration of the server and installation of an agent. Instead we opt for a simple validation of a code generated by the server that is served in a file and should be available at the following location:

https://<redirect_uri_domain_common_name>/synapse/<VALIDATION_CODE>.txt

The <VALIDATION_CODE> is a code generated at submission time (the user should be able to retrieve this code for the verification), the server should reply for a GET request with a 200 status and the body content should contain the <VALIDATION_CODE> as well. We do not put constraints on the content-type or any other headers, as long as the code matches the one generated by the server the domain is considered validated.

We do not need to generate a file handle in the backend, we can generate this file on the fly if needed (e.g. the user might want to download the file instead of creating it from the validation code) or let the web client generate it for the user (?).

Note that while we can use a .well-known directory, I could not find a suitable one in the official registry: https://www.iana.org/assignments/well-known-uris/well-known-uris.xhtml (Godaddy uses pki-validation, Lets Encrypt use their own ACME challenge). I'm not sure at this point if it's ok for us to still use the .well-known path. If might be easier rather than using a custom /synapse path as the .well-known path might be already setup.

The validation code is verified by the backend asynchronously, a worker will try and verify all the clients that are still pending. We might want to implement multiple retries before automatically rejecting a verification submission because of a failed domain validation (e.g. code mismatch).

If the sector_identifier_uri is provided during the client registration the list of redirect uris might potentially contain multiple domains. In order for a client to be validated all of the URIs needs to pass the domain validation.

Verification Status

A verification submission is associated with a status, one of: SUBMITTED, REJECTED, APPROVED. A verification approval is tied to the domain validation, the verification will include the information about the domain validation status. The verification cannot be approved if the domain is not validated.

If the verification is rejected the user should resubmit the verification with any changes indicated in the rejection reason.

Once the verification is approved the client verified property is set. Any change to the client metadata that is required for verification leads to the client verified property to be reset and a new verification should be submitted by the user. Similarly to the user verification we maintain a list of status changes with the user who changed the status. Multiple verification submissions might be present for a single client. The domain validation is tied to the verification submission.

We should allow to bypass the verification submission (e.g. for internal use or other mean of verification), in this case a submission from the user is not necessary and the client can be verified with a dedicated API call.

Proposed API

DT = Designated team for the verification review.

METHOD	URI	Notification	Request	Response	Description

METHOD	URI	Notification	Request	Response	Description
POST	/oauth2/client/{id}/verification	To the DT to review the verification request	OAuthClientVerification	OAuthClientVerification	Allows the user to submit a new verification for the client. Only the creator of the client can make this call.
GET	/oauth2/client/{id}/verification			OAuthClientVerification	Gets the current verification for the client if any. Only the creator of the client can make this call.
GET	/oauth2/client/{id}/verification/validationCode			OAuthClientValidationCode	Allows the user to retrieve the validation code used for domain validation. The code is generated once when the verification is submitted.
POST	POST /oauth2/client/{id}/verification/status	To the verification creator	OAuthClientVerificationStatus	OAuthClientVerificationStatus	Allows the DT to approve/reject the verification. The possible state transitions are: SUBMITTED → APPROVED SUBMITTED → REJECTED
PUT	/admin/oauth2/client/{id}/verified?status=<boolean>				Allows the DT to bypass the verification and set the client as verified or not independently from the verification.
GET	/oauth2/client/verification			OAuthClientVerificationList	Allows the DT to retrieve a paginated list of verifications (sorted by creation date desc), optional parameters: status: Filter by a specific status, defaults to SUBMITTED createdBy: Filter by the a specific creator clientId: Filter by the given client id. nextPageToken: Token received in the previous page, default null

API Models

OAuthClientVerificationStatus

Field	Type	Description

Field	Type	Description
status	string	the status of the verification, one of SUBMITTED, APPROVED, REJECTED
reason	string	The reason for the rejection
createdOn	date	The creation date
createdBy	long	The id of the user that pushed this status. Only present when the caller is the DT

OAuthClientDomainValidationStatus

Field	Type	Description

Field	Type	Description
status	string	the status of the domain validation, one of PENDING, VALIDATED, FAILED
reason	string	The reason for the validation failure
createdOn	date	The creation date
modifiedOn	date	The last modification date

OAuthClientVerification

Field	Type	Description

Field	Type	Description
clientId	long	The id of the client this verification refers to
clientDescription	string	Required description for the submission
createdOn	date	The creation date
createdBy	long	The id of the user that created the verification
verificationStatus	OAuthClientVerificationStatus	The current verification status
domainValidationStatus	OAuthClientDomainValidationStatus	The status of the domain validation

OAuthClientVerificationList

Field	Type	Description

Field	Type	Description
results	List<OAuthClientVerification>	The page of results
nextPageToken	string	The token used to retrieve the next page if any.

OAuthClientValidationCode

Field	Type	Description

Field	Type	Description
cliendId	long	The id of the client
code	string	The code for the domain validation for any of the registred redirect uris

User Interaction

There are three main actors involved in the verification

A. The user that registered a client and needs to submit the verification.

B. The user that reviews the submission

C. The end user using the client

For A (tracked in

SWC-4957 - Getting issue details... STATUS ) the user should be able to:

See that the client is not verified with information about the consequences
Submit a verification if not verified already
Get the domain validation code, with instructions on how to use it. Potentially a file download with the validation code to be placed as is on a web server.
Check the status of the verification, including the domain validation status.

For B the user should be able to:

See the list of submitted verifications
Review the information of the client, including the description submitted with the verification, the client, tos and policy uris and a link to the user profile.
Check the domain validation status for the client, if the domain is not validated yet approval is not allowed.
Approve or reject the verification, allowing to input a reason

For C, the user should be informed if the client is not verified and the consent should be discouraged.