OAuth Client Verification

With the implementation of https://sagebionetworks.jira.com/browse/PLFM-4585 synapse can now act as an OAuth 2.0 Provider. In order for application clients (See roles in https://tools.ietf.org/html/rfc6749#section-1.1) to use Synapse as a resource server our implementation of the authorization server requires the client application to be registered before issuing authorization codes and access tokens.

Once a client is registered it can perform scoped operations on behalf of the user, at the time of writing we support the openid scope for the Open ID Connect /userInfo service. In the future we will extend the scope (and claims) that would allow access to data that with the user consent.

One of the issues in the OAuth architecture is that while authentication and authorization is decoupled from a client application in a secure manner, establishing trust on the client application is not defined by the specification.

Since that data that synapse users can access might potentially be sensitive according to the scopes and claims, we need a way to establish trust with a client application by having a process in place to verify that the application will not misuse and/or abuse the system. There is not a standard (automated or manual) procedure for verifying a client in the industry and different companies take different approaches (or choose not to have a verification process).

Google and Facebook have both an automated and manual verification and review process, according to the type of client, scope and claims the client has access to. (See https://developers.google.com/apps-script/guides/client-verification and https://support.google.com/cloud/answer/9110914?hl=en).

Proposed Approach for Synapse

We propose to introduce a (manual) verification step for application clients that are registered in synapse which is similar to the user verification (https://sagebionetworks.jira.com/wiki/spaces/PLFM/pages/82935813/Verified+User). The manual verification will be implemented on top of an automatic validation of ownership of the domain in the redirect uri(s) submitted by the client. We allow registration and usage without verification but the web client should show a banner if a user consenting to an application that has not been verified. The banner should clearly state that the application is not verified and discourage the user to consent.

Allowing the usage of a client which is not verified can be necessary for testing (note that we could think of a sandbox/public environment for clients but for the time being we want to keep the process as simple as possible).

When a client register an application in synapse currently the following required information is supplied:

• Name of the client

• Redirect URIs + (Optional) Sector Identifier URI (See https://openid.net/specs/openid-connect-registration-1_0.html#ClientMetadata)

We do not require the response_type as we default to "code" nor the the application_type (which defaults to "web").

Additionally the following optional information can be supplied:

• client_uri: URL of the home page of the Client

• policy_uri: URL that the Relying Party Client provides to the End-User to read about the how the profile data will be used

• tos_uri: URL that the Relying Party Client provides to the End-User to read about the Relying Party's terms of service

The information above can all be useful to establish trust in a client. We propose the following pre-conditions for submitting a verification:

1. The client, policy and tos uris should be filled out

2. The client, policy and tos uris domain should match that of the redirect uris (As suggested in https://openid.net/specs/openid-connect-registration-1_0.html#Impersonation). This should be validated at registration time

3. The redirect uris scheme should be https and the host should not be localhost or loopback (127.0.0.1, ::1)

4. The client should have generated a secret

5. A natural language description of the application must be included in the submission

The verification of a client will be performed manually and approved or rejected by a member of a designated team (ACT?). The user that submitted the verification should be able to check the verification status at any time.

The designated team will be able to access the list of submissions along with the domain verification information. A submission shall not be approved if the domain verification is not complete.

In case of rejection a reason must be included explaining why. A user should be able to update the client and perform another submission after being rejected.

Domain validation

In addition to the manual approval or rejection, we should establish ownership of the domain referred to by the redirect uri (s) in the client as an additional trust layer. Note that while the specification does not enforce the scheme for redirect uris for web application types (See application_type in https://openid.net/specs/openid-connect-registration-1_0.html#ClientMetadata), it allows custom schemes for "native" applications. Since we do not support native applications and the specification does not explicitly forbid it, we can enforce the redirect uri scheme to be https. In this way we can assume that we should always be able to verify the domain ownership.

An exception applies if the host in the redirect uri is the loopback, this might be used for two reasons:

1. Testing

2. Native Applications

We currently do not support 2. but we probably should in the future to allow our own clients to authenticate using OAuth 2.0 (With PKCE https://tools.ietf.org/html/rfc7636). For a native client the the loopback might be used as the redirect uri: the reason is that the user authentication should be executed through a third party user-agent (such as the browser) and not the application itself (See https://tools.ietf.org/html/rfc8252 for best practices), since the application that launches the browser to authenticate the user needs to receive a callback the idea is to start a temporary server on the device listening on an available port and wait for the callback from the browser. It's also worth mentioning that there are other implementations that achieve the same goal without the loopback "workaround". An example is the heroku cli which releases a temporary authorization code and the cli performs a long polling operation waiting for the callback from the server itself with the verified code when the user performed the login (See https://github.com/heroku/heroku-cli-command/blob/master/src/login.ts for its the implementation). This same mechanism could be adopted for our clients. For native clients (if supported) we might decide not to validate the redirect uri.

For the time being we can enforce the scheme for a redirect uri to be https at registration time or http for the loopback (for testing). For the domain validation we can adopt a common solution that involves generating a unique validation code that needs to be placed on the target server and verified by our backend. We will not support DNS record based validation for the time being.

There exists a standard HTTP challenge used by lets encrypt (https://letsencrypt.org/docs/challenge-types) for automatically certify domain ownership when in the context of automated certificate management, this would not be practical in our case as it requires a complex configuration of the server and installation of an agent. Instead we opt for a simple validation of a code generated by the server that is served in a file and should be available at the following location:

https://<redirect_uri_domain_common_name>/synapse/<VALIDATION_CODE>.txt

The <VALIDATION_CODE> is a code generated at submission time (the user should be able to retrieve this code for the verification), the server should reply for a GET request with a 200 status and the body content should contain the <VALIDATION_CODE> as well. We do not put constraints on the content-type or any other headers, as long as the code matches the one generated by the server the domain is considered validated.

We do not need to generate a file handle in the backend, we can generate this file on the fly if needed (e.g. the user might want to download the file instead of creating it from the validation code) or let the web client generate it for the user (?).

Note that while we can use a .well-known directory, I could not find a suitable one in the official registry: https://www.iana.org/assignments/well-known-uris/well-known-uris.xhtml (Godaddy uses pki-validation, Lets Encrypt use their own ACME challenge). I'm not sure at this point if it's ok for us to still use the .well-known path. If might be easier rather than using a custom /synapse path as the .well-known path might be already setup.

The validation code is verified by the backend asynchronously, a worker will try and verify all the clients that are still pending. We might want to implement multiple retries before automatically rejecting a verification submission because of a failed domain validation (e.g. code mismatch).

Verification Status

A verification submission is associated with a status, one of: SUBMITTED, REJECTED, APPROVED. A verification approval is tied to the domain validation, the verification will include the information about the domain validation status. The verification cannot be approved if the domain is not validated.

If the verification is reject the user should resubmit the verification with any changes indicated in the rejection reason.

Once the verification is approved the client verified property is set. Any change to the client metadata that is required for verification leads to the client verified property to be reset and a new verification should be submitted by the user. Similarly to the user verification we maintain a list of status changes with the user who changed the status. Multiple verification submissions might be present for a single client. The domain validation is tied to the verification submission.

We should allow to bypass the verification submission (e.g. for internal use or other mean of verification), in this case a submission from the user is not necessary and the client can be verified with a dedicated API call.

API Changes

There are some API changes needed in order to support this workflow, in particular a client is not usable if a secret was not generated and we therefore required the presence of a secret for verification. Our implementation deviates somehow from the specification, in the sense that the secret is not generated at registration time and is not returned in the GET /oauth2/client response. Instead we have a dedicated POST /oauth2/client/secret/{id} (shouldn't this be /oauth2/client/{id}/secret?) endpoint to (re)generate a secret.

For the web client to show the requirements for verification it needs to be able to gather the current status of the secret, we propose to include a boolean property secret_generated in the response of the GET /oauth/client/{id} which would also allow the web client to inform the user that a secret needs to be generated.

Additionally we might want to add a new API call that returns the current secret for the user

GET /oauth2/client/secret/{id}

Proposed API

DT = Designated team for the verification review.

API Models:

OAuthClientVerificationStatus

OAuthClientDomainValidationStatus

OAuthClientVerification

OAuthClientVerificationList

References:
https://tools.ietf.org/html/rfc6749

https://tools.ietf.org/html/rfc8252

https://tools.ietf.org/html/rfc7636

https://www.oauth.com/oauth2-servers/client-registration

https://letsencrypt.org/docs/challenge-types
https://developers.facebook.com/docs/apps/review/
https://api.slack.com/security-review
https://support.google.com/cloud/answer/9110914?hl=en
https://www.iana.org/assignments/well-known-uris/well-known-uris.xhtml
https://openid.net/specs/openid-connect-core-1_0.html
https://openid.net/specs/openid-connect-registration-1_0.html

https://github.com/heroku/heroku-cli-command/blob/master/src/login.ts
https://developer.github.com/apps/installing-github-apps/

OAuth One Pager (Bruce)

https://sagebionetworks.jira.com/wiki/spaces/PLFM/pages/643530753/Implementing+OAuth2+into+Synapse

https://sagebionetworks.jira.com/wiki/spaces/PLFM/pages/153518104/Synapse+as+OAuth+2.0+Provider

PLFM-4585 - Getting issue details... STATUS

PLFM-5170 - Getting issue details... STATUS