1 Gather Requirements
2 Code Changes
3 Pull Requests
- 3.1 Code Reviews
- 3.2 Build and Test Locally
- 3.3 Integration Tests
- 3.4 Emergency Code Changes
4 Release Process

Gather Requirements

All code changes are documented in Jira under the BRIDGE project. This includes bug fixes, feature changes, work requested by people outside of the Bridge Server Team, and work deem necessary by members of Bridge Server Team.

Code Changes

No code changes are made directly to Bridge github repos. Instead, developers must fork the repo, make the code changes in their fork, and then merge the changes back in using pull requests (see the Pull Requests section below).

This applies to all repositories managed by the Bridge Server Team, including:

Pull Requests

All code changes are submitted through pull requests. Before the pull request is merged, the developer must get a code review from another Bridge Server Developer, build and test the changes locally, and run integration tests.

RECOMMENDATION: The Pull Request description should include a link to the Jira, to make it easier to track changes. Likewise, the Jira should contain a comment with a link to the implementing pull requests.

Code Reviews

Code reviewer reviews the pull request and either requests changes or approves the pull request. Subsequent changes are re-reviewed and either more requests are changed or the subsequent changes are reviewed.

Build and Test Locally

Changes are built on the developer’s development machine. For Maven-based packages, this is usually accomplished by running mvn verify. This includes a full compilation, unit tests, and usually Jacoco code coverage and FindBugs. This is the same command that is run in our Travis build.

Jacoco will fail the build if line coverage and branch coverage drop below the pre-configured thresholds. Developers should ensure that their code is adequately covered by unit tests. However, some code, such as Spring Config and Exceptions, can be excluded from code coverage in the pom.xml.

FindBugs performs static code analysis and flags certain code patterns that need to be fixed. Developers should either fix these FindBugs issues or add an entry to findbugs-exclude.xml with a comment describing why the pattern is being excluded.

Integration Tests

Developers run integration tests locally and ensure all integration tests pass.

RECOMMENDATION: Laptops are slow and integration tests take significantly longer on laptops. We should invest in something similar to Synapse’s Stack-Builder and figure out how to run integration changes for developers' local changes on the cloud.

Emergency Code Changes

Sometimes, it becomes necessary for emergency code changes, such as Travis build failures or integration test failures. If the build failure is in Dev or Staging, and the fix is relatively minor, the developer can submit and merge an emergency pull request in order to unblock the build pipeline. The developer should notify the other Bridge developers and link to the pull request for a “retroactive code review”.

If the fix is not minor (but the build failure doesn’t affect Prod), changes should be rolled back in GitHub. This can usually be accomplished by going to the original pull request and clicking the Revert button.

For emergency changes in Prod, see the Release Process section below.

Release Process

The developer releasing to production notifies the other Bridge Server Team members that they plan to release to production.

If integration tests haven’t been run in Dev yet, the developer will first run the integration tests in Dev.

The developer then merges from Dev to Staging by merging from the develop git branch to the uat (staging) git branch, generally by using the following steps in the Git command line.

git checkout develop
git pull
git checkout uat
git merge --ff-only develop
git push

This kicks off the Travis build, which then deploys to AWS (where applicable). The developer waits for both the Travis build and the AWS deployment. Once those are complete, the developer kicks off integration tests.

When the build, deploy, and integration tests complete, the developer pushes from staging to production using the same steps.

Proposed Lightweight Process

The developer sends an email to Bridge Server Team (either email Bridge Server devs directly, to ensure the email doesn’t get caught by filters, or we should create a mailing list specific to Bridge Server Deployment), and cc’s bridge@ mailing list.

This email should include the Jira link and title for all changes in this deployment.

If the deployment also includes integration test changes that are not strictly additive changes, this should be called out in the deployment notification email. This allows developers and stakeholders to evaluate any potential breaking changes.

The email should also include the planned date and time that the deployment will happen. The planned deployment time should not be on a Friday (or on a Thursday before a Friday holiday), since there will be no devs available over the weekend if a problem happens with the deployment.

This email should be sent out at least 24 hours before the deployment time, and the deployment should be approved by another member of the Bridge Server Team.

Proposed Heavyweight Process

Deployments to Prod are scheduled, interval TBD. We schedule a release meeting with Bridge Server Team and stakeholders to review and approve changes. After the meeting, Bridge dev sends out an email, similar to the one described in the above Lightweight Process.

Emergency Changes to Prod

If a deployment to Prod needs to be rolled back, the developer first determines the last known stable version. This can usually be accomplished by looking at Travis and looking at the previous Travis Prod build. The developer can then look at Elastic Beanstalk and find the corresponding application version, and then manually deploy that version to the Elastic Beanstalk environment.

OPEN QUESTION: What are the processes for approving emergency changes in Prod?