This document attempts to cover development and release practices for the synapser projects and synapserutils projects. As of synapser 1.0.0+, PythonEmbedInR is no longer used.

...

Github repositories

The code base for Synapse R client (wrap Python client) can be found at https://github.com/Sage-Bionetworks/synapser and the code base for Synapserutils package can be found at https://github.com/Sage-Bionetworks/synapserutils .

Branches

• master: tracking latest release

• develop: tracking latest development work

• release candidate branches: for each release, a new release candidate branch will be created with name set to the version to release (v1.0-rc).

...

Development and Release

Releasing a new version of synapser Synapser is a process that involves client engineers, product manager, validators, and end users.

Step 1: Identifying the next release

This step starts with client engineers and product manager going over R JIRA tickets and determining their priority, tagging them with the new release version, and assigning them to client engineers.

Step 2: Development

Develop work will be checked into develop branch. 

1. Fork https://github.com/Sage-Bionetworks/synapser

2. Checkout a new branch from develop branch to work on a feature / Jira

3. GitHub Actions are configured in the repositories. Every push will build and test your changes in your forked repo (see the GitHub Actions tab of the repo). On your fork of the synapser repo, add each of the following secrets, the values can be obtained from the "Python Client dev stack openssl keys" entry of LastPass. These allow a test configuration to be installed during builds which enables vignette integration tests to run against test services. 

   1. encrypted_d17283647768_iv

   2. encrypted_d17283647768_key

4. Once your change is ready for review, create a Pull Request with 

   1. base fork: https://github.com/Sage-Bionetworks/synapser

   2. base: develop

   3. head fork: <your Github account>/synapser

   4. compare: <your feature branch>

5. Code view and reviewer merges your change to the develop branch.

References:

• Commit Best Practices: http://r-pkgs.had.co.nz/git.html#commit-best-practices

• Git Branch: http://r-pkgs.had.co.nz/git.html#git-branch

• Style: tidyverse style guidelines
  

  • Run

    Code Block
    styler::style_file()

Configuration

Add this file to your home directory, uncommenting headers and fields as necessary. Heres an example:https://github.com/Sage-Bionetworks/synapsePythonClient/blob/develop/synapseclient/.synapseConfig

Changing The Version of the Underlying Python Client in Synapser

The synapser package 'wraps' a specific version of the Synapse Python Client, specified in R/zzz.R, and tools/installPythonClient.R for example:

...

PYTHON_CLIENT_VERSION <- '2.7.0'

Upgrading the Python client dependency is a matter of changing this line.  synapser will look for the specified version in PyPI. Like all R packages, synapser has embedded reference doc's.  These doc's are in a LaTeX-like format, have the file suffix ".Rd" and go in a directory called man/.  This is a "source" folder which the package build process uses as input. 

...

Changing The Version of the Underlying R Client in synapserutils

The synapserutils built on top of the synapser package and the synapser version is specified in the DESCRIPTION file in the Remotes section and would be attached to the package when installing it, for example:

Sage-Bionetworks/synapser@2.1.1

Moreover, functions in synapserutils should mirror what are in the synapseutils. Changes need to be made to synapseUtilsWrapper.py and reference documents.

Step 3: Deploy Staging

Once all JIRA tickets for the new release version are RESOLVED, we are ready to deploy staging version for validation process.

1. Create a new release candidate branch from develop with branch name as the version to release (for example v1.0-rc)

2. Update the docs in the release candidate branch:

   1. ensure you have the R pkgdown package installed (on a Mac you may need to brew install harfbuzz, fribidi, and pandoc if you haven't already)

   2. Update the changelog contained in NEWS.md

   3. Update the version in the DESCRIPTION file, as the version is reflected in the generated documentation.

   4. from the repo directory run the following command:

      Code Block
      R -e "pkgdown::build_site()"

   5. Review the changes by inspecting docs/index.html file.

   6. Commit the changes to the docs directory . When this release branch is validated and merged to master it will automatically publish to https://r-docs.synapse.org via GitHub Pages.and push to the release candicate branch

3. Create a new staging release

   1. Go to the releases of the appropriate repo https://github.com/Sage-Bionetworks/synapser/releases

   2. Click the "Draft a new release" button and fill the following values:
      
      Tag version: X.Y-rc where X.Y where X.Y is the release version (e.g. 0.11-rc)
      Target: the previously created vX.Y-rc branch
      Release title: Same as tag version (X.Y-rc)
      
      Important: Check the "This is Set as a pre-release" checkbox. This will cause the release to be deployed to a staging ran.
      

   3. Use the GitHub release “Generate release notes” feature to create release notes. Copy related content from the NEWS.md to the top of the release notes.

   4. Hit the publish “Publish release“ button, this will trigger a GitHub Action that will test and deploy the staging release to http://staging-ran.synapse.org/

4. Notify validators about the available version. The validation version will be in format <version-to-build>.<build-number> (For example: 1.0.87, for build number 87).

Step 4: Validation

Validators can download and/or install the new pre-release version via install.packages() (recommended) or using devtools (not-recommended). After installing the pre-release version, a validator can validate the JIRA tickets that they were assigned and change their status to

Option 1: Installing from Staging RAN

Staging RAN Repo: http://staging-ran.synapse.org. For example, synapser

staging-ran

install.packages("synapser", repos=c("http://staging-ran.synapse.org"))

Option 2: Using devtools

devtools-staging

# replace version with the new release version 
devtools::install_github("Sage-Bionetworks/synapser@v1.0-rc") 

Step 5: Patch Staging

When a critical bug is found during validation, and it needed to be addressed before releasing, a patch need to go into "staging" without changing the <major>.<minor> release version.

1. A client engineer fix a bug, run private build, and create Pull Request to the release branch (v1.0-rc)

2. After the Pull Request is merged, start the staging build to update the artifacts. Please see Step 3 to recreate a staging build.

3. Notify the validator to re-validate the ticket with the new artifact.

Step 6: Release

After all JIRA issue are CLOSED, the new version is ready to be released. Creating a production deployment is similar to the staging deployment and is accomplished by publishing a GitHub release:

1. Go to the releases of the appropriate repo https://github.com/Sage-Bionetworks/synapser/releases

2. Click the "Draft a new release" button and fill the following values:
   
   Tag version: X.Y where X.Y is the release version (e.g. 0.11)
   Target: the previously created vX.Y-rc branch
   Release title: Same as tag version (X.Y)
   
   Do NOT check the "This is a pre-release" checkbox.
   
   

3. Hit the publish button, this will trigger a GitHub Action that will test and deploy the production release to http://ran.synapse.org/

4. Merge the develop branch to the master branch. When this release branch is validated and merged to master it will automatically publish to https://r-docs.synapse.org  via GitHub Pages.

git checkout master
git merge vX.Y.Z-rc
git push upstream master

5. Merge the master branch to develop:

git checkout develop
git merge master
git push upstream develop

Step 7: Users install the released version

Option 1: Installing from RAN

RAN Repo: http://ran.synapse.org

ran

install.packages("synapser", repos=c("http://ran.synapse.org"))

Option 2: Using devtools

devtools-prod

# replace version with the new released version 
remotes::install_github("Sage-Bionetworks/synapser@v1.0") 

...

Publishing Documentation 

Users come to the synapser package through:

...

• While building artifacts and pushing them to staging ran, we will generate docs and merge back to release candidate branch.

• When we release the artifact to prod ran, we merge release candidate branch to master. This will update the docs site.

CI/CD configuration

As discussed above, the CI/CD for synapser is driven through GitHub Actions. The build is contained in the contained build.yml workflows of the respective repositories (synapser).

...