Developer Bootstrap

On This page

On Related Pages

Getting Started

Install the following dependencies:

Java Development Kit - Version 11: we use Amazon Corretto in production.
Git: Set up your local git environment according to Sage’s GitHub Security guidance
Maven 3
Eclipse (or other preferred Java IDE, like IntelliJ IDEA, VS Code etc)
Note: Spring Tools provides an Eclipse build that bundles a lot of useful plugins
MySQL server (version 8.0.33 or check which version is currently used for production under the "EngineVersion" key, newest releases may have issues): See https://downloads.mysql.com/archives/community/ and select the same version as used in production. As an alternative the official docker image https://hub.docker.com/_/mysql can be used to start an instance. Mac user should Launching MySQL with Docker to start an instance through docker compose.

There are also GUIs available for MySQL that can be useful for management, among which MySQL Workbench and HeidiSQL.

Note: explicit_defaults_for_timestamp needs to be set to ON.
Instruction on setting explicit_defaults_for_timestamp:
1. Open localhost connection. On the left panel, Under INSTANCE, choose "Options File"
2. Under the "General" tab, scroll down to "SQL", make sure that explicit_defaults_for_timestamp is checked
3. Click Apply
4. Restart MySQL server

See Developer Tools for links to the installers.

Setting up dev environment on Sage laptop – Windows x64

Install Java JDK (64-bit, 11) – set up JAVA_HOME variable (JDK's install directory, not the /bin folder) and add JAVA_HOME/bin to PATH
1. It is possible to have multiple JDK versions installed, just make sure that the IDE and maven are using 11 when building the project
Install Eclipse (64-bit)
Install Maven (most recent stable release); and add to PATH
Install M2Eclipse plugin from http://download.eclipse.org/technology/m2e/releases
1. Sometimes, fresh copies of Eclipse will not have all the dependencies installed. Particularly, M2E may complain about SLF4J 1.6.2.
2. Either reinstall Eclipse or search the marketplace for the module.
Point Eclipse to the JDK
1. Open Eclipse and navigate to Window->Preferences
2. Open the submenu Java->Installed JREs
3. Click Add..->Standard VM->Directory, and input the JDK directory
4. Then finish and check the JDK (uncheck the JRE)
5. Make sure the path to the JDK is ahead of "%SystemRoot%\system32", since Eclipse will otherwise use some other Java VM when starting up Eclipse.
Turn off indexing (at least on the project directory).

Setting up dev environment on macOS (June 2018)

Make sure you have the Xcode Command Line Tools, which you can install with

xcode-select --install

I recommend using Homebrew to install Maven and Git. You can also install Eclipse and MySQLWorkbench with Homebrew if desired.

brew install maven git
brew tap homebrew/cask
brew install eclipse-ide mysqlworkbench

Using Homebrew to install the JDK or MySQL should be done with caution, because can be difficult to configure Homebrew to use older versions of this software (which Synapse may require as new versions of the JDK and MySQL are released). You can access binaries for the older versions of Java and MySQL with an Oracle account (Google should lead you to these downloads easily).

Your ~/.bash_profile (Note: will be ~/.zshenv if you are on MacOS Catalina or later) should look something like this (where version numbers may differ slightly, and locations may differ entirely if you installed your software differently; check your machine to make sure these folders exist):

export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk11.jdk/Contents/Home/
export M2_HOME=/usr/local/Cellar/maven/3.5.3/libexec
export M2=$M2_HOME/bin

# The following may not be necessary: 
export PATH=${PATH}:/usr/local/mysql/bin 	# To fix MySQL installation after removing a previous installation via Homebrew

Using the steps above, you can configure your MySQL server with MySQLWorkbench. You may need to create a file at /etc/my.cnf to edit the configuration and set explicit_defaults_for_timestamp.

Though it might be dated, consider checking out Macintosh Bootstrap Tips for additional reference.

Synapse Platform Codebase

Get the Maven Build working

Fork the Sage-Bionetworks Synapse-Repository-Services repository into your own GitHub account: https://help.github.com/articles/fork-a-repo
Make sure you do not have any spaces in the path of your development directory, the parent directory for PLFM and SWC. For example: C:\cygwin64\home\Njand\Sage Bionetworks\Synapse-Repository-Services needs to become: C:\cygwin64\home\Njand\SageBionetworks\Synapse-Repository-Services
Check out everything
git clone https://github.com/[YOUR GITHUB NAME]/Synapse-Repository-Services.git
Setup the Sage Bionetwork Synapse-Repository-Services repository to be the "upstream" remote for your clone
# change into local clone directory cd Synapse-Repository-Services # set Sage-Bionetwork's Synapse-Repository-Services as remote called "upstream" git remote add upstream https://github.com/Sage-Bionetworks/Synapse-Repository-Services
Download your forked repo's develop branch, then pull in changes from the central Sage-Bionetwork's Synapse-Repository-Services develop branch
# bring origin's develop branch down locally git checkout -b develop remotes/origin/develop # fetch and merge changes from the Sage-Bionetworks repo into your local clone git fetch upstream git merge upstream/develop
Note: this is NOT how you should update your local repo in your regular workflow. For that see the Git Workflow page.
Stack Name
- Your stack name will be dev
- Your stack instance will be an alpha' string, seven characters or less, e.g., your unix username
Setup MySQL
1. Mac user should Launching MySQL with Docker to start an instance through docker compose.
2. Start a command-line mysql session with "sudo mysql"
3. Create a MySQL user with your named dev[YourUsername] with a password of platform.
  
  create user 'dev[YourUsername]'@'%' identified BY 'platform'; grant all on *.* to 'dev[YourUsername]'@'%' with grant option;
4. Create a MySQL schema named dev[YourUsername] and grant permissions to your user.
  create database `dev[YourUsername]`; # This might not be needed anymore grant all on `dev[YourUsername]`.* to 'dev[YourUsername]'@'%';
5. Create a schema for the tables feature named dev[YourUsername]tables
  create database `dev[YourUsername]tables`; # This might not be needed anymore grant all on `dev[YourUsername]tables`.* to 'dev[YourUsername]'@'%';
6. This allows legacy MySQL stored functions to work. It may not be necessary in the future if all MySQL functions are updated to MySQL 8 standards.
  set global log_bin_trust_function_creators=1;
  Note: Use set PERSIST in order to keep the configuration parameter between MySQL restarts (e.g. if the MySQL service is not setup to start automatically). Alternatively, use the Options Files to set this checkbox (under the Logging tab, under Binlog Options).
7. This allows queries with ORDER BY/GROUP BY clauses where the SELECT list does not match the ORDER BY/GROUP BY clause.
  SET GLOBAL sql_mode=(SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY',''));
Setup AWS Resources
The build expects certain AWS resources to be instantiated. Use the GitHub stack builder workflow to automatically instantiate the stack for you. (If you wish later to delete any of the resources you created see the "Deleting CloudFormation Stacks" section.)
1. Visit https://github.com/Sage-Bionetworks/synapse-ops-dev/actions/workflows/build_synapse_dev_stack.yml and click "Run workflow" at the right hand side. Fill in the instance (="YourUsername", above), and optionally override the GitHub source for the stack builder with a different owner/fork and branch. (This is only necessary when you are modifying the stack builder itself, e.g., to add new AWS resources for a new feature.)
2. Click "Run workflow" to start the build. Proceed to the next steps while waiting for this build to complete.
Create your maven settings file ~/.m2/settings.xml
1. The settings file tells maven where to find your property file and what encryption key should be used to decrypt passwords.
2. Use this settings.xml as your template
  
  settings.xml
  <settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"> <localRepository/> <interactiveMode/> <usePluginRegistry/> <offline/> <pluginGroups/> <servers/> <mirrors/> <proxies/> <profiles> <profile> <id>dev-environment</id> <activation> <activeByDefault>true</activeByDefault> </activation> <properties> <org.sagebionetworks.stackEncryptionKey>20c832f5c262b9d228c721c190567ae0</org.sagebionetworks.stackEncryptionKey> <org.sagebionetworks.developer>YourUsername</org.sagebionetworks.developer> <org.sagebionetworks.stack.instance>YourUsername</org.sagebionetworks.stack.instance> <org.sagebionetworks.google.cloud.enabled>true</org.sagebionetworks.google.cloud.enabled>  <org.sagebionetworks.google.cloud.client.id>googleCloudDevAccountId</org.sagebionetworks.google.cloud.client.id> <org.sagebionetworks.google.cloud.client.email>googleCloudDevAccountEmail</org.sagebionetworks.google.cloud.client.email> <org.sagebionetworks.google.cloud.key>googleCloudDevAccountPrivateKey</org.sagebionetworks.google.cloud.key> <org.sagebionetworks.google.cloud.key.id>googleCloudDevAccountPrivateKeyId</org.sagebionetworks.google.cloud.key.id> <org.sagebionetworks.stack>dev</org.sagebionetworks.stack> </properties> </profile> </profiles> <activeProfiles/> </settings>
3. Everywhere you see "YourUsername", change it to your stack instance name (i.e. the username you used to set up MySQL and the AWS shared-resources)
4. If using Google Cloud (recommended if you are working on file upload features), get the Google Cloud developer credentials from the shared LastPass entry titled 'synapse-google-storage-dev-key'. Paste the corresponding fields in the attached JSON file into the org.sagebionetworks.google.cloud elements. Note the private key should be given on one line, with new lines separated with '\n' and NOT '\\n'.
  1. If you don't need to use Google Cloud features, then set org.sagebionetworks.google.cloud.enabled to be false. The other Google Cloud parameters can be omitted.
5. Once your settings file is setup, you can Build everything via `mvn` which will run all the tests and then install any resulting jars and wars in your local maven repository.
  cd Synapse-Repository-Services mvn install
  1. Note that you might get an error that says Illegal access: this web application instance has been stopped already. This is normal. Scroll up to see whether the build succeeded or failed. (You may have to scroll through several pages of this error.)
  2. If tests fail, then you should run mvn clean install after debugging your failed build.
  3. Many tests assume you have a clean database. Before running tests, be sure to drop all tables in your database. Synapse will automatically recreate them as part of the tests.
  4. When you pull updates from upstream, be sure to run mvn clean install -DskipTests from the root project, which will rebuild everything. If necessary, run mvn test from the root to ensure all tests pass.
  5. Tests can also fail because sometimes IntelliJ does not configure itself appropriately, if this case, please following these steps.
    1. Go to the Synapse-Repository-Services repository and execute ls -a to find '.idea' folder, which is the configuration folder of IntelliJ IDEA.
    2. Execute 'rm -r .idea' to delete configuration folder and re-configure IntelliJ IDEA (please follow 'Configuring IntelliJ IDEA' section again)
    3. Either terminal in IntelliJ IDEA or own terminal, execute 'mvn clean install -Dmaven.test.skip=true' to make sure all files are configured correctly. (skipTests are deprecated in Maven Surefire Plugin Version 3.0.0-M3)
  6. If the integration tests fail, try restarting the computer. This resets the open connections that may have become saturated while building/testing.
Take a look at the javadocs, they are in target/apidocs/index.html and target/testapidocs/index.html for each package