Page Comparison

...

1. Fork the Sage-Bionetworks Synapse-Repository-Services repository into your own GitHub account: https://help.github.com/articles/fork-a-repo

2. Make sure you do not have any spaces in the path of your development directory, the parent directory for PLFM and SWC. For example: C:\cygwin64\home\Njand\Sage Bionetworks\Synapse-Repository-Services needs to become: C:\cygwin64\home\Njand\SageBionetworks\Synapse-Repository-Services

3. Check out everything

   Code Block
   git clone https://github.com/[YOUR GITHUB NAME]/Synapse-Repository-Services.git

   
   

4. Setup the Sage Bionetwork Synapse-Repository-Services repository to be the "upstream" remote for your clone

   Code Block
   language bash
   # change into local clone directory cd Synapse-Repository-Services   # set Sage-Bionetwork's Synapse-Repository-Services as remote called "upstream" git remote add upstream https://github.com/Sage-Bionetworks/Synapse-Repository-Services

   
   

5. Download your forked repo's develop branch, then pull in changes from the central Sage-Bionetwork's Synapse-Repository-Services develop branch

   Code Block
   language bash
   # bring origin's develop branch down locally git checkout -b develop remotes/origin/develop   # fetch and merge changes from the Sage-Bionetworks repo into your local clone git fetch upstream git merge upstream/develop

   Note: this is NOT how you should update your local repo in your regular workflow.  For that see the Git Workflow page.

6. Stack Name
   
   • Your stack name will be dev
   • Your stack instance will be whatever you like, we recommend your unix username
7. Setup MySQL 
   1. Mac user should  Launching MySQL with Docker to start an instance through docker compose.
   2. Start a command-line mysql session with "sudo mysql"

   3. Create a MySQL user with your named dev[YourUsername] with a password of platform.


      Code Block
      create user 'dev[YourUsername]'@'%' identified BY 'platform'; grant all on *.* to 'dev[YourUsername]'@'%' with grant option;

      
      

   4. Create a MySQL schema named dev[YourUsername] and grant permissions to your user.

      Code Block
      create database `dev[YourUsername]`; # This might not be needed anymore grant all on `dev[YourUsername]`.* to 'dev[YourUsername]'@'%';

      
      

   5. Create a schema for the tables feature named dev[YourUsername]tables

      Code Block
      create database `dev[YourUsername]tables`; # This might not be needed anymore grant all on `dev[YourUsername]tables`.* to 'dev[YourUsername]'@'%';

      
      

   6. This allows legacy MySQL stored functions to work. It may not be necessary in the future if all MySQL functions are updated to MySQL 8 standards.

      Code Block
      set global log_bin_trust_function_creators=1;

      Note: Use set PERSIST in order to keep the configuration parameter between MySQL restarts (e.g. if the MySQL service is not setup to start automatically). Alternatively, use the Options Files to set this checkbox (under the Logging tab, under Binlog Options).

   7. This allows queries with ORDER BY/GROUP BY clauses where the SELECT list does not match the ORDER BY/GROUP BY clause.

      Code Block
      language sql
      SET GLOBAL sql_mode=(SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY',''));

      
      

8. Setup AWS Resources
   The build expects certain AWS resources to be instantiated. Use a Jenkins stack builder job to automatically instantiate the build for you. Note that you will need a IAM account before the build can provision resources on AWS. To create an IAM account, fork and edit this template. (See /wiki/spaces/IT/pages/392658945).
   
   If you wish to delete any of the resources you created see the "141813264" section.
   
   1. Log in to https://build-system-synapse.dev.sagebase.org and create a new job (click New Item in dashboard).
      You will need to be on the VPN to access the Jenkins Server, directions to set up VPN can be found here, as well as an account on the Jenkins server (contact IT).
   2. name the project stackbuilder-dev-[YourUsername] and copy settings from stackbuilder-repo-develop-build. Then click OK
      
   3. Click on the "Build" tab and modify the line
      " -Dorg.sagebionetworks.instance=YourUsername"\ 
      Then click Save.
      
       
      Note: This may be represented as a String Parameter: " -Dorg.sagebionetworks.instance=${INSTANCE}"\, in that case, change the value of the INSTANCE parameter to YourUsername


   4. On the next page, click "Build with Parameters" in the left navigation then "Build". Proceed to the next steps while waiting for this build to complete.
      
      
      
9. Create your maven settings file ~/.m2/settings.xml
   
   1. The settings file tells maven where to find your property file and what encryption key should be used to decrypt passwords.

   2. Use this settings.xml as your template
       

      Code Block
      language xml title settings.xml
      <settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"> <localRepository/> <interactiveMode/> <usePluginRegistry/> <offline/> <pluginGroups/> <servers/> <mirrors/> <proxies/> <profiles> <profile> <id>dev-environment</id> <activation> <activeByDefault>true</activeByDefault> </activation> <properties> <org.sagebionetworks.stackEncryptionKey>20c832f5c262b9d228c721c190567ae0</org.sagebionetworks.stackEncryptionKey> <org.sagebionetworks.developer>YourUsername</org.sagebionetworks.developer> <org.sagebionetworks.stack.instance>YourUsername</org.sagebionetworks.stack.instance> <org.sagebionetworks.stack.iam.id>yourAwsDevIamId</org.sagebionetworks.stack.iam.id> <org.sagebionetworks.stack.iam.key>yourAwsDevIamKey</org.sagebionetworks.stack.iam.key> <org.sagebionetworks.google.cloud.enabled>true</org.sagebionetworks.google.cloud.enabled> <!-- If using not using Google Cloud, set the above setting to false. The following org.sagebionetworks.google.cloud.* settings may be omitted --> <org.sagebionetworks.google.cloud.client.id>googleCloudDevAccountId</org.sagebionetworks.google.cloud.client.id> <org.sagebionetworks.google.cloud.client.email>googleCloudDevAccountEmail</org.sagebionetworks.google.cloud.client.email> <org.sagebionetworks.google.cloud.key>googleCloudDevAccountPrivateKey</org.sagebionetworks.google.cloud.key> <org.sagebionetworks.google.cloud.key.id>googleCloudDevAccountPrivateKeyId</org.sagebionetworks.google.cloud.key.id> <org.sagebionetworks.stack>dev</org.sagebionetworks.stack> </properties> </profile> </profiles> <activeProfiles/> </settings>

      
      

   3. Everywhere you see "YourUsername", change it to your stack instance name (i.e. the username you used to set up MySQL and the AWS shared-resources)
   4. Get AWS IAM ID and Key from http://aws.amazon.com/iam/ (see whoever manages those accounts) and get admin privileges for AWS IAM
      1. Click on "My security credentials"
      2. On the AWS IAM credentials tab and under "Access keys for CLI, SDK, & API access", click on "Create access key" 
      3. The ID will look something like AKIAIOSFODNN7EXAMPLE, and the Key will look something like wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
   5. If using Google Cloud (recommended if you are working on file upload features), get the Google Cloud developer credentials from the shared LastPass entry titled 'synapse-google-storage-dev-key'. Paste the corresponding fields in the attached JSON file into the org.sagebionetworks.google.cloud elements. Note the private key should be given on one line, with new lines separated with '\n' and NOT '\\n'.
      1. If you don't need to use Google Cloud features, then set org.sagebionetworks.google.cloud.enabled to be false. The other Google Cloud parameters can be omitted.

   6. Once your settings file is setup, you can Build everything via `mvn` which will run all the tests and then install any resulting jars and wars in your local maven repository.

      Code Block
      cd Synapse-Repository-Services mvn install

      
      

      1. Note that you might get an error that says Illegal access: this web application instance has been stopped already. This is normal. Scroll up to see whether the build succeeded or failed. (You may have to scroll through several pages of this error.)
      2. If tests fail, then you should run mvn clean install after debugging your failed build.
      3. Many tests assume you have a clean database. Before running tests, be sure to drop all tables in your database. Synapse will automatically recreate them as part of the tests.
      4. When you pull updates from upstream, be sure to run mvn clean install -DskipTests from the root project, which will rebuild everything. If necessary, run mvn test from the root to ensure all tests pass.
      5. Tests can also fail because sometimes IntelliJ does not configure itself appropriately, if this case, please following these steps.
         1. Go to the Synapse-Repository-Services repository and execute ls -a to find '.idea' folder, which is the configuration folder of IntelliJ IDEA.
         2. Execute 'rm -r .idea' to delete configuration folder and re-configure IntelliJ IDEA (please follow 'Configuring IntelliJ IDEA' section again)
         3. Either terminal in IntelliJ IDEA  or own terminal, execute 'mvn clean install -Dmaven.test.skip=true' to make sure all files are configured correctly.  (skipTests are deprecated in Maven Surefire Plugin Version 3.0.0-M3)

      6. If the integration tests fail, try restarting the computer.  This resets the open connections that may have become saturated while building/testing.  

10. Take a look at the javadocs, they are in target/apidocs/index.html and target/testapidocs/index.html for each package

...

export user=p<user>
export org_sagebionetworks_stack_iam_id=
export org_sagebionetworks_stack_iam_key=
export org_sagebionetworks_stackEncryptionKey=
export rds_password=platform
export github_token=<token-created-from-github>

/var/lib/jenkins/workspace/${JOB_NAME}/jenkins_build.sh

NOTE: You will need to be a member of the Sage GitHub Organization (https://github.com/orgs/Sage-Bionetworks/teams/synapse-developers) in order to for your token to work. Reach out to IT if you are not yet a member.