Page Comparison

UI Development

Set up the Maven Build

Fork the Sage-Bionetworks SynapseWebClient repository into your own GitHub account: https://help.github.com/articles/fork-a-repo
Clone the SynapseWebClient project down to your computer: git clone https://github.com/[YOUR GITHUB NAME]/SynapseWebClient.git
Follow the instructions on the Platform Bootstrap.
Change the synapse.version tag to develop-SNAPSHOT.
1. The synapse.version tag in root pom.xml has the form 2012-08-06-3e90a85-44, it has the date (2012-08-06) and the first 7 chars of the commit hash, 3e90a85.
Now, from an up-to-date clone of Synapse-Repository-Services on your local box:
1. mvn clean install -Dorg.sagebionetworks.database.drop.schema=true
2. cd integration-test
3. mvn cargo:run
4. Verify the services are running correctly by visiting http://localhost:8080/services-repository-develop-SNAPSHOT/repo/v1/version you should see something like this:
  Code Block
  {"version":"develop-SNAPSHOT"}
Now, in your up-to-date clone of SynapseWebClient:
1. mvn gwt:run
2. A GWT Development Mode log window should pop-up.
Now try to open the web-app from the Dev Mode window. You should be prompted to install the GWT Dev Mode plugin (assuming your browser is supported).

Set up the Eclipse Build

...

Internal Sage developers will be added as a developer on the project and will be able to push directly
External contributors should fork the repository and submit GitHub Pull Requests for code inclusion

...

Set up Git: https://help.github.com/articles/set-up-git

...

File -> Import -> Maven -> Existing Maven Projects
Next, in Root Directory enter the local path to your repository clone
Next, Finish

...

Now right click on portal (project name) -> Team -> Share Project
Next, Select repository type Git
Next, Check the box "Use or create repository in parent folder of project
Next, Finish

...

Right-click on portal (project name) ->Build Path>Configure Build Path...
Remove the "Exclude" filter from the source tab on the src/main/resources and src/test/resources directories.

...

From the resulting dialog make sure the "Use Google Web Tookit" check box is selected.
Also make sure you are using GWT 2.4.0 (or 2.5.1).

...

From the tree on the left of the dialog navigate to Google->Web Application
Check the the check box: "This project has a WAR directory"
With the "Browse" button, select "src/main/webapp"
UN-CHECK the "Launch and deploy from this directory" This is very important, if you keep this checked then Maven will not be able to generate a clean WAR file. If you see "GWT needs to recompile" when you deploy your WAR then you probably have this box checked.

...

Right-Click on the project from the package explorer.
Select Google->Web Toolkit Settings...
Under the "entry point modules" you should see "Portal - org.sagebionetworks.web", if not, then add it with the add button.

...

Table of Contents

minLevel	3

Child pages (Children Display)

UI Development

Set up the Maven Build

Create a GitHub user account if you don't already have one.
Set up your local git environment according to Sage’s GitHub Security guidance
Fork the Sage-Bionetworks SynapseWebClient repository into your own GitHub account: https://help.github.com/articles/fork-a-repo
Clone your fork of SynapseWebClient project to your local machine and enter the project root directory
Code Block
language bash
git clone https://github.com/[YOUR GITHUB NAME]/SynapseWebClient.git cd SynapseWebClient
If you didn't set up the global pre-commit hook in step #2, set up a local pre-commit hook to detect secrets (do this for all repos that you clone in the future!):
Code Block
git secrets --install git secrets --register-aws

Set up upstream with

Code Block
git remote add upstream https://github.com/Sage-Bionetworks/SynapseWebClient

Fetch and merge changes from the Sage Bionetworks repo, which was named upstream:
Code Block
git fetch upstream

Configure or update your Maven configuration settings.xml

The file needs to placed in $HOME/.m2/settings.xml (e.g. jane/.m2/settings.xml)

This should be the content of this file:

settings.xml

Code Block

language	xml

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
                      http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <localRepository/>
  <interactiveMode/>
  <usePluginRegistry/>
  <offline/>
  <pluginGroups/>
  <servers/>
  <mirrors/>
  <proxies/>
  <profiles>
    <profile>
      <id>dev-environment</id>
      <activation>
        <activeByDefault>true</activeByDefault>
      </activation>
      <properties>
		<org.sagebionetworks.portal.endpoint>http://127.0.0.1:8888/Portal.html</org.sagebionetworks.portal.endpoint>
        <org.sagebionetworks.repositoryservice.endpoint>https://repo-prod.prod.sagebase.org/repo/v1</org.sagebionetworks.repositoryservice.endpoint>
        <!-- insert properties described below here -->
        <!-- <org.sagebionetworks.repositoryservice.endpoint>https://repo-staging.prod.sagebase.org/repo/v1</org.sagebionetworks.repositoryservice.endpoint> -->
      </properties>
    </profile>
  </profiles>
  <activeProfiles/>
</settings>

Development

You may use an IDE of your choice. Instructions for setting up IntelliJ IDEA and Eclipse can be found below.

To run the app in development mode, run yarn install then yarn dev. This will launch two processes:

The GWT Codeserver, which watches for source changes
An Apache Tomcat docker container that will host the web application

As you make changes to the source code, changes will be picked up in the app when you refresh the page.

Instead of running yarn dev, you can alternatively use a Maven plugin (e.g. mvn gwt:run) or IDE integration. Please note that depending on your IDE/plugin configuration, your environment may be subtly different than the production system (e.g. many GWT plugins deploy a Jetty server, when we use Tomcat in production), which can result in differences in behavior.

IDE Setup Instructions

This section contains instructions for setting up various IDEs to develop SynapseWebClient

Expand

title	Setup with Eclipse

Install the latest Eclipse (for Java Development)
Install the Google Plug-in for Eclipse. Do not include GWT SDK in the install (GWT SDK will be installed by the maven install). Do not install the Google App Engine Maven Integration.
Import the project as a maven project
1. File → Import → Maven → Existing Maven Projects
2. Next, in Root Directory enter the local path to your repository clone
3. Next, Finish
You need to make sure the GWT xml files are on the classpath. All of these resources can be found in src/main/resources and src/test/resources directories.
1. Right-click on portal (project name) → Build Path → Configure Build Path...
2. Remove the "Exclude" filter from the source tab on the src/main/resources and src/test/resources directories.
We need to tell the GWT Eclipse-plugin that this is a GWT project. Do this by Right-Clicking on the project in the package explorer and select: Google → Web Toolkit Settings...
1. From the resulting dialog make sure the "Use Google Web Toolkit" check box is selected.
2. Also make sure you are using the GWT SDK from your local maven repository (.m2/repository/com/google/gwt)
Verify GWT can compile your code
1. Right-Click on the project

...

1. from the package explorer.

...

1. Select Google→GWT Compile...
2. Under the "Entry Point Modules" you should see "PortalDebug - org.sagebionetworks.web", if not, then add it with the add button.
3. Remove the "Portal - org.sagebionetworks.web" entry.
4. The first time you run this you will be asked to select the output directory where GWT will compile the code. You want this to match the Maven WAR output directory, so use "target/portal-<VERSION>-SNAPSHOT"
5. If you get compilation errors from the JavaScript validator (i.e. NullPointerException), navigate to the project's

...

1. properties → JavaScript → Include Path → Source, and exclude all files from the source. If an OutOfMemoryError was thrown while compiling, you can click the "Advanced" tab towards the bottom of the Google

...

When the root pom.xml's version changes (upon each sprint's release), your dev mode will be broken (giving either a 404 or a 503 error when you try to view the page at http://127.0.0.1:8888/Portal.html?gwt.codesvr=127.0.0.1:9997. To fix this you can try the following things:

Remove all of your PLFM projects, check out a fresh copy of the PLFM project and start from the beginning of this document.
Performing a Maven update.
Alternatively, you can just rebuild your Eclipse's Portal project and keep your files in place:

Git pull update to the newest version
Delete your project from Eclipse (leave "Delete project contents on disk" UNCHECKED)
Go to the project root on your file system and delete .classpath, .project and the .settings/ folder
Go to Step 7 in "UI Development above"

If this still doesn't work, or your dev mode is broken for another reason, there are some further steps you can take.
(Check out this site for more information: http://www.eclipsezone.com/eclipse/forums/t61566.html)

Use the '-clean' argument.
1. One way to do this is to add -clean on its own line as the first line on the eclipse.ini file, and restart Eclipse.
Use a new workspace.
1. I followed the instructions in the site linked above. The gist of it is:
  1. Export your preferences to somewhere outside your Eclipse environment environment (Export -> General -> Preferences. Make sure that 'Export All' is checked.)
  2. Switch to a new workspace. (Switch Workspace -> Other, then name your new workspace.)
  3. Import your preferences into this new workspace (Import -> General -> Preferences, make sure 'Import All' is checked.)
  4. Import trunk as usual.

Technologies used

There are several technologies that we are currently utilizing in the portal. The following is a list of primers for each:

For dependency injection (IoC) we are using a combination of GIN and Guice.
- Guice - is the base IoC technology but it is only fully functional in Server-side. http://code.google.com/docreader/#p=google-guice&s=google-guice&t=Motivation
- Gin - provides basic IoC for GWT client-side code. http://code.google.com/p/google-gin/wiki/GinTutorial
- Guice Servlet - This is a sub-project of Guice that allows dependency inject into Servelets. http://code.google.com/p/google-guice/wiki/ServletModule
Http REST calls are currently made using Spring's RestTemplate. http://blog.springsource.com/2009/03/27/rest-in-spring-3-resttemplate/
To create API REST stub-services we are using a combination of two technologies:
- Jersey - provides a quick and easy method for transforming simple POJO's into RESTful web services using annotations. http://jersey.java.net/nonav/documentation/latest/user-guide.html
- Grizzly - The GrizzlyWebContainerFactory makes it simple to start a local web container. http://blog.msbbc.co.uk/2008/11/java-using-jersey-and-grizzly-to-create.html

Virtualization

We have a shared account to use BrowserStack to test the website for cross browser compatibility issues.
Alternatively, for testing IE on Mac I use VirtualBox with an image provided by Microsoft from modern.IE. For this to work, in your Windows VM, you will need to map an ip to outer in order to access your local machine (note, Windows UAC requires you to do this as an admin).

...

1. → GWT Compile window and add "-Xms512M -Xmx1524M" to the VM arguments to increase heap space.

...

Right-Click on the project in the package explorer
Select: Run As->Web Application
Double-click on the link provided in the Development Mode window to view the portal in your browser. If you get asked what page to start on choose Portal.html
If you get a 404 error, you may need to navigate to the project root directory and "mvn clean install".

Setup GWT SuperDevMode

As of Chrome version 39, the old GWT Development Mode is no longer available.
Using Super Dev Mode (SDM) means running two servers, one for the web application and one for the Code Server that compiles classes for SDM when the compile button is pushed on the web page or in the bookmark.

First thing to do is to make sure that your Google Plugin for Eclipse is up to date (version >= 3.8.0.v201410302155-rel-r42).
NOTE: After we update to GWT v2.6 or greater, this section needs to be rewritten (can be a single launch configuration if using GWT 2.6).

In your existing web application Portal launch configuration (created in the previous section above), add the following to the VM Arguments: "-Xmx512m -XX:MaxPermSize=512m -Dgwt.codeserver.port=9876"

Now it's time to create a new Run Configuration for the Code Server:

From the Run Configurations window, add a new Java Application, named "Portal GWT Super Dev Mode Code Server" (for example).
Set the Project to the Portal project.
Set the Main Class to com.google.gwt.dev.codeserver.CodeServer
Set the Program Arguments to "-src <SynapseWebClient_root_directory>/src/main/java -port 9876 org.sagebionetworks.web.Portal"

You should now be ready to launch the Code Server. Here are a couple of common problems...

Missing jars in your WEB-INF/lib directory (the maven dependencies)? Go to the command line and run: "mvn eclipse:eclipse -Dwtpversion=2.0" to automatically add these libraries to the Deployment Assembly. After running this command, you may get this error in Eclipse: "Java compiler level does not match the version of the installed Java project facet." If so, bring up the Portal project Properties, select Project Facets from the list, and fix the Java version.
Serialization problems? Due to this known issue that applies to GWT v2.5.1, you should always run the Code Server first, then start the Portal web application once the code server is available. We now copy over gwt.rpc files on servlet init as a workaround until we upgrade to GWT v2.6.

Almost done!

Launch the Code Server, and go to the code server webpage (http://localhost:9876/) when it is ready. Now drag the bookmarklets to your browser's bookmark bar.
Launch the Portal Web Application run configuration, and go to http://127.0.0.1:8888/Portal.html . In this window, click the Dev Mode On bookmarklet that you created from the previous step. This will cause the code server to compile code for your user agent.
The Portal should now be visible. To debug, bring up the Developer Tools. In Chrome, select the Sources tab, and expand the code server url (localhost:9876) until you find the java file that you would like to debug. You can set breakpoints in this view.

Eclipse Workspace Settings

Set file encoding to UTF-8 and the line-endings (for new files) to Unix, so that text files are saved in a format that is not specific to the Windows or Mac OS and most easily shared across heterogeneous developer desktops:

Open Preferences
Navigate to the Workspace preferences: General->Workspace
Change the Text File Encoding to UTF-8
Change the New Text File Line Delimiter to Other and select Unix from the drop-down

...

SWC uses Sass. SCSS compilation is done by Maven during the compile step. To do this automatically in Eclipse (without compiling the entire app), we can add a custom builder

Open the project properties and navigate to Builders

Create a new Program Builder

Set the Location value to the installed node binary: ${workspace_loc:/portal/node/node}
Set the Working directory to the SWC project (e.g. ${workspace_loc:/portal})

Set the following arguments:

Code Block

${workspace_loc:/portal/node/yarn/dist/bin/yarn.js}
sass
--no-source-map
--style=compressed
--unicode
--stop-on-error
--update 
--load-path=node_modules
src/main/webapp/sass:target/portal-develop-SNAPSHOT/generated

Under Build Options - Allocate Console, Launch in background, and run this builder during all operations listed.

Since we're using Jetty in Eclipse, we also need to make sure our JavaScript dependencies that we pulled from NPM are copied to the src/main/webapp folder
1. Open the project properties and navigate to Builders
2. Create a new Ant Builder
  1. Set the Buildfile to ${workspace_loc:/portal/copy_eclipse_deps.xml}
  2. Set the Base Directory to ${workspace_loc:/portal}
  3. Go to the Targets tab. For "Auto Build", click "Set Targets" and select copy
If the GWT Compile successfully compiled, then you're ready to run the application.
1. Right-Click on the project in the package explorer
2. Select: Run As → GWT Development Mode with Jetty, and select Portal.html
3. Double-click on the link provided in the Development Mode window to view the portal in your browser.
To run the unit tests:
1. Right-Click on the project from the package explorer.
2. Select: Run As → JUnit Test

Eclipse Workspace Settings verification:

Set file encoding to UTF-8 and the line-endings (for new files) to Unix, so that text files are saved in a format that is not specific to the Windows or Mac OS and most easily shared across heterogeneous developer desktops:

Open Preferences
Navigate to the Workspace preferences: General → Workspace
Change the Text File Encoding to UTF-8
Change the New Text File Line Delimiter to Other and select Unix from the drop-down

Expand

title	Setup with Intellij IDEA

Download IntelliJ IDEA
1. The community edition is free
2. The Ultimate edition requires purchasing a license. It includes enhanced support for GWT, but it’s not necessary for development.
Import the SynapseWebClient project into IntelliJ by File → New Project From Existing Sources... <Select Path to SWC Location>
(Optional) Setup running the GWT codeserver and Jetty instance from IntelliJ IDEA Ultimate
1. In the Menu bar Run > Edit configurations
2. Click the "+" button to add an new configuration and choose "GWT Configuration".
3. This is what the configuration looks like on Windows, adapt the file paths to your project repository's location
  1. Dev Mode Parameter should be:
    1. -war target/portal-develop-SNAPSHOT
      Image Added
4. The "Before Launch" tools can be ignored.
  1. It is simply a shortcut a execute the same command executed by the libsass-maven-plugin in the root pom.xml if using Maven to run GWT.
  2. The alternative is to run this command manually in the command line, whenever the CSS gets messed up.
    sass --no-source-map --load-path=node_modules --style=compressed --unicode --stop-on-error --update "./src/main/webapp/sass":"./target/portal-develop-SNAPSHOT"
    - This is caused by the GWT runner in IntelliJ sometimes clearing out the /target directory when recompiling
    - Even setting up SASS command in the "before launch" does ensure the CSS is compiled automatically
      - The CSS will be compiled immediately before the GWT server runs. The GWT server then clears out the /target folder containing the compiled CSS
      - The only benefit to the set up is that a tab for SASS is created next to the GWT Server's tab, which you can then manually run after the GWT server has started up.
      - This is very annoying and any improvements to this workflow would be greatly appreciated.
If running Maven manually run like so (see note below on saving build time by running in debug mode):
1. Run 'mvn clean install'
2. Run 'mvn gwt:run'
  1. Not recommended because it currently does not detect changes to the .ui.xml files, which forces you to kill, redo mvn clean install, and restart the server.
  2. Running in the IDE will detect .ui.xml changes

Troubleshooting

Check your exclusion filters through configure build path in Eclipse to assure nothing is excluded, then refresh and clean the SWC project.
(git) Pull the latest changes from the upstream repository.
Try to rebuild your Eclipse's Portal project and keep your files in place.
Perform a Maven update.

Delete your project from Eclipse (leave "Delete project contents on disk" UNCHECKED)
Go to the project root on your file system and delete .classpath, .project and the .settings/ folder
Go to "Set up with Eclipse" or "Set up with Intellij"

If using Eclipse, verify the web app directory is correct. Right-click on the project in the package explorer and select Properties:

From the tree on the left of the dialog navigate to Google → Web Application
Check the the check box: "This project has a WAR directory"
With the "Browse" button, select "src/main/webapp"
UN-CHECK the "Launch and deploy from this directory" This is very important, if you keep this checked then Maven will not be able to generate a clean WAR file. If you see "GWT needs to recompile" when you deploy your WAR then you probably have this box checked

If this still doesn't work, or your dev mode is broken for another reason, there are some further steps you can take.
(Check out this site for more information: http://www.eclipsezone.com/eclipse/forums/t61566.html)

Use the '-clean' argument.
1. One way to do this is to add -clean on its own line as the first line on the eclipse.ini file, and restart Eclipse.
Use a new workspace.
1. I followed the instructions in the site linked above. The gist of it is:
  1. Export your preferences to somewhere outside your Eclipse environment environment (Export → General → Preferences. Make sure that 'Export All' is checked.)
  2. Switch to a new workspace. (Switch Workspace → Other, then name your new workspace.)
  3. Import your preferences into this new workspace (Import → General → Preferences, make sure 'Import All' is checked.)
  4. Import trunk as usual.

Technologies used

There are several technologies that we are currently utilizing in the portal. The following is a list of primers for each:

Java!
Google Web Toolkit
React and the Synapse React Client
For dependency injection, an example of Inversion of Control (IoC), we are using a combination of GIN and Guice.
- Guice - is the base IoC technology but it is only fully functional in Server-side.
- Gin - provides basic IoC for GWT client-side code.
- Guice Servlet - This is a sub-project of Guice that allows dependency inject into Servelets.
Http REST calls are currently made using Spring's RestTemplate. http://blog.springsource.com/2009/03/27/rest-in-spring-3-resttemplate/
Mockito testing framework - mock dependent classes/interfaces to isolate test. Verify expected interactions (including async calls).
Bootstrap 3 UI framework (including a number of extras widgets), wrapped for GWT.
Markdown-it - for markdown processing . Extended the library by adding a number of plugins, available on npm. The Node.js plugin that's used to orchestrate the processing is called markdown-it-synapse
AWS js sdk - direct browser upload/download from s3-like storage.
jquery, moment js, font-awesome, mathjax, plot.ly, twitter, nodeca/pica (browser image resize), SparkMD5 (browser side md5 calculation), nchart and jsplumb (Dave's charting libraries for provenance), Google (analytics, search, closure-library, single sign on).
frontend-maven-plugin to install Node and Yarn at build-time, which provides numerous JavaScript dependencies via NPM.

End-to-end testing

Instructions on running end-to-end tests are included in this README.

How to limit GWT build permutations

To speed up build time, add the following property to your Maven settings.xml file

Code Block
<gwt.module.suffix>Debug</gwt.module.suffix>

How to point to a local instance of the Synapse backend

Follow the instructions on the Platform Bootstrap to set up local backend stack.
Build and launch a local backend stack. From an up-to-date clone of Synapse-Repository-Services on your local box:
1. Build the local stack by running mvn clean install -Dorg.sagebionetworks.database.drop.schema=true.
2. Now launch it:
  Code Block
  cd integration-test mvn cargo:run
3. Verify the services are running correctly by visiting http://localhost:8080/services-repository-develop-SNAPSHOT/repo/v1/version. You should see something like this:
  Code Block
  {"version":"develop-SNAPSHOT"}
In the SWC project root pom.xml, change the synapse.version tag to develop-SNAPSHOT, so that Maven utilizes artifacts that were built by running on your local stack.

Update the repository endpoint property in your maven settings.xml file:

Code Block
<org.sagebionetworks.repositoryservice.endpoint>http://localhost:8080/services-repository-develop-SNAPSHOT/repo/v1</org.sagebionetworks.repositoryservice.endpoint>

Restart the Portal app to load these properties from settings.xml.

Emails will not be sent when pointing to a local repo, the SynapseEmailService will instead write files to the devdata.sagebase.org s3 bucket in format <to_email_address>.json

How to point to a remote stack

Set the repository endpoint parameters in your maven settings.xml file.

To point to the development stack set:

Code Block
<org.sagebionetworks.repositoryservice.endpoint>https://repo-dev.dev.sagebase.org/repo/v1</org.sagebionetworks.repositoryservice.endpoint>

To point to staging set:

Code Block
<org.sagebionetworks.repositoryservice.endpoint>https://repo-staging.prod.sagebase.org/repo/v1</org.sagebionetworks.repositoryservice.endpoint>

To point to production set:

Code Block
<org.sagebionetworks.repositoryservice.endpoint>https://repo-prod.prod.sagebase.org/repo/v1</org.sagebionetworks.repositoryservice.endpoint>

Restart the Portal app to load this property from settings.xml.

Documenting complicated UI in the code

To get the big picture of widget relationships, sometimes it is helpful to have a picture. We are testing the use of tools like https://mermaid.js.org/ to help in these situations.

Compile Report

To generate a compile report, set enableClosureCompiler to false in the root pom.xml

Versions Compared

Old Version 58

New Version Current

Key

Table of Contents

UI Development

Set up the Maven Build

Set up the Eclipse Build

UI Development

Set up the Maven Build

Development

IDE Setup Instructions

Technologies used

Virtualization

Setup GWT SuperDevMode

Eclipse Workspace Settings

Troubleshooting

Technologies used

End-to-end testing

How to limit GWT build permutations

How to point to a local instance of the Synapse backend

How to point to a remote stack

Documenting complicated UI in the code

Compile Report