Page Comparison

Getting Started

Make sure you can run the HelloWorld workflow. In three xterms do the following:

(1) Start the HelloWorld Activity worker

...

Console https://console.aws.amazon.com/swf/home
Documentation https://aws.amazon.com/documentation/swf/
- be sure to read the AWS Flow Framework Developer Guide
User forum https://forums.aws.amazon.com/forum.jspa?forumID=133
AWS Java SDK http://aws.amazon.com/sdkforjava/
- be sure to look at the sample code in aws-java-sdk-x.y.z/samples/AwsFlowFramework/
Make sure you are able to build and run Synapse locally. See Workflow Developer Bootstrap or Developer Bootstrap for instructions.

SageBio Workflow Examples

Hello World

Make sure you can run the HelloWorld workflow https://sagebionetworks.jira.com/source/browse/PLFM/trunk/lib/lib-workflow

In three shells do the following:

(1) Start the HelloWorld Activity worker

Code Block

~/platform/trunk/lib/lib-workflow>export MAVEN_OPTS=-javaagent:/Users/deflaux/.m2/repository/org/aspectj/aspectjweaver/1.6.10/aspectjweaver-1.6.10.jar ; mvn exec:java -Dexec.mainClass="com.amazonaws.services.simpleworkflow.flow.examples.helloworld.ActivityHost"
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'exec'.
[INFO] ------------------------------------------------------------------------
[INFO] Building Workflow Library
[INFO]    task-segment: [exec:java]
[INFO] ------------------------------------------------------------------------
[INFO] Preparing exec:java
[INFO] No goals needed for project - skipping
[INFO] [exec:java {execution: default-cli}]
[URLClassLoader@5898bbf6] info AspectJ Weaver Version 1.6.10 built on Friday Oct 22, 2010 at 03:50:26 GMT
[URLClassLoader@5898bbf6] info register classloader java.net.URLClassLoader@5898bbf6
[URLClassLoader@5898bbf6] info using configuration /Users/deflaux/platform/trunk/lib/lib-workflow/target/classes/META-INF/aop.xml
[URLClassLoader@5898bbf6] info register aspect com.amazonaws.services.simpleworkflow.flow.aspectj.AsynchronousAspect
[URLClassLoader@5898bbf6] info register aspect com.amazonaws.services.simpleworkflow.flow.aspectj.ExponentialRetryAspect
start: GenericActivityWorker [super=GenericActivityWorker[service=com.amazonaws.services.simpleworkflow.AmazonSimpleWorkflowClient@1769df00, domain=dev, taskListToPoll=HelloWorld, identity=6022@dhcp149230.fhcrc.org, backoffInitialInterval=100, backoffMaximumInterval=60000, backoffCoefficient=2.0], taskExecutorThreadPoolSize=100]
0 [com.amazonaws.services.simpleworkflow.flow.examples.helloworld.ActivityHost.main()] INFO com.amazonaws.services.simpleworkflow.flow.worker.GenericWorker  - start: GenericActivityWorker [super=GenericActivityWorker[service=com.amazonaws.services.simpleworkflow.AmazonSimpleWorkflowClient@1769df00, domain=dev, taskListToPoll=HelloWorld, identity=6022@dhcp149230.fhcrc.org, backoffInitialInterval=100, backoffMaximumInterval=60000, backoffCoefficient=2.0], taskExecutorThreadPoolSize=100]
Activity Worker Started for Task List: HelloWorld
Please press any key to terminate service.

...

Code Block

~/platform/trunk/lib/lib-workflow>export MAVEN_OPTS=-javaagent:/Users/deflaux/.m2/repository/org/aspectj/aspectjweaver/1.6.10/aspectjweaver-1.6.10.jar ; mvn exec:java -Dexec.mainClass="com.amazonaws.services.simpleworkflow.flow.examples.helloworld.WorkflowExecutionStarter"
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'exec'.
[INFO] ------------------------------------------------------------------------
[INFO] Building Workflow Library
[INFO]    task-segment: [exec:java]
[INFO] ------------------------------------------------------------------------
[INFO] Preparing exec:java
[INFO] No goals needed for project - skipping
[INFO] [exec:java {execution: default-cli}]
[URLClassLoader@1d162212] info AspectJ Weaver Version 1.6.10 built on Friday Oct 22, 2010 at 03:50:26 GMT
[URLClassLoader@1d162212] info register classloader java.net.URLClassLoader@1d162212
[URLClassLoader@1d162212] info using configuration /Users/deflaux/platform/trunk/lib/lib-workflow/target/classes/META-INF/aop.xml
[URLClassLoader@1d162212] info register aspect com.amazonaws.services.simpleworkflow.flow.aspectj.AsynchronousAspect
[URLClassLoader@1d162212] info register aspect com.amazonaws.services.simpleworkflow.flow.aspectj.ExponentialRetryAspect
Started helloWorld workflow with workflowId="c4bc4010-d1b6-4415-8d9a-40b7c410c1a5" and runId="57e222a2-52fb-4318-9cea-359bd5fdf11a"
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4 seconds
[INFO] Finished at: Mon Feb 27 16:40:56 PST 2012
[INFO] Final Memory: 14M/81M
[INFO] ------------------------------------------------------------------------
~/platform/trunk/lib/lib-workflow>

(4) Each time you run the Initiator, you should see some "hello world!" messages in the Activity worker output. If not, log into the AWS console and look at the SWF tab to see what's going on with your workflows.

MetaGenomics Pipeline

https://sagebionetworks.jira.com/source/browse/PLFM/trunk/tools/gePipeline

TcgaWorkflow

https://sagebionetworks.jira.com/source/browse/PLFM/trunk/tools/tcgaWorkflow

How to run it:

(1) run a local instance of Synapse with some TCGA datasets created

Include Page

	How to populate a local Synapse Stack with data
	How to populate a local Synapse Stack with data

(2) run the decider java -javaagent:/Users/deflaux/Downloads/aws-java-sdk-1.3.3/third-party/aspectj-1.6/aspectjweaver.jar -cp ~/platform/trunk/tools/tcgaWorkflow/target/tcgaWorkflow-0.11-SNAPSHOT.jar org.sagebionetworks.workflow.curation.TcgaWorkflowDeciderService

(3) run the activity worker java -javaagent:/Users/deflaux/Downloads/aws-java-sdk-1.3.3/third-party/aspectj-1.6/aspectjweaver.jar -cp ~/platform/trunk/tools/tcgaWorkflow/target/tcgaWorkflow-0.11-SNAPSHOT.jar org.sagebionetworks.workflow.curation.TcgaWorkflowActivityService

(4) Run the initiator java -javaagent:/Users/deflaux/Downloads/aws-java-sdk-1.3.3/third-party/aspectj-1.6/aspectjweaver.jar -cp ~/platform/trunk/tools/tcgaWorkflow/target/tcgaWorkflow-0.11-SNAPSHOT.jar org.sagebionetworks.workflow.curation.TcgaWorkflowInitiator

How to make a new workflow

(1) under PLFM/trunk/tools mvn archetype:generate to make your new code module

(2) use PLFM/trunk/tools/tcgaWorkflow/pom.xml as the example for your pom. You'll want to copy the <dependencies>, <properties>, and <build> sections. There's a lot of java magic wrapped up in that pom.

(3) sanity check your new package, every SageBio workflow has the Hello World workflow embedded in it. Make sure you can run it from your new package:

~/platform/trunk/tools/MyNewWorkflow>java -javaagent:/Users/deflaux/Downloads/aws-java-sdk-1.3.3/third-party/aspectj-1.6/aspectjweaver.jar -cp target/MyNewWorkflow-

...

(4) Each time you run the Initiator, you should see some "hello world!" messages in the Activity worker output. If not, log into the AWS console and look at the SWF tab to see what's going on with your workflows.

How to make a new workflow

(1) under PLFM/trunk/tools mvn archetype:generate to make your new code module

(2) use PLFM/trunk/tools/tcgaWorkflow/pom.xml as the example for your pom. You'll want to copy the <dependencies>, <properties>, and <build> sections. There's a lot of java magic wrapped up in that pom.

...

0.11-SNAPSHOT.jar com.amazonaws.services.simpleworkflow.flow.examples.helloworld.ActivityHost
~/platform/trunk/tools/MyNewWorkflow>java -javaagent:/Users/deflaux/Downloads/aws-java-sdk-1.3.3/third-party/aspectj-1.6/aspectjweaver.jar -cp target/MyNewWorkflow-0.11-SNAPSHOT.jar com.amazonaws.services.simpleworkflow.flow.examples.helloworld.WorkflowHost
java -javaagent:/Users/deflaux/Downloads/aws-java-sdk-1.3.3/third-party/aspectj-1.6/aspectjweaver.jar -cp target/MyNewWorkflow-0.11-SNAPSHOT.jar com.amazonaws.services.simpleworkflow.flow.examples.helloworld.WorkflowExecutionStarter

(4) if the Java package for the workflow code is not already listed in PLFM/trunk/lib/lib-workflow/src/main/resources/META-INF/aop.xml add it there

(45) write your code, at a minimum you will likely want code similar to the following TODO ADD LINKS AFTER THE MERGE TO TRUNK

workflow interface:
activity interface:
workflow implementation:
activity implementation:
workflow runner (runs the decider):
activity runner (runs the activities):workflow initiator (kicks off workflows):runs the activities)
workflow initiator (kicks off workflows) note that this particular example is rather complicated, an initiator could just kick off a single workflow instance
workflow configuration utility
workflow configuration properties
workflow configuration properties template
workflow test (important this will help you ensure that your Promise variables are configured correctly):

(56) run a local instance of Synapse

Include Page

	How to populate a local Synapse Stack with data
	How to populate a local Synapse Stack with data

(67) run your activity runner, workflow runner, and workflow initiator in a manner similar to what you see above for the HelloWorld workflow

...

Talk to Nicole and Xa, we will set you up with something similar to Search, Daemons and Monitoring

...

Daemons and Monitoring

Common Errors

Workflow Unit Tests Hang

This can happen when the activity or workflow code throws a runtime exception such as a NullPointerException. Unfortunately the unit testing framework provided by SWF does not log them.

"java.lang.IllegalStateException",{"cause":null,"message":"not ready","localizedMessage":"not ready"

Promise Variables were used Incorrectly

This happens when you try to access a Promise variable in a context that does not ensure that the value must be "ready". Here is an anti-example:

...

Code Block
Promise<String> message = client.formulateNotificationMessage(layerId); client.notifyFollowers(Promise.asPromise(NOTIFICATION_SNS_TOPIC), Promise.asPromise(NOTIFICATION_SUBJECT), message);

Load-Time Weaving Not Enabled

This can also happen when you have @Asynchronous annotations but aspectj weaving is not correctly configured and enabled. Make sure:(1) loadsure load-time weaving is enabled by passing JVM argument -javaagent:/Path/To/aspectj-1.6/aspectjweaver.jar

for mvn exec:java you do this via setting the environment variable
Code Block
export MAVEN_OPTS=-javaagent:/Path/To/aspectj-1.6/aspectjweaver.jar

for the surefire plugin, this is configured via

Code Block

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <configuration>
          <forkMode>pertest</forkMode>
          <argLine>-javaagent:${settings.localRepository}/org/aspectj/aspectjweaver/${org.aspectj.version}/aspectjweaver-${org.aspectj.version}.jar</argLine>
        </configuration>
      </plugin>

for Java it might look something like this

Code Block

java -javaagent:/Users/deflaux/Downloads/a-java-sdk-1.3.3/third-party/aspectj-1.6/aspectjweaver.jar -cp target/tcgaWorkflow-0.11-SNAPSHOT.jar org.sagebionetworks.workflow.curation.TcgaWorkflowInitiator

...

Load-Time Weaving Not Configured

This can also happen when you have @Asynchronous annotations but aspectj weaving is not correctly configured. Make sure load-time weaving is configured correctly in PLFM/trunk/lib/lib-workflow/src/main/resources/META-INF/aop.xml You should see some log output like the following:

...

Versions Compared

Old Version 6

New Version Current

Key

Table of Contents

Getting Started

SageBio Workflow Examples

Hello World

MetaGenomics Pipeline

TcgaWorkflow

How to make a new workflow

How to make a new workflow

Common Errors

Workflow Unit Tests Hang

"java.lang.IllegalStateException",{"cause":null,"message":"not ready","localizedMessage":"not ready"

Promise Variables were used Incorrectly

Load-Time Weaving Not Enabled

Load-Time Weaving Not Configured