Skip to end of banner
Go to start of banner

Developing SWF Workflows

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Next »

Getting Started

Make sure you can run the HelloWorld workflow.  In three xterms do the following:

(1) Start the HelloWorld Activity worker

~/platform/trunk/lib/lib-workflow>export MAVEN_OPTS=-javaagent:/Users/deflaux/.m2/repository/org/aspectj/aspectjweaver/1.6.10/aspectjweaver-1.6.10.jar ; mvn exec:java -Dexec.mainClass="com.amazonaws.services.simpleworkflow.flow.examples.helloworld.ActivityHost"
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'exec'.
[INFO] ------------------------------------------------------------------------
[INFO] Building Workflow Library
[INFO]    task-segment: [exec:java]
[INFO] ------------------------------------------------------------------------
[INFO] Preparing exec:java
[INFO] No goals needed for project - skipping
[INFO] [exec:java {execution: default-cli}]
[URLClassLoader@5898bbf6] info AspectJ Weaver Version 1.6.10 built on Friday Oct 22, 2010 at 03:50:26 GMT
[URLClassLoader@5898bbf6] info register classloader java.net.URLClassLoader@5898bbf6
[URLClassLoader@5898bbf6] info using configuration /Users/deflaux/platform/trunk/lib/lib-workflow/target/classes/META-INF/aop.xml
[URLClassLoader@5898bbf6] info register aspect com.amazonaws.services.simpleworkflow.flow.aspectj.AsynchronousAspect
[URLClassLoader@5898bbf6] info register aspect com.amazonaws.services.simpleworkflow.flow.aspectj.ExponentialRetryAspect
start: GenericActivityWorker [super=GenericActivityWorker[service=com.amazonaws.services.simpleworkflow.AmazonSimpleWorkflowClient@1769df00, domain=dev, taskListToPoll=HelloWorld, identity=6022@dhcp149230.fhcrc.org, backoffInitialInterval=100, backoffMaximumInterval=60000, backoffCoefficient=2.0], taskExecutorThreadPoolSize=100]
0 [com.amazonaws.services.simpleworkflow.flow.examples.helloworld.ActivityHost.main()] INFO com.amazonaws.services.simpleworkflow.flow.worker.GenericWorker  - start: GenericActivityWorker [super=GenericActivityWorker[service=com.amazonaws.services.simpleworkflow.AmazonSimpleWorkflowClient@1769df00, domain=dev, taskListToPoll=HelloWorld, identity=6022@dhcp149230.fhcrc.org, backoffInitialInterval=100, backoffMaximumInterval=60000, backoffCoefficient=2.0], taskExecutorThreadPoolSize=100]
Activity Worker Started for Task List: HelloWorld
Please press any key to terminate service.

(2) Start the HelloWorld Decider

~/platform/trunk/lib/lib-workflow>export MAVEN_OPTS=-javaagent:/Users/deflaux/.m2/repository/org/aspectj/aspectjweaver/1.6.10/aspectjweaver-1.6.10.jar ; mvn exec:java -Dexec.mainClass="com.amazonaws.services.simpleworkflow.flow.examples.helloworld.WorkflowHost"
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'exec'.
[INFO] ------------------------------------------------------------------------
[INFO] Building Workflow Library
[INFO]    task-segment: [exec:java]
[INFO] ------------------------------------------------------------------------
[INFO] Preparing exec:java
[INFO] No goals needed for project - skipping
[INFO] [exec:java {execution: default-cli}]
[URLClassLoader@2d35da43] info AspectJ Weaver Version 1.6.10 built on Friday Oct 22, 2010 at 03:50:26 GMT
[URLClassLoader@2d35da43] info register classloader java.net.URLClassLoader@2d35da43
[URLClassLoader@2d35da43] info using configuration /Users/deflaux/platform/trunk/lib/lib-workflow/target/classes/META-INF/aop.xml
[URLClassLoader@2d35da43] info register aspect com.amazonaws.services.simpleworkflow.flow.aspectj.AsynchronousAspect
[URLClassLoader@2d35da43] info register aspect com.amazonaws.services.simpleworkflow.flow.aspectj.ExponentialRetryAspect
start: GenericWorkflowWorker[super=GenericWorkflowWorker[service=com.amazonaws.services.simpleworkflow.AmazonSimpleWorkflowClient@786167d, domain=dev, taskListToPoll=HelloWorldWorkflow, identity=6036@dhcp149230.fhcrc.org, backoffInitialInterval=100, backoffMaximumInterval=60000, backoffCoefficient=2.0], workflowDefinitionFactoryFactory=com.amazonaws.services.simpleworkflow.flow.pojo.POJOWorkflowDefinitionFactoryFactory@742e6c66]
0 [com.amazonaws.services.simpleworkflow.flow.examples.helloworld.WorkflowHost.main()] INFO com.amazonaws.services.simpleworkflow.flow.worker.GenericWorker  - start: GenericWorkflowWorker[super=GenericWorkflowWorker[service=com.amazonaws.services.simpleworkflow.AmazonSimpleWorkflowClient@786167d, domain=dev, taskListToPoll=HelloWorldWorkflow, identity=6036@dhcp149230.fhcrc.org, backoffInitialInterval=100, backoffMaximumInterval=60000, backoffCoefficient=2.0], workflowDefinitionFactoryFactory=com.amazonaws.services.simpleworkflow.flow.pojo.POJOWorkflowDefinitionFactoryFactory@742e6c66]
Workflow Host Service Started...
Please press any key to terminate service.

(3) Run the HelloWorld Initiator to kick off one workflow

~/platform/trunk/lib/lib-workflow>export MAVEN_OPTS=-javaagent:/Users/deflaux/.m2/repository/org/aspectj/aspectjweaver/1.6.10/aspectjweaver-1.6.10.jar ; mvn exec:java -Dexec.mainClass="com.amazonaws.services.simpleworkflow.flow.examples.helloworld.WorkflowExecutionStarter"
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'exec'.
[INFO] ------------------------------------------------------------------------
[INFO] Building Workflow Library
[INFO]    task-segment: [exec:java]
[INFO] ------------------------------------------------------------------------
[INFO] Preparing exec:java
[INFO] No goals needed for project - skipping
[INFO] [exec:java {execution: default-cli}]
[URLClassLoader@1d162212] info AspectJ Weaver Version 1.6.10 built on Friday Oct 22, 2010 at 03:50:26 GMT
[URLClassLoader@1d162212] info register classloader java.net.URLClassLoader@1d162212
[URLClassLoader@1d162212] info using configuration /Users/deflaux/platform/trunk/lib/lib-workflow/target/classes/META-INF/aop.xml
[URLClassLoader@1d162212] info register aspect com.amazonaws.services.simpleworkflow.flow.aspectj.AsynchronousAspect
[URLClassLoader@1d162212] info register aspect com.amazonaws.services.simpleworkflow.flow.aspectj.ExponentialRetryAspect
Started helloWorld workflow with workflowId="c4bc4010-d1b6-4415-8d9a-40b7c410c1a5" and runId="57e222a2-52fb-4318-9cea-359bd5fdf11a"
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4 seconds
[INFO] Finished at: Mon Feb 27 16:40:56 PST 2012
[INFO] Final Memory: 14M/81M
[INFO] ------------------------------------------------------------------------
~/platform/trunk/lib/lib-workflow>

(4) Each time you run the Initiator, you should see some "hello world!" messages in the Activity worker output.  If not, log into the AWS console and look at the SWF tab to see what's going on with your workflows.

How to make a new workflow

(1) under PLFM/trunk/tools mvn archetype:generate to make your new code module

(2) use PLFM/trunk/tools/tcgaWorkflow/pom.xml as the example for your pom.  You'll want to copy the <dependencies>, <properties>, and <build> sections.  There's a lot of java magic wrapped up in that pom.

(3) if package for the workflow code is not already listed in PLFM/trunk/lib/lib-workflow/src/main/resources/META-INF/aop.xml add it there

(4) write your code, at a minimum you will likely want code similar to the following  TODO ADD LINKS AFTER THE MERGE TO TRUNK

  • workflow interface: 
  • activity interface:
  • workflow implementation:
  • activity implementation:
  • workflow runner (runs the decider):
  • activity runner (runs the activities):
  • workflow initiator (kicks off workflows):
  • workflow configuration utility:
  • workflow configuration properties:
  • workflow configuration properties template:
  • workflow test (important this will help you ensure that your Promise variables are configured correctly):
(5) run a local instance of Synapse 

The wiki generator and several of our integration tests expect a subset of prod SageBioCurated data to exist in the service. To populate a local stack with that data, do this by

  1. running the following single integration tests: 
    ~/platform/trunk/integration-test>mvn -Dit.test=IT100BackupRestoration verify
  2. and then once the database is populated we can restart the local stack in debug mode: 
    ~/platform/trunk/integration-test>mvn cargo:run
(6) run your activity runner, workflow runner, and workflow initiator in a manner similar to what you see above for the HelloWorld workflow

How to deploy a workflow

Talk to Nicole and Xa, we will set you up with something similar to Search, Daemons and Monitoring

Common Errors

Workflow Unit Tests Hang

This can happen when the activity or workflow code throws a runtime exception.  Unfortunately the unit testing framework provided by SWF does not log them.

"java.lang.IllegalStateException",{"cause":null,"message":"not ready","localizedMessage":"not ready"

This happens when you try to access a Promise variable in a context that does not ensure that the value must be "ready".  Here is an anti-example:

Promise<String> message = client.formulateNotificationMessage(layerId);
client.notifyFollowers(NOTIFICATION_SNS_TOPIC,
                       NOTIFICATION_SUBJECT, 
                       message.get(),         // WRONG!!!
                       message);              // trying to use this as a waitFor parameter, but this isn't the right way to do it

Here's the fixed example:

Promise<String> message = client.formulateNotificationMessage(layerId);
client.notifyFollowers(Promise.asPromise(NOTIFICATION_SNS_TOPIC), 
                       Promise.asPromise(NOTIFICATION_SUBJECT), 
                       message);

 

This can also happen when you have @Asynchronous annotations but aspectj weaving is not correctly configured and enabled.  Make sure:

(1) load-time weaving is enabled by passing JVM argument -javaagent:/Path/To/aspectj-1.6/aspectjweaver.jar

    1. for mvn exec:java you do this via setting the environment variable 

      export MAVEN_OPTS=-javaagent:/Path/To/aspectj-1.6/aspectjweaver.jar

    2. for the surefire plugin, this is configured via 

            <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <configuration>
          <forkMode>pertest</forkMode>
          <argLine>-javaagent:${settings.localRepository}/org/aspectj/aspectjweaver/${org.aspectj.version}/aspectjweaver-${org.aspectj.version}.jar</argLine>
        </configuration>
      </plugin>

    3. for Java it might look something like this 

      java -javaagent:/Users/deflaux/Downloads/a-java-sdk-1.3.3/third-party/aspectj-1.6/aspectjweaver.jar -cp target/tcgaWorkflow-0.11-SNAPSHOT.jar org.sagebionetworks.workflow.curation.TcgaWorkflowInitiator

(2) load-time weaving is configured correctly in PLFM/trunk/lib/lib-workflow/src/main/resources/META-INF/aop.xml You should see some log output like the following:

[AppClassLoader@1ef6a746] info AspectJ Weaver Version 1.6.10 built on Friday Oct 22, 2010 at 03:50:26 GMT
[AppClassLoader@1ef6a746] info register classloader sun.misc.Launcher$AppClassLoader@1ef6a746
[AppClassLoader@1ef6a746] info using configuration file:/Users/deflaux/platform/trunk/tools/tcgaWorkflow/target/tcgaWorkflow-0.11-SNAPSHOT.jar!/META-INF/aop.xml
[AppClassLoader@1ef6a746] info register aspect com.amazonaws.services.simpleworkflow.flow.aspectj.AsynchronousAspect
[AppClassLoader@1ef6a746] info register aspect com.amazonaws.services.simpleworkflow.flow.aspectj.ExponentialRetryAspect
  • No labels