...
Code Block |
---|
~>cat phase.json [ { "Name": "MapReduce Step 1: Run Phase", "ActionOnFailure": "CANCEL_AND_WAIT", "HadoopJarStep": { "Jar": "/home/hadoop/contrib/streaming/hadoop-streaming.jar", "Args": [ "-input", "s3n://sagetest-YourUsername/input/phaseInput.txt", "-output", "s3n://sagetest-YourUsername/output/phaseTryNumber1phaseTry1", "-mapper", "s3n://sagetest-YourUsername/scripts/phaseMapper.sh", "-reducer", "s3n://sagetest-YourUsername/scripts/echoReducer.sh", "-cacheFile", "s3n://sagetest-YourUsername/scripts/phase#phase", "-jobconf", "mapred.map.tasks=1", "-jobconf", "mapred.reduce.tasks=1", ] } } ] |
...
Code Block |
---|
~>/work/platform/bin/elastic-mapreduce-cli/elastic-mapreduce --credentials ~/$USER-credentials.json --create --num-instances=1 --master-instance-type=m1.large --name phaseTry1 --json phase.json --name phaseTry1 Created job flow j-GA47B7VD991Q |
...
Code Block |
---|
~>/work/platform/bin/elastic-mapreduce-cli/elastic-mapreduce --credentials ~/$USER-credentials.json --list --jobflow j-GA47B7VD991Q --list j-GA47B7VD991Q RUNNING ec2-174-129-134-200.compute-1.amazonaws.com filesysTry1 RUNNING MapReduce Step 1: Run Phase |
...
Code Block |
---|
~>/work/platform/bin/elastic-mapreduce-cli/elastic-mapreduce --credentials ~/$USER-credentials.json --json phase.json --jobflow j-GA47B7VD991Q --json phase.json Added jobflow steps |
Get your results
...