| Section | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Computing squares in R
...
The following script will download and install the latest version of R on each of your Elastic MapReduce hosts. (The default version of R is very old.)
Name this Download script bootstrapLatestR.sh and it should contain the following code:
| Iframe | ||||
|---|---|---|---|---|
|
...
Iframe
...
The following script will download and install several packages needed for RHadoop.
Name this Download script bootstrapRHadoop.sh and it should contain the following code:
| Iframe | ||||
|---|---|---|---|---|
|
...
| Iframe |
|---|
Upload your scripts to S3
...
| Code Block |
|---|
~>elastic-mapreduce --credentials ~/.ssh/$USER-credentials.json --create \ --master-instance-type=m1.small --slave-instance-type=m1.small \ --num-instances=1 --enable-debugging \ --bootstrap-action s3://sagebio-$USER/scripts/bootstrapLatestR.sh \ --bootstrap-action s3://sagebio-ndeflaux$USER$USER/scripts/bootstrapRHadoop.sh \ --name rmrTry1 --alive Created job flow j-79VXH9Z07ECL |
...
| Code Block |
|---|
~>elastic-mapreduce --credentials ~/.ssh/$USER-credentials.json --ssh --jobflow j-79VXH9Z07ECL ssh -i /home/ndeflaux/.ssh/SageKeyPair.pem hadoop@ec2-107-20-44-27.compute-1.amazonaws.com Linux domU-12-31-39-04-08-C8 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 -------------------------------------------------------------------------------- Welcome to Amazon Elastic MapReduce running Hadoop and Debian/Lenny. Hadoop is installed in /home/hadoop. Log files are in /mnt/var/log/hadoop. Check /mnt/var/log/hadoop/steps for diagnosing step failures. The Hadoop UI can be accessed via the following commands: JobTracker lynx http://localhost:9100/ NameNode lynx http://localhost:9101/ -------------------------------------------------------------------------------- hadoop@domU-12-31-39-04-08-C8:~$ |
Set JAVA_HOME and start R
...
| Code Block |
|---|
> q()Save workspace image? [y/n/c]: n hadoop@ip-10-114-89-121:/mnt/var/log/bootstrap-actions$ exit logout Connection to ec2-107-20-108-57.compute-1.amazonaws.com closed. ~>elastic-mapreduce --credentials ~/.ssh/$USER-credentials.json --terminate --jobflow j-79VXH9Z07ECL Terminated job flow j-79VXH9Z07ECL |
What next?
- Try the more complicated examples such as Logistic Regression and K-means in https://github.com/RevolutionAnalytics/RHadoop/wiki/Tutorial.
- Take a look at the Elastic MapReduce FAQ for how to SCP files to the Hadoop master host.
- Take a look at the other Computation Examples