Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Use the AWS console to create and download an a new SSH key named SageKeyPair and store
  2. Download it in to your home directory on the shared servers
  3. Fix the permissions on it
    Code Block
    
    chmod 600 SageKeyPair.pem
    

Configure S3

  1. Use the AWS console to make a new S3 bucket named sagetest-YourUsername and make these four subdirectories
    1. scripts
    2. input
    3. output
    4. results
    5. logs

...

  1. ssh to belltown
  2. Create the configuration file for the Elastic Map Reduce command line tool
    Code Block
    ~>cat YourUsername$USER-credentials.json 
    {
    "access_id": "YourAccessKeyID",
    "private_key": "YourSecretAccessKey",
    "keypair": "SageKeyPair",
    "key-pair-file": "/home/$user~/SageKeyPair.pem",
    "log_uri": "s3n://sagetest-YourUsername/logs/",
    "region": "us-east-1"
    }
    
  3. Test that you can run it
    Code Block
    ~>/work/platform/bin/elastic-mapreduce-cli/elastic-mapreduce --credentials ~/$USER-credentials.json --help
    Usage: elastic-mapreduce [options]
    
      Creating Job Flows
            --create                     Create a new job flow
            --name NAME                  The name of the job flow being created
            --alive                      Create a job flow that stays running even though it has executed all its steps
            --with-termination-protection
                                         Create a job with termination protection (default is no termination protection)
            --num-instances NUM          Number of instances in the job flow
    ...