Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Automatically configuring the nodes for arbitrary requirements;

 

Resources

Sources of Cloud Computing Resources

AWS

Google Big Compute

It should be easy to move the solution among different computing resources based on changing business needs.  The solution should be tolerant to moderately high failures in worker nodes; it's highly likely that we'd want to use AWS spot instances to reduce cost as many of our use cases do not require immediate results.

Process Initialization (allocate nodes, run generic 'worker' process on each node)

Sun Grid Engine

MapReduce/Hadoop

Job Assignment / Monitoring (create queue of jobs, assign jobs to nodes)

AWS Simple Workflow

AWS Simple Queue Service

...