Big Data on Amazon: Elastic MapReduce, step by step

Big Data on Amazon: Elastic MapReduce, step by step

Summary: Curious how to go about doing Hadoop in Amazon's cloud? Here's some guidance.

TOPICS: Big Data

 |  Image 7 of 29

  • Thumbnail 1
  • Thumbnail 2
  • Thumbnail 3
  • Thumbnail 4
  • Thumbnail 5
  • Thumbnail 6
  • Thumbnail 7
  • Thumbnail 8
  • Thumbnail 9
  • Thumbnail 10
  • Thumbnail 11
  • Thumbnail 12
  • Thumbnail 13
  • Thumbnail 14
  • Thumbnail 15
  • Thumbnail 16
  • Thumbnail 17
  • Thumbnail 18
  • Thumbnail 19
  • Thumbnail 20
  • Thumbnail 21
  • Thumbnail 22
  • Thumbnail 23
  • Thumbnail 24
  • Thumbnail 25
  • Thumbnail 26
  • Thumbnail 27
  • Thumbnail 28
  • Thumbnail 29
  • Configure EC2 instances

    In the Configure EC2 Instances screen, you'll need to select an Instance Type for your Master and Core Instance groups.  Amazon's "m1.large" instance type is the minimum required for an EMR cluster.  If you're creating a cluster just for learning purposes, this will be your least expensive and therefore most sensible option.  Select it for both the Master Instance Group and Core Instance Group.

  • Instance counts

    With your instance types selected, you now need to set the number of instances in your Core and Task Instance groups.  Again, if you're just putting up a cluster for learning purposes, you will want to minimize the resources you're using, so change the Core Instance Group's Instance Count from the default setting of 2 to just 1.  Leave the same setting for the Task Instance Group at 0, and click Continue.

  • Advanced options

    When you provisioned your AWS account, you should have created at least one EC2 key pair.  Pick one for your EMR cluster.  Without it, you won't be able to establish a secure terminal session and work interactively with Hadoop.  Once you've selected a key pair, click Continue.

Topic: Big Data

Andrew Brust

About Andrew Brust

Andrew J. Brust has worked in the software industry for 25 years as a developer, consultant, entrepreneur and CTO, specializing in application development, databases and business intelligence technology.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Related Stories


Log in or register to start the discussion