A $525, 2048-core cluster in 45 minutes

Summary:10 years ago that would have been a top 10 supercomputer. Now all you need is a credit card and the Internet. Still think cloud computing is a fad?

10 years ago that would have been a top 5 supercomputer. Now all you need is a credit card and the Internet. Still think cloud computing is a fad?

A large biotech research group needed a lot of CPU cycles to run some simulations. Their own 2000-core cluster was fully booked. Call of Duty: Black Ops was just out!

They turned to CycleComputing's CycleCloud automated compute cluster creation system. You can build a cluster without them, but CycleCloud also handles details like:

  • Scheduling
  • Maintaining OS images
  • Encryption and associated key management
  • Deploying file systems
  • Load-based scaling

Amazon Web Services (AWS) provided the horsepower.

Benchmarking AWS With AWS rolling out new services monthly it seems, figuring out what to use is a problem. CycleComputing ran some benchmarks to understand the cost-efficiency of AWS cluster options.

This job was compute, not I/O, intensive. AWS offers new Cluster Compute instances configured with storage and I/O to handle either 8 or 16 concurrent jobs, as well as the older 8-way High-CPU instances.

They found that while the 8 & 16 CC instances were faster than the 8 H-CPU instance, the latter was more cost-effective because it cost so much less. The 8 H-CPU got 25% more work done per dollar.

I know, it seems almost un-American for the latest and greatest to not deliver the best bang for the buck. Know your application!

Set up, take down They started setting up the cluster at 1030am and by a 1115 they were maxing out over 2000 cores. The job was done by 2pm and it took another 10 minutes to bring the cluster down. Total AWS price: $525.

The Storage Bits take With 1.7 TB of RAM and 2048 cores, this rent-a-cluster's computing power was mind-boggling a decade ago. There is a new job here: cloud computing architect.

But there's a larger lesson for the industry here: this was all done with commodity hardware. All the fancy branded storage, servers and networking that the BigCo's are peddling just isn't part of this equation.

Which is why you can do this for $525.

Cloud computing and storage are driving a new round of competition and consolidation in the IT industry. It will be rough on the companies, but good for us users.

Comments welcome, of course. CycleComputing wrote this up in their blog.

Topics: Software, Amazon, Browser, Cloud, Enterprise Software, Networking, Storage

About

Harris has been working with computers for over 35 years and selling and marketing data storage for over 30 in companies large and small. He introduced a couple of multi-billion dollar storage products (DLT, the first Fibre Channel array) to market, as well as a many smaller ones. Earlier he spent 10 years marketing servers and networks.... Full Bio

zdnet_core.socialButton.googleLabel Contact Disclosure

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Related Stories

The best of ZDNet, delivered

You have been successfully signed up. To sign up for more newsletters or to manage your account, visit the Newsletter Subscription Center.
Subscription failed.