A $525, 2048-core cluster in 45 minutes
10 years ago that would have been a top 5 supercomputer. Now all you need is a credit card and the Internet. Still think cloud computing is a fad?
A large biotech research group needed a lot of CPU cycles to run some simulations. Their own 2000-core cluster was fully booked. Call of Duty: Black Ops was just out!
They turned to CycleComputing's CycleCloud automated compute cluster creation system. You can build a cluster without them, but CycleCloud also handles details like:
- Scheduling
- Maintaining OS images
- Encryption and associated key management
- Deploying file systems
- Load-based scaling
Amazon Web Services (AWS) provided the horsepower.
Benchmarking AWS With AWS rolling out new services monthly it seems, figuring out what to use is a problem. CycleComputing ran some benchmarks to understand the cost-efficiency of AWS cluster options.
This job was compute, not I/O, intensive. AWS offers new Cluster Compute instances configured with storage and I/O to handle either 8 or 16 concurrent jobs, as well as the older 8-way High-CPU instances.
They found that while the 8 & 16 CC instances were faster than the 8 H-CPU instance, the latter was more cost-effective because it cost so much less. The 8 H-CPU got 25% more work done per dollar.
I know, it seems almost un-American for the latest and greatest to not deliver the best bang for the buck. Know your application!
Set up, take down They started setting up the cluster at 1030am and by a 1115 they were maxing out over 2000 cores. The job was done by 2pm and it took another 10 minutes to bring the cluster down. Total AWS price: $525.
The Storage Bits take With 1.7 TB of RAM and 2048 cores, this rent-a-cluster's computing power was mind-boggling a decade ago. There is a new job here: cloud computing architect.
But there's a larger lesson for the industry here: this was all done with commodity hardware. All the fancy branded storage, servers and networking that the BigCo's are peddling just isn't part of this equation.
Which is why you can do this for $525.
Cloud computing and storage are driving a new round of competition and consolidation in the IT industry. It will be rough on the companies, but good for us users.
Comments welcome, of course. CycleComputing wrote this up in their blog.