Build an 8 PS3 supercomputer

Build an 8 PS3 supercomputer

Summary: Less than a 10th the cost per GFlop of the $2500 supercomputerTake 8 PS 3 consoles, Yellow Dog Linux, a Gigabit Ethernet switch and your favorite protein folding or gravitational wave modeling codes and you're doing real science. On a Playstation!

SHARE:

Less than a 10th the cost per GFlop of the $2500 supercomputer Take 8 PS 3 consoles, Yellow Dog Linux, a Gigabit Ethernet switch and your favorite protein folding or gravitational wave modeling codes and you're doing real science. On a Playstation!

Try playing Ratchet & Clank on a Cray Most scientific computing is done on cluster computers. Blue Gene/L, the world's fastest supercomputer, uses 130,000 processors. Plus a lot of money, power and cooling.

At about $4 per billion of floating point operations (GFlops) the PS3 is the cheapest supercomputer building block available today.

Look under the hood The PS3's Cell Broadband Engine processor, or Cell, is a heterogenous multiprocessor. Instead of identical cores - like the Intel and AMD multi-core processors - the Cell consists of a 64bit PowerPC core and 8 “synergistic co-processor elements” (SPEs).

Each SPE has 256 KB local store, a memory controller and a “synergistic processing unit” (SPU) with a Single Instruction, Multiple Data processing unit and 128 registers of 128 bits each. They're connected by a bus with an internal bandwidth of more than 300 GB/s that transfers data between the SPEs.

The bottom line: you can go to Toys-R-Us and toss 200 GFlops into your shopping cart.

Sony, your friendly supercomputer vendor Sony generously donated 8 PS3 consoles to Professor Gaurav Khanna of the University of Massachusetts for his research on black holes and quantum cosmology.

picture-2.jpg This is a graphic of one black hole spiraling into another. It is representative of the problems Prof. Khanna is analyzing.

Doing a run on a conventional supercomputer cost him about $5,000 in grant money. For less than that he could have built the PS3 cluster and run anything he wanted. But Sony saved him even that trouble by donating the equipment.

This is serious stuff, right? So it has to be rack mounted. But the PS3 is so tiny:

ps3-super.jpg [photo courtesy of Prof. Khanna]

Do real work on a Playstation cluster Go to Terrasoft to get PowerPC Linux that runs on the PS3's . Go to IBM for version 3.0 of the developers kit.

Pick up a SCOP3, A Rough Guide to Scientific Computing On the PlayStation 3 by a team from the University of Tennessee that includes Jack Dongarra, longtime publisher of the Top 500 supercomputer list.

Get the MIT lecture notes from the Cell programming course.

Interested in ray tracing? Check out Ray Tracing on the Cell Processor (pdf) by Carsten Benthin, Ingo Wald, Michael Scherbaum andHeiko Friedrich. Note: if you don't already understand the math behind ray tracing you'll be lost in this highly technical paper.

Protein folding Your standalone PS3 can be part of a supercomputer project even if you don't build it yourself. Stanford's Folding@home protein-folding research can use your PS3's cycles to help understand the causes of Alzheimer's and many other diseases. Help save the *real* world.

The Storage Bits take A single Cell processor is roughly equivalent to 25 nodes on Blue Gene/L. While there are a number of architectural limitations to the Cell and the PS3 that limit its general applicability, it enables researchers to apply an incredible number of cycles to certain classes of problem. And Sony, IBM and Toshiba are hard at work on the next generation of the Cell.

On StorageMojo I've often addressed the consumerization of IT. The PS3 represents the consumerization of supercomputing. That will benefit us all.

Comments welcome, of course.

Update: One commenter wrote in to disparage the PS3 as a supercomputer, saying he'd had an 8 node cluster and found it no more than a toy. I asked Professor Khanna about it and he said:

I had done my homework before I built the cluster; the problems of memory and network latency he mentions were not an issue for me. So, its actually working really well for me. I am doing research-grade simulations on the cluster.

BTW, if he doesn't need his PS3s anymore .. maybe he can consider donating them to us here ;-)

Topics: Processors, Hardware, Mobility

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

49 comments
Log in or register to join the discussion
  • Consumerisation=toys not computing

    "On StorageMojo I?ve often addressed the consumerization of IT. The PS3 represents the consumerization of supercomputing. That will benefit us all."

    I'd change the last sentance to "That COULD benefit us all."
    But will it? Reading through a few channel 9 blogs a couple of days ago the backgrounder for one M$ senior developer read "joined M$ 13 years ago to work on a supercomputer project which evapourated 2 days before he arrived on campus." I don't think it's in the the interests of M$/Intel/DELL/CISCO et al to do some real design work and deliver a (home) supercomputer. Nor is it in their interests to develop a business supercomputer. They much prefer to package things up into PC's, expensive servers and 'hey you're not doing nuclear experimental modelling in your business, so why would you need a supercomputer?'

    Contrast this to Google's design for their global search engine and file system: a massive grid of commodity items tuned for value. The company is paying the bill you see!

    Can you think of a reason for the tier one vendors to start offering value? The only scenario I can think of is Google fail to attract enough business for apps. over the NET ... and decide to release a home network supercomputer in direct opposition to Vista and Windows Home Server and other such outdated, poor-value offerings.
    jacksonjohn
    • PS3 insides are no toys

      Game console need huge floating point computation power in SIMD configuration, and as it is not game consol business to design chips offering such capabilities, they turn tochip mker that know the job. That leaves IBM ATI and Nvidia. Thay take part in financing the hardware, and then ship the consoles at subsidized price.

      But at the same time the chip ends up in supercomputing configuration. Cell processor is to be integrated along Opteron for Los Alamos national laboratory to build a 32000 node supercomputer. It is just the same processor, transistor for transistor, they just hit different market segment.

      AT the same time it is possible, because for once Sony decided to open a little their infrastructure, to use the PS3 cell to build clusters at subsidised prices. Just to bad the PS3 contians unsable blue ray and graphic chip.
      s_souche
      • More Usable Than You Think

        GPUs can also be used to do <a href="http://en.wikipedia.org/wiki/Gpu#Stream_processing.2FGPGPU">significant amounts of processing</a>. It doesn't have to be just for visuals. So that leaves only a useless Blu-Ray in each unit. :)
        pj_mouse
        • to my knowledge

          PS3 running linux has a hardware abstraction layer that prevent you from accessing the GPU and eats up one of the SPE - ie 12% of its advantage over a PPC. To bad IBM HS blades are so expensive. I wonder if it would possible to retrieve the PS3 processor to plug it to a standard PPC workstation to gain real memory and complete access to hardware
          s_souche
          • Well..

            Linux runs on the PowerPC processor, not on a SPE, inside the
            Cell chip. The cell blades are called QS21 and aren't cheap.
            10K USD for one dual blade with 2Gigs of RAM.

            // Jesper
            JesperFrimann
        • Useless?

          I wouldn't say useless... pop them out and sell them on Ebay - you might get back close to 50% of the PS3 investment. Those drives are also part of the Sony subsidy...
          robaaa10
          • Why does everyone think Blu-ray useless?

            I for one, love watching movies in 1080P on Blu-Ray.
            What makes you all think they are useless?
            50GB's on a disc, seems benficial to me.

            All I've heard is talk of Sony, bad this & that.
            I have a 25+ year old 13" Color Sony TV. While, it wasn't built cable ready back then, it still works fine, with a cable box hooked to a converter!

            My 270 Watts x 4 150Watts x 2
            Sony Rack system, was bought in 1992. Every piece of it still works, although I bought a 25 CD changer & sold the 5CD changer that came with it to defer costs.

            Yes Sony made a big mistake, with the "ARccOS" Rootkit.
            No doubt. Yes they lost the Beta vs VHS war.

            Does this mean that it will be either HD or Blu-ray?
            I bet I'll see players with both one day...


            KelticDude
            KelticDude
    • Oh Ye of Little Faith

      I think we all know business models have nothing to do with benefits to society. However, there are those in the world who dedicate themselves, their time, and their efforts to benefit others. I believe the point of the article was to show what can be done with the Sony hardware by an interested third party, not to show that Sony had the world's interest at heart. Sony donated the PS3s because of the axiom "any publicity is good publicity"--all the better when it's actually something worthwhile. What the PS3 "can" do has little to do with Sony's marketing strategy or intent. They "happened" to have a rather powerful computing package developed for their purposes and someone else was able to put it to good use. It's the whole "unintended consequence" thing. The only question that remains unanswered is, "Will the people trying to do good win the race against the people who only have finiancial gain as their goal?" Time will tell.
      butlerwm
      • False dichotomy

        You're talking as if good is diametrically opposed to financial gain.
        ed.ahlsen-girard
  • Similar events with the PS2

    I recall having read an article several years ago where a university had created a cluster of PS2's and was leveraging their GPU's to perform complex calculations.

    Several years ago, I bought the Linux toolkit for the PS2 and couldn't do too much with it because of the 32MB of RAM. The PS3 changes that with its impressive specs.

    The bottom line is that Sony is allowing the community to benefit by opening its consoles and allowing them to be used as users see fit, while Microsoft (in typical fashion) is closed, proprietary, and anti-academia.
    davidr69
    • look, the MS bashing has begun...

      Sony is the KING of closed off everything. I give you Betamax, MiniDisc(which i LOVe but in order to use the Hi Speed recording on them, i HAVE to use Sony's clunky, crappy SonicStage).

      So no, "typical" fashion for Sony would be to lock it down as much as possible. This whole computers to the doc? yes it helps, but I can see from a mile away it is a publicity stunt for Sony to try to boost sales of a flagging, expensive console.
      ivanotter
      • Whaat?

        I agree that it is possible that Sony is looking for some marketing points here, but where do you get that it's MS bashing? Do you live in Redmond or something? Do you realize that you just did the same thing to Sony? Get a grip and a life, MS crony.
        drsparks9
        • its true

          Otter is correct. Sony is the king of proprietary.

          dont forget the Sony memory stick.
          lets not forget the rootkit viruses spawned from their copy protection either.

          Its a PR stunt. its a good one, but nothing more. Sony is far worse at over protecting their products compared to Msoft.
          zkrieger
          • Rootkit

            [i]lets not forget the rootkit viruses spawned from their copy protection either.[/i]

            That's why I dumped my Sony stuff, and why I'm getting my grandkids a X-box and NOT the PS3 they ask for.
            Me_too
          • I'd rather have a Wii anyday! (NT)

            nt
            JCitizen
          • Rootkit was a mistake, don't let it cloud your judgement

            If you let, that mistake, cloud your judgement, your grandkids will only suffer. The PS3, is capable, of letting them learn way more then the XBOX 360, ever could. This coming from a person who loves Windows, but knows the values of Learning Linux, as a youth!

            KelticDude
            KelticDude
          • A Senseless King at that...

            "its true
            Otter is correct. Sony is the king of proprietary"

            Sony is so proprietary that it refuses to provide customers with even so much as a simple Motherboard Manual for an OEM
            Asus supplied motherboard installed in my Vaio PC. Asus will
            not provide one either...as that data "belongs" to Sony.
            michaelleo9
        • I beleive it was the paragraph

          [i]The bottom line is that Sony is allowing the community to benefit by opening its consoles and allowing them to be used as users see fit, while Microsoft (in typical fashion) is closed, proprietary, and anti-academia.[/i]

          Now, I may have missed something, but was Microsoft or Xbox ever mentioned in the article?
          GuidingLight
      • They did the same thing with XBoxes

        If I remember correctly, Bill and co were getting pissed because people were building super clusters out of their toys.

        This was upsetting them because they were losing money on sales of the device and couldn't gain that money back if the Xbox wasn't used to play games.
        nucrash
        • They were shortsighted, if that was the case...

          Imagine the commercial hype you could put out for gaming TV commercials if you could claim your gaming consol could transform into a supercomputer!

          I'm speaking of Redmond, of course. I can't stand Sony, but it was a genius idea for PR blitz!
          JCitizen