20 great years of Linux and supercomputers

20 great years of Linux and supercomputers

Summary: Today, Linux rules supercomputing. It wasn't always that way. Here's how Linux moved from being Linus Torvald's hobby operating system to being the OS of choice for high-performance computing.


In the latest Top500 supercomputer rankings, 476 of the top 500 fastest supercomputers, 95.2 percent, in the world run Linux. Linux has ruled supercomputing for years. But, it wasn't always that way.

First Unix, and now Linux for the last few years, has ruled supercomputing.

When the first Top500 supercomputer list was compiled in June 1993, Linux was just gathering steam. Indeed, in 1993, the first successful Linux distributions, Slackware and Debian were only just getting off the ground.

What happened next, as reported in The Linux Foundation's report, 20 years of Top500.org Supercomputer Data Links Linux With Advances in Computing Performance, was that "after first appearing on the list in 1998, Linux has consistently dominated the top 10 over the past decade and has comprised more than 90 percent of the list since June 2010."

Before Linux made its move, Unix was supercomputing's dominant operating system. Since 2003, the top operating system by performance share on the Top500 List underwent a complete flip from 96 percent Unix to 96 percent Linux. By 2004, Linux had taken over the lead for good.

According to The Linux Foundation, "Linux [became] the driving force behind the breakthroughs in computing power that have fueled research and technological innovation. In other words, Linux is dominant in supercomputing, at least in part, because it is what is helping researchers push the limits on computing power."

The Foundation believes that this has happened because of two reasons. First, since most of the world’s top supercomputers are superscalar research machines built for specialized tasks, each supercomputer is a standalone project with unique characteristics and optimization requirements. Thus, it's not affordable for anyone to develop a custom operating system for each system. With Linux, however, research teams can easily modify and optimize Linux to the one-off, groundbreaking designs that characterize the modern generation of supercomputers.

And, just as importantly, "The licensing cost of a custom, self-supported Linux distribution is the same, whether you’re using 20 nodes or 20-million nodes." Thus, "by tapping into the vast open-source Linux community, projects had access to free support and developer resources to help keep developer costs on par with, or below other operating systems."

The result of this has been supercomputers that are going faster than ever. By total RMax, a supercomputer's maximum achieved performance on the Linpack benchmark, supercomputer performance has outpaced Moore’s Law (The number of transistors incorporated in a chip will approximately double every 24 months.) by doubling roughly every 14 months. At the top end, supercomputing is progressing at even more rapid rate. The RMax of the fastest supercomputer on the Top500 list has increased by a factor of three to reach the Tianhe-2’s 33.86 petaflop/second in 2013 from the CM-5’s 59.7 gigaflop/s in 1993."

Therefore, The Linux Foundation concluded, "By isolating RMax by operating system using the past 20 years of Top500 data, it’s clear that Linux is not only responsible for supporting the majority of supercomputers today, but is a driving force behind the disproportionate growth in supercomputing capacity over the past decade. In continuing to drive progress and innovation in computing, Linux is also helping to explore the mysteries of the universe and solve our toughest problems."

I can only agree with these conclusions.

Related Stories:

Topics: Hardware, Linux, Open Source, Operating Systems, Servers

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • Good article

    Linux powered super computers prove the case for open source. If not for the open source power of Linux there would only be a few supercomputers around the world and most of those would have costs taxpayers much much more....

    Even commercial operations such as Google would have not grown as fast if open source was not present in their business plan.
  • So, am curious. Why not BSD?

    1. BSD is open-source
    2. BSD is free (as in free beer)
    Rabid Howler Monkey
    • The BSD allows others to steal your improvements, and not share them.

      The GPL requires you to provide the source to them (as in share) when you distribute to your customers.

      It is also the case that the multiple supercomputer vendors are SHARING the development cost of the software. This reduces the costs of development (the software), which can then be put into more hardware development.

      Since the improvements are also shared with the upstream project(s), the kernel improves for everyone; from embedded linux systems with no memory management, Android, desktops, and anyone else using Linux.

      Thus the overall expenses ALSO go down.
      • The license is not relevant

        As many, if not all, of the recent Linux-based supercomputers that Steven is writing about are not distributed (no pun intended). They're unique and are built by and/or for the organizations that, ultimately, will be using them. And while most of these organizations are public (e.g., university, government) entities, some among them are part of the national defense infrastructure. Here's the current top 5 list:

        o Tianhe-2, or Milky Way-2, developed by China’s National University of Defense Technology, will be deployed at the National Supercomputer Center in Guangzho, China
        o Titan, a Cray XK7 system installed at the U.S. Department of Energy’s (DOE) Oak Ridge National Laboratory
        o Sequoia, an IBM BlueGene/Q system installed at DOE’s Lawrence Livermore National Laboratory
        o Fujitsu’s “K computer” installed at the RIKEN Advanced Institute for Computational Science (AICS) in Kobe, Japan
        o A second BlueGene/Q system, Mira, installed at Argonne National Laboratory

        Are you implying that Cray, IBM and/or U.S. DOE Laboratories have shared their source code with China’s National University of Defense Technology? And, vice versa? And that IBM and Cray have shared their source code with each other? And might there be any NDAs involved with Cray/IBM and U.S. DOE Laboratories?

        Organizations building supercomputers with BSD could choose to share their source code just as many of these organizations could choose to share their GNU/Linux source code.
        Rabid Howler Monkey
        • Guess who wrote SELinux - the Security Enhanced Linux layer


          Yeah, they do have to share their improvements. But there's a huge difference between sharing Linux kernel improvements and writing custom application software that doesn't have to be shared.
          • SELinux is distributed (it's not used soley at U.S. gov't installations)

            If SELinux were a Linux kernel enhancement available only for U.S. government use, then sharing would not be required. Instead, it would be a kernel patch (similar to the grsecurity patch), but available only to the U.S. government.

            Two prominent examples of SELinux distribution:

            o Red Hat has for quite some time incorporated SELinux into community-based Fedora and RHEL server, desktop and workstation
            o Google has very recently incorporated elements of SELinux into Android

            Also note that in addition to custom application software (a good point), there may be operating system software outside of the Linux kernel (think GNU/Linux as an example) that has not been open-sourced.
            Rabid Howler Monkey
        • GPL obligations

          As far as I know, an organisation can keep its source code secret as long as it doesn't publish its binaries.
    • BSD limitations

      As noted by others, the BSD licence not forcing IBM and others to share their changes with others (unless they only use it internally and not give it to customers), thus not promoting rapid kernel development to other Linux users that indirectly leaves BSD in the dust in terms of development. BSD is a great license for core + add on. GPL is great in getting those add ons shared.

      To take a simple example, do you know BSD doesn't even support NFS client volumes >2TB yet? Much less all the kernel tuning Linux has for multiple core and cpu support. In summary, BSD does not have all the research of scaling added back into that Linux has.
      John Lauro
      • "the BSD licence not forcing IBM and others to share their changes"

        Do changes need to be forced? Again, we are talking about supercomputers built for and used by public entities such as universities and federal government departments (e.g., U.S. DOE).

        I'll remind everyone that BSD (Berkeley Software Distribution) was created by a university, the University of California, Berkeley:

        "Berkeley Software Distribution (BSD, sometimes called Berkeley Unix) is a Unix operating system derivative developed and distributed by the Computer Systems Research Group (CSRG) of the University of California, Berkeley, from 1977 to 1995."

        Here's a link to the U.S. Argonne National Laboratory/IBM wiki for the open-source BlueGene project (I *believe* BlueGene/P):

        "Welcome to the wiki for the BG/P Open Source project sponsored by Argonne National Laboratory and IBM."
        "Argonne and IBM are partnering to support the Open Source development of Blue Gene software. The computing community will have access to the bulk of the Blue Gene systems software and can participate in its development. A tentative schedule is shown below. The main open source mailing list is bg-opensource."

        Such cooperation and sharing couldn't happen with a BSD licensed OS such as FreeBSD, as an example? [Note: did anyone catch the reference to "the bulk of the Blue Gene systems software"? 'Bulk' means majority, not entirety.]

        Finally, a link regarding Linux kernel patches relating to IBM BlueGene/P:


        Again, why could this not happen with BSD?
        Rabid Howler Monkey
        • Re: Do changes need to be forced?

          I think several companies have tried to create proprietary enhancements on top of *BSD, only to go bust. Each such effort siphons off creative talent from the open-source community, but when it dies, all the work it has done dies with it, giving nothing back. So you have a net brain drain.

          This is less likely to happen with Linux. It has had its share of commercial failures, but at least they leave a legacy behind for others to build on.
        • Apparently

          Yes, changes could be made available for BSD. However, they don't have to be, and people are lazy. If they don't have to, many simple will not be Your example case, if that was based off of a BSD kernel, then they would likely not bother making the patches public and keep it to themselves for possible competitive advantage. However, it was based off of Linux and so they are legally bound to make the patches available.

          It's not so much that it couldn't happen with BSD, but BSD is just too far behind now to even consider it for anything complex.
          John Lauro
        • An example, TrustedBSD (see the SELinux comment above)

          Like SELinux does for GNU/Linux and Android, TrustedBSD provides Mandatory Access Controls (MAC) for FreeBSD:


          And if you click on the "Source code" link, you will learn that TrustedBSD is licensed "under a two-clause BSD-style license". Where else is Trusted BSD used?

          o NetBSD (open-source)
          o OpenBSD (open-source)
          o OS X (commercial)
          o iOS (commercial)

          Who sponsored and supported TrustedBSD?

          o Defence Advanced Research Projects Agency (DARPA)
          o National Security Agency (NSA)
          o Network Associates Laboratories
          o Safeport Network Services
          o University of Pennsylvania
          o Yahoo!, McAfee Research
          o SPARTA, Inc.
          o Apple Computer, Inc.
          o nCirce Network Security, Inc.
          o Google, Inc.
          o University of Cambridge Computer Laboratory
          o FreeBSD Foundation
          o and others

          Can you find the three-letter agency largely responsible for SELinux in this list? Also note the other U.S. government organization, DARPA. And universities, the University of Pennsylvania and University of Cambridge Computer Laboratory. Government agencies and universities, just like with supercomputers.

          So, stop already with the myth that nobody contributes to BSD.
          Rabid Howler Monkey
        • You seem to know more than the rest of us.

          I don't know why it can't happen. Please tell us though... why hasn't it happened?
    • Let me try

      I think all the previous answers are missing the obvious answer: Linux-based OSes are more widely-distrubuted than BSD. It's simply more familiar to more developers, including the ones responsible for these supercomputers.
      • foolishgrunt: "Let me try"

        Your answer makes great sense to me.

        In my own words, GNU/Linux won. And I don't believe that it was because of the GPL used for the Linux kernel. The AT&T subsidiary (Unix System Laboratories) lawsuit against BSD in the early '90's was a turning point.
        Rabid Howler Monkey
        • Allow me to add

          Not only the lawsuit helped in the adoption of Linux over BSD, which at the time was far more advance than the puny linux kernel. It is also the huge corporate support behind Linux. Taking from the graphic, it almost coincide with the time IBM started to promote Linux (remember the Eminem kid commercial), add to that CGI, SUN, and the plethora of big companies in the Linux foundation.

          If instead of supporting Linux, they would have supported BSD, the graph today would indicate BSD dominance instead of Linux.

          And for all those Linux advocates that are blind the beautiful idea of sharing the code, don't forget that are the big corporations driving those advances, which is way I, personally, don't use Linux and prefer BSD. Linux is as corporate as Microsoft, Apple, IBM or any other company.
  • so what...

    Unix...Linux... same principles...

    SJVN, please don't talk about Linux solving universe problems, I have to call you a total and utter moron... computers and software and built upon the principles of mathematics and physics...these has nothing to do with Linux and Windows.... so please cut the fanboy crap...
    • I guess you are one of the idiots that don't know software is mathematics.

      Because your statement makes no sense.
    • Sometimes I am really impressed

      by comments here and how insightful the person making them can be. Unfortunately you are not among of them.
    • Please Owlnet, I have great respect for you but this is wrong

      We need to ENCOURAGE SJVN to write blogs like this. It was factual, it wasn't snarky, there were no lies in it, and he didn't bring up any non relevant OS.

      When SJVN writes a bad blog (and he does this very very often) then he deserves to be slammed for it. However, let's encourage him to write more blogs like this one. This is a perfect example of how a good open source blog should be written.

      I can't help but wonder if some of the credit for this fine blog from SJVN is because so many responses in Andrew's recent blog about how to improve ZDNet resulted in people saying simply:
      "Less SJVN."

      So kudos to SJVN. I'm VERY happy that I got the opportunity to give you kudos. I like giving kudos because I like seeing a job well done. SJVN, this was a job well done. Keep up the good work.

      Todd Bottom