Heartbleed: Is the open source development model broken?

Moderated by Zack Whittaker | May 12, 2014 -- 07:00 GMT (00:00 PDT)

Summary: After Heartbleed, must open source development change?

Ed Bott

Ed Bott

Yes

or

No

Steven J. Vaughan-Nichols

Steven J. Vaughan-Nichols

Best Argument: Yes

35%
65%

Audience Favored: No (65%)

The Rebuttal

  • Great Debate Moderator

    Welcome to the Great Debate

    This week we have two of our top debaters, Ed Bott and Steven J. Vaughan-Nichols, ready to argue the fallout after the Heartbleed security flaw. Be ready for some action.

    Posted by Zack Whittaker

    I'm ready.

    And I really hope you are too Steven.

    Ed Bott

    I am for Yes

    I've done my homework

    Vote for me.

    Steven J. Vaughan-Nichols

    I am for No

  • Great Debate Moderator

    Inevitable?

    Let's start off simple enough. Could Heartbleed have been prevented? Yes or no, and why?

    Posted by Zack Whittaker

    It was a rookie mistake

    Of course this bug could have been prevented. The author of that section of code admits he made a rookie coding error: “I missed validating a variable containing a length," he said. And the one person who reviewed that code missed it as well.

    But that’s what happens when you don’t have a formal security review process. That is a requirement of modern software, especially code intended for use in the infrastructure of the Internet. And that level of process requires an owner and a management structure. That doesn’t imply an economic model, but it does require a level of financial and human commitment that wasn’t there in this example.

    Ed Bott

    I am for Yes

    It should have been prevented

    It certainly could have been prevented and it sure should have been prevented.

    The real problem was the blind faith that hundreds of thousands of programmers, Web site developers and Web administrators put into OpenSSL. If anyone with a programming clue had simply looked at the code they would have spotted it. They then could have reported it and what turned into an enormous security fiasco would have been a minor, short-lived embarrassment.

    Open source only works if people actually look at the code. If you've ever done serious skydiving you know that you must check your parachute before you get on the plane. For years, no one checked OpenSSL and so an obvious error hid in plain sight for years. The really amazing thing about Heartbleed is that it seems to have caused as little damage as it did. By rights, millions of "secure" Web sites should have suffered a failure as fatal as your chute note opening on the way down to the cold, hard ground.  

    Steven J. Vaughan-Nichols

    I am for No

  • Great Debate Moderator

    Define open source

    Which kinds of software, if any, should be open-source by definition? Explain why.

    Posted by Zack Whittaker

    Wrong question

    There are superb open-source programs. There are superb closed-source programs. There are also mediocre examples of each group. The people who write the code and the people who manage the processes that create the end product matter more than whether the code is available for public inspection.

    Ultimately it comes down to a matter of trust. Some people will want the ability to inspect code. Others don’t believe that’s important. There’s no evidence that either group is right or wrong.

    Ed Bott

    I am for Yes

    Open source should be everywhere

     Every kind of software can, and should be, open sourced. Indeed, I can't think of any programs that aren't open sourced these days.

    Oh sure, Microsoft doesn't open source Windows and Oracle doesn't open source Oracle Database, but we have dozens of Linux desktops such as Linux Mint  and Ubuntu and databases including MySQL and MariaDB.

    Indeed, once you get away from the desktop, it gets hard to find popular programs that aren't open source. The top Web server? Apache. Supercomputers overwhelming run Linux. The top 25 Web sites in the world, by Alexa's count, all run Linux.

    What about office software you say? Say hi to OpenOffice and LibreOffice. Games?  Valve CEO Gabe Newell is moving his company's entire Steam-based game portfolio to Linux. Mobile devices? You did know Android rules the smartphone market right?  Heck, Android-based tablets are now even more popular than iPads.

    Steven J. Vaughan-Nichols

    I am for No

  • Great Debate Moderator

    Open source or closed source?

    Is open-source software inherently more secure, or less secure than their 'closed-source' cousins?

    Posted by Zack Whittaker

    Again, neither

    You can make an interesting theoretical argument on behalf of either approach. Publishing source code makes it easier for good guys and bad guys to discover flaws. Closed source programs require would be attackers, with white or black hats, to test the program’s output.

    The larger issue is even when open source software is fixed, it can take a long, long time for those fixes to make it into the installed base. One study found that more than 98 percent of open source libraries with vulnerabilities were not updated in a timely fashion, even after those flaws were found and fixed.

    This is a fundamental problem faced by every organization that uses Open Source software and hasn’t implemented good patch management practices. Thousands and thousands of important software packages use open source libraries, but unless there’s someone assigned to monitor those libraries they risk being neglected.

    Ed Bott

    I am for Yes

    Open source - if they do their job

    It can be more secure if, and only if, it's actually done properly and people actually examine the blasted code. If they don't then it's no more secure than any other code.

    Objective studies, such as the one recently done by Coverity, have found that open source code has fewer errors per thousand lines of code than its proprietary brother. And, it's hard to ignore the Communications-Electronics Security Group (CESG), the group within the UK Government Communications Headquarters (GCHQ) that assesses operating systems and software for security issues, when they said that that while no end-user operating system is as secure as they'd like it to be, Ubuntu 12.04 is the best of the lot.

    That said, simply because a program is open source doesn't mean that it's magically more secure. On the other hand, if security really does matter to your company, you, if no one else, can actually dig through the code to look for problems. Just trying getting Microsoft to let you look at Internet Explorer's (IE) code for a bug hunt!

    Come to think of it, after the latest IE security hole, which hit versions from IE 6 to the latest IE 11, governments started recommending WIndows' users switch from IE to Chrome or Firefox (both open-source programs notch) that may not be such a bad idea.

    Steven J. Vaughan-Nichols

    I am for No

  • Great Debate Moderator

    The proprietary software question

    Particularly with security-related code and cryptographic libraries, such as OpenSSL, should these crucial foundations of the Internet be turned into proprietary software? Explain your answer.

    Posted by Zack Whittaker

    I’m not sure how that would work

    Again, there’s nothing magical about open or closed source. But the Heartbleed episode should raise awareness on the part of the maintainers of important programs of how critical it is to question security instead of just accepting it.

    My opponent wants us to believe that large organizations will step up to the plate and make this happen. That ignores the realities of software management in large organizations, where this is most definitely a cost center. Even companies like IBM and Red Hat that use open source software extensively don’t treat these projects as if they owned them. There’s no guarantee they will review the entire code base of a particular project, only the portions that directly affect them.

    From a selfish standpoint, a lot of businesses might be smart to take a closer look at commercial software (which is usually closed source) for crucial security infrastructure. With a single patch source and a single point of source control, it’s certainly easier to maintain.

    Ed Bott

    I am for Yes

    Are you serious

    You want me to trust a black box where I have no idea what's really happening inside it? II Don't Think So.

    I believe that security-related code and cryptographic library should be open-sourced. At the same time, we need to make sure that critical open-source projects, such as OpenSSL, aren't allowed to go for years without enough financing. That's why I think it was well past time that such initiatives as the Core Infrastructure Initiative, is getting ready to sponsor vital underfunded, open-source projects.

    Steven J. Vaughan-Nichols

    I am for No

  • Great Debate Moderator

    Should Apple share?

    We saw with Apple's 'got fail;' bug that even the major Silicon Valley titans are not perfect and can miss flaws — even with their vast resources. Would Apple have benefited by making some of its most critical security-related code available for public inspection?

    Posted by Zack Whittaker

    Probably not

    The Heartbleed flaw was out there for more than a year before anyone noticed it. Even large, well-run projects like Ubuntu can suffer from really stupid security flaws, like one last month that allowed anyone to bypass the lockscreen by just holding down Enter. (That was fixed quickly.)

    Without getting sidetracked into an Apple-versus-the-world tangent, I’ll repeat what I said earlier: This sort of problem is best addressed with a fundamental security review process that is specifically designed to find the most common errors. Whether it’s open or closed source, that is the proven way to deliver (more) secure code.

    Ed Bott

    I am for Yes

    Yes, they would have

    The irony is that Mac OS X is, at its foundation, based on the 4.4BSD-Lite2 open-source UNIX and the Open Software Foundation Mach 3 microkernel. Of course, Steve Jobs, a control freak's control freak as well as a genius, wasn't about to let anyone outside his reality distortion field, touch his baby, so it's all moot.

    That said, the OpenSSL error was almost as dumb as the goto fail and it went undetected for ages.

    Steven J. Vaughan-Nichols

    I am for No

  • Great Debate Moderator

    Who's responsible?

    Who, if anyone, is ultimately responsible for bugs or flaws in open-source? The creators/founders, the community, the end-users (including business customers) of the software, or a mixture of all three?

    Posted by Zack Whittaker

    The blame game

    I’m not sure there’s anything to be gained by playing this version of the blame game.

    Should we blame a community because it doesn’t have the resources to properly test code before shipping it? Should we blame customers who deploy software that appears to be stable, solid, and well tested? Of course not.

    I do think there is a case to be made for large businesses opening up their checkbooks and paying more to help maintain critical infrastructure like this. But in many cases they’re already paying hefty amounts to Red Hat or Oracle or IBM or another large company for support of that Open Source software. So maybe those companies that are directly benefiting from Open Source should be the ones writing those checks.

    Ed Bott

    I am for Yes

    Take charge

    Open source is often said to be created by the community. While some groups, say Debian Linux developers, interpret  that rather narrowly, most recognize that everyone, the first creators, the current generation of developers and maintainers, and the users are all members of the community.

    Some open-source projects, such as OpenStack, come right out and acknowledge that the users are an important part of the community. That doesn't mean that every user needs to be an expert programmer. Far from it! But, it does mean that if your company depends on a certain open-source program for its livelihood you darn well better have someone on staff who does know the code.  

    Steven J. Vaughan-Nichols

    I am for No

  • Great Debate Moderator

    Sharing responsibility

    Major enterprise firms don't contribute to the development, testing, and validation of the open-source software they often depend on. Should they?

    Posted by Zack Whittaker

    In a perfect world, yes

    The trouble is, the appeal of most Open Source software is that it’s free. And those big companies are beholden to their shareholders, who are likely to look at any significant line item and say, “Wait a minute, why are you paying for this stuff? I thought it was free?”

    And even when big companies choose to participate, they still don’t have unlimited resources, which means they have to pick and choose which parts they focus on and which parts they (continue to) neglect.

    The fact that something like the Core Infrastructure Initiative was “inspired by the Heartbleed OpenSSL crisis” is proof that the current Open Source development model is broken and needs fixing. But depending on how many projects are deemed “core infrastructure,” the costs could soar into the hundreds of millions of dollars. I’m skeptical that we’ll see that kind of commitment.

    Ed Bott

    I am for Yes

    Few are willing to pay the price

    Oh brother should they ever!

    That's been one of the great failings of open source. Everyone wants its goodies, but few are willing to pay for it. That's especially true of those important, but largely invisible plumbing programs, such as OpenSSL, that keeps the Web economy running.

    If your business lives and dies by such a program--and in the case of OpenSSL that's about 2/3rds of all e-commerce companies--for your own sake you should pay something to making sure that program will avoid problems that can knock your business out for the count.

    The saddest thing about this is that for a program, such as OpenSSL, even a few dollars a year from every business that used it--say the price of a fast food lunch--could have made the difference.

    Steven J. Vaughan-Nichols

    I am for No

  • Great Debate Moderator

    Public disclosure?

     Many security researchers and hackers decide to publicly disclose vulnerabilities in proprietary code, often because those companies ignore reports or prioritize them incorrectly.  Should open-source code be treated the same way?

    Posted by Zack Whittaker

    Why not

    There’s a double standard at work here, to be sure. Many commercial software companies devote enormous resources to security work and have a great customer focus, but if they don’t dance in response to a security researcher’s demands their customers suffer.

    That said, this outcome is unlikely precisely because there’s often no management to shame in an Open source project.

    Ed Bott

    I am for Yes

    It's happening now

    Actually that already happens. If you look at any major open-source program's bug reports you'll find users screaming that X problem hasn't been addressed and it needs to be fixed RIght Now.

    If the problem is ignored, what usually happens is people start complaining on the project's programming mailing lists. If that doesn't work, blog reports on the problems start appearing and so on with the news of the bug of the day getting ever wider until the developers are shamed into fixing the problem.

    And, if that doesn't work, you can always fix it yourself. If they don't accept your patch, then, if you really feel strongly enough about it, you can fork the code. If you were right and it was a major problem you might find yourself the proud parent of a new project. This is how open source works.

    In the particular case of OpenSSL and Heartbleed, that's exactly what the OpenBSD Unix developers have done. They've created LibreSSL. This fixes not only the Heartbleed problem but a lot of other rough spots in the OpenSSL code as well.

    Steven J. Vaughan-Nichols

    I am for No

  • Great Debate Moderator

    Elephant in the room

    The NSA said it had no knowledge of Heartbleed before it was first disclosed. Considering our tax dollars go to support the NSA's work (including cracking encryption, but also disclosing major flaws when it finds them), did the NSA drop the ball by not finding the flaw in the widely available code?

    Posted by Zack Whittaker

    Flawed premise

    The NSA is not tasked with disclosing major flaws when it finds them. That’s the job of US-CERT. The NSA’s job is to gather intelligence about foreign governments and potential threats to the United States. It can and should share information intelligently with US-CERT, but that’s not part of the core mission.

    Oh, and intelligence services from other countries, who had access to the same source code, apparently also dropped the ball.

    I’m certain the NSA works very hard with open and closed source projects to find ways to spy on its designated targets. Doesn’t really have much to do with this question.

    Ed Bott

    I am for Yes

    Shame, shame

    Here we are thinking that the NSA is the be-all and end-all of online peeping toms and what do we find? They seemingly never found the Heartbleed hole either! Oh the shame of it all!

    I'm reminded of the old saying that you should Never attribute to malice that which is adequately explained by stupidity. In this case, both the OpenSSL programmers, untold number of users, developers, and Web site administrators, and yes, the NSA as well, were all guilty of stupidity.

    Steven J. Vaughan-Nichols

    I am for No

  • Great Debate Moderator

    Lessons learned

    What is the most practical way of avoiding Heartbleed-like issues in the future?

    Posted by Zack Whittaker

    Security flaws will always be with us

    As long as code is written by human beings and delivered by organizations that have limited resources, vulnerabilities like this one will continue to keep security experts and IT pros busy for a long, long time.

    Fundamentally, it comes down to an inconvenient truth: Open Source software isn’t really free. Someone has to pay the costs of development, testing, and management. Part of that cost will continue to come from developers doing volunteer work, but it’s also up to big for-profit companies that benefit from Open Source to pony up as well.

    The real solution, the one that my opponent alluded to in his opening remarks, is twofold. First, increase the funding for crucial projects to pay developers properly instead of expecting them to write code in their spare time without compensation. Second, and more importantly, provide professional management and security review processes.

    Make those two fundamental changes, and we have a fighting chance to lessen the risk of further incidents like this one.

    Ed Bott

    I am for Yes

    Do open source right

    1. Check the code, then check it again, then a year from now have some intern check it again. Do you use the code? Check it yourself. Repeat, repeat, repeat. Some bugs really are subtle and can hide for years or maybe the chip architecture has changed and what was perfectly safe in an old server is now just asking for trouble on a new one.

    2. Invest money in open-source projects. Yes, many people write open-source code out of passion. Love doesn't pay the bills. You need to make sure that the top developers are paid what they're worth.

    3. Hire people to do the hard work of security auditing. Let's face it. Programmers have loads of fun making new things. They hate to document, they hate to do security audits and boy do they hate squashing bugs in their own or someone else's code.  Those all may be dirty jobs, but darn it, they're jobs that have to be done on both open-source and proprietary programs.


    Steven J. Vaughan-Nichols

    I am for No

  • Great Debate Moderator

    Thanks for joining us

    Once again the debaters did a great job and it's up to you to help pick the winner. On Wednesday Ed and Steven will submit their closing statements and I will post my decision. The comments are for your reading pleasure, please add your opinion. Don't forget to vote.

    Posted by Zack Whittaker

Talkback

101 comments
Log in or register to join the discussion
  • All models are "broken."

    All models are "broken." Software is written by imperfect humans, and debugged by imperfect humans and tools written by imperfect humans.

    That's just the reality we live in, sorry. Everything is equally "broken" in the sense that we're never going to see the end of bugs.

    Being open source may make it easier for the software to be vetted by random people, but is no guarantee that random people will vet it, or that those people will spot the problem and share it with the community.

    There are also things like the halting problem which say not everything can be solved. Not because we're human, but because the problem is actually mathematically unsolvable.

    Open source is not perfect - but it's probably the least broken model we've got. And the transparency of the code and the ability to share it gives it advantages well beyond debugging.
    CobraA1
    Reply 72 Votes I'm Undecided
    • But we carry on as if it isn't broken

      Open source just means you don't have to be a Wikileaks insider to se the code, and that the code is portable out from dying developers (e.g. post-Oracle Open Office).

      Those are two massive wins, but whether anyone actually picks up and fixes bugs is a matter of committed resources (i.e. folks paid for being responsible for doing that). Having a rudder on a boat isn't much good if there's no hand on the tiller.

      Code quality is so poor that we have to leave the door open for repairs on a pushed basis - yet we still develop as if one could actually trust code not to suck, blobbing everything together in one sprawling cloudy mass. We panic when a "12 year old OS" ceases to get patches, even after 12 years of repairs.

      This is akin to ignoring the Halting Problem, or the assertion that perpetual motion machines are impossible. It's really irresponsible to create an increasing dependence on materials known and proven to be unreliable.
      cquirke
      Reply 50 Votes I'm Undecided
    • The difference between OSS and Proprietary is testing. QA if you will.

      Good proprietary companies do code review but they also expose it to a QA group who's job it is to look for bugs.

      Some OSS may have QA but I don't know that is true for all the many libraries out there.

      I agree that flaws happen but mitigating risk involves a multi-tier approach.

      Speaking as one who works for a medical device company.

      Microsoft uses its employees computers for testing in addition to automated tests - every night and for many of them must run the latest build during the day. One test connects 128 usb devices at once. This is my personal favorite style of testing - stress testing. It isn't the most effective but it can be a lot of fun.
      MeMyselfAndI_z
      Reply 49 Votes I'm Undecided
      • OSS won the mind-share: 65% : 35%

        nt
        ac1234555
        Reply 40 Votes I'm Undecided
        • What mind share?

          An odd post.
          William.Farrel
          Reply 52 Votes I'm Undecided
          • Yet, you left out what is happening........ Now is later.....

            “it's happening right now.”
            daikon
            Reply 49 Votes I'm Undecided
      • Some OSS may have QA but I don't know that is true for all the many lib...

        You don't know that for proprietary software either.
        daddmac
        Reply 31 Votes I'm Undecided
    • True but, 95% of vulnerabilities already have patches published

      Whitesource just broke this down by the numbers in a study of 6,000 commercial projects. While 33% had vulnerabilities, 95% could have been patched. So updates need to happen regularly and yes to your point the resources need to be dedicated…

      Check out the infogrphic on this : http://bit.ly/forms1zoss
      Patricia.Johnson
      Reply Vote I'm Undecided
  • Funny. Stephen merely backed up what Ed Bott wrote.

    Stephen's 'defense' of open source seemed to just back up what Ed Bott wrote (and what I have believed for years). Trusting in volunteer, or poorly paid and largely uncoordinated people to vet code that someone else has written is fraught with dangers. No method is infallible but I doubt that all open source code has been vetted thoroughly. Up until recently, it hasn't been as big an issue but then again, up until the past few years, other OSs and software have dominated. I think the rise of Android has thrust Open source much more into the field of view of the 'bad guy' and we can only expect more flaws to be found as the years pass. If no one is paying me (a la open source programmers), why should I dedicate years of my life to maintaining a code? I just don't get it.
    mjm5
    Reply 64 Votes I'm for Yes
  • Open source isn't broken but free as in money is

    The issue isn't with source code being open or closed. The software socialism being these days is to blame for Heartbleed, not open source per se. If a company had to pay for an asset (Open SSL in this case) they would be far more likely to look at what they were actually paying for rather than just blindly using it because everyone else does and because "it's free." Ownership and stewardship go hand in hand, to use a loose analogy it's why renters are generally worse for a neighborhood than owners.

    On the other side, I am a developer and I can't understand what drives so many amazingly talented developers spend so much of their brainpower making other people rich. Maybe it's because they too don't want to own any failures and have any accountability as there's no one to get angry at you when no one paid you. To me this is a very immature way of doing things, developers should build great things, get paid for them when they are successful, and own up to failures when they happen. That's called being a grown up and until people understand that those who give away their work for free are not behaving as such we are doomed to repeat Heartbleed many times over.
    stiphy
    Reply 58 Votes I'm Undecided