Which Web browser crashes the most?

Which Web browser crashes the most?

Summary: When it comes to crashing, just like with speed, Sauce Labs finds that not all Web browsers are created equally.

SHARE:

Sauce Labs, an Internet software testing and development company, specializes in automated testing for mobile and Web applications by delivering a test environment that accommodates over 150 browsers, operating systems, and device platforms. In the course of all that testing they've gotten to know which ones break the most and which ones break the least. Now, for the second time, Sauce is revealing which browsers can stand the test of the Web and which ones buckle under the strain.

Browser_Errors_2013 (1)
Taken all supported versions together, IE is the worst, but considering only up-to-date browsers, Safari 6 is the worst and Firefox 22 is the best. (Credit: Sauce)

Sauce has kept records of every Web browser failure for all the tests that ever ran on their service—all 50 million of them. For the purposes of the report Sauce only analyzed browser versions for which the company had at least 1,000 jobs in the last two years. This means that the most recent versions of some browsers aren't represented in the report.

Taken all-in-all, the bottom of the barrel Web browser family was Internet Explorer (IE). But, that's not the entire story.

True, with a 0.25 percent error rate, IE was the worst Web browser, but each browser has many different versions and a close look reveals that Microsoft's Web browsers have greatly improved since IE 7 first appeared in 2006. As for IE 6, let's not even talk about it!

While IE 7 has an error rate of .29 percent, the latest version, IE 10, has "an impressive 0.05 percent error rate." Sauce has found that Microsoft has gotten much better at "squashing their historically error rates."

So, if IE has been significantly improved, which browser has the "honor" of being the poorest program? That regrettable position, of currently deployed and supported Web browsers, goes to Apple's Safari 6.

The best browser, in terms of the fewest number of defects, is Firefox 22—followed by Chrome 27, IE10, Opera 12, and Safari 6. According to Sauce, "Most versions of Chrome have error rates low enough that they didn't show up on the graph. Neither do later versions of Firefox. That means that Chrome and Firefox are solid in terms of performance. For the tech savvy that isn't too surprisingly."

All-in-all, compared to the last Sauce evaluation of browsers in 2011, the company found that "Half of the browser versions we analyzed had error rates lower than 0.07-percent. That's pretty low, and suggests that browsers are getting more reliable as more versions come out."

Related Stories:

Topics: Networking, Browser, Web development

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

80 comments
Log in or register to join the discussion
  • IE

    I work, read, get informed every single day on my conected life on IE. I can't remember the last it crashed on Windows 7 or 8. IE11 is faster but it is not stable for now (Windows RT 8.1 preview)
    gbouchard99@...
    • Yes, but Sauce can remember. And not just for IE.

      "I can't remember the last it crashed on Windows 7 or 8."

      That's not really relevant to the article. Sauce ran the automated tests, Sauce *can* remember the last time each browser crashed, and now Sauce has published its results.

      And for what it's worth, it's not even saying that IE10 is "bad" - just that Firefox 22 and Chrome 27 are better.
      Zogg
    • IE Problems

      Results are about what I expected, and I can confirm that the latest version of Safari stinks. I'm glad to hear that IE has improved. I retired at the end of 2008, and in my work supporting computer users, IE resulted in more support calls than all others combined, and not because of market share, as this was in a university environment, where Safari was the most used browser.
      S_Deemer
      • But that's because it's running on Windows

        Rather than OS X to get crash rates that high.
        Troll Hunter J
        • So what you're saying is that Apple can't code for Windows?

          Case in point, iTunes. Its the worst program to ever be written for Windows. To be fair, its not all that stellar on OSx, but is it better there.
          A Gray
          • Re: So what you're saying is that Apple can't code for Windows?

            Nobody can. Even Microsoft's own Internet Explorer has a long history of being crashy and buggy. If Microsoft cannot bring out high-quality Windows software, who can?
            ldo17
          • Read the article, not just the comments...

            "The latest version, IE 10, has "an impressive 0.05 percent error rate." Sauce has found that Microsoft has gotten much better at "squashing their historically [sic] error rates.""
            Nierteroth9
          • Nobody can code for Windows...

            Other than all the developers who created the hundreds of programs I've used over the years. Seriously, during my times using iTunes, it had more problems than all of my other programs. The idea is obvious: Apple wants to make Windows look buggy. That tactic has definitely worked on you.
            trainman261-2
    • IE

      Thank god. Now, when performing a rebuild, I can confidently expect it to perform long enough to download another browser.
      FI3RYPH03N1X
  • Presumably, This Is On Dimdows

    I run Firefox/Iceweasel on Debian Unstable, and it works fine for weeks at a time without needing to be quit and relaunched. And I can't remember the last time it crashed--must have been some years ago.
    ldo17
    • And you would fail, then.

      "...delivering a test environment that accommodates over 150 browsers, operating systems, and device platforms"

      Note the plural (operating systems) is used, so it's going to be more than just a Windows operating system.

      Besides, does it matter if it's Windows or not, since the biggest loser here is Apple's Safari 6 browser?
      spdragoo@...
      • taking into account

        the fact that IE is supported on one OS only, while Firefox (my favorite) and Chrome(ium) run on most platforms. Which makes the job to maintain the former somewhat easier . Yet the latter are the most stable!
        eulampius
        • Not by a significant amount

          First off, none of these numbers are truly statistically significant. Even the 0.29% failure rate for IE 7 is incredibly low. Assuming you had 1 page click per minute, then you would need to be online for a straight 345 minutes (almost 6 hours) before having *one* failure in IE7. The most current stable version for Windows 7 (IE 10) would have to run for *2,000 minutes* [over 33 hours, BTW] to average a *single* failure.

          Looking at it from another perspective, for every 10,000 site visits a person makes during the day, you could reasonably expect 1 failure using Firefox 22, 2 failures with Chrome 27, 4-5 failures with IE 10, 8 failures with Opera 12, or 12 failures with Safari 6. Of course, very few individuals are ever going to visit 10,000 sites in even a week or 2, let alone a single day.

          Granted, it would be nice to know if they had a breakdown of the categories of the crashes (i.e. plain old browser failure, add-on/plug-in failure, particular web site, etc.), as well as the frequency of each category and whether external applications were a factor. And, in order to *really* claim that the OS is the primary factor behind a particular browser's failure, you would need to have a breakdown by browser/OS configuration....which we do not have here.

          On that last note, however, note that when a browser is designed for multiple operating systems usually makes development more complex. So, on the one hand, the low rates for Firefox and Chrome are very impressive; again, though, it would be nice to see the browser/OS breakdown. However, whenever the same company is responsible for both browser & OS development, one would expect them to be able to deliver a more stable software application than a 3rd-party vendor...yet the performance of Safari 6 (exclusively available from Apple only on Apple products) is obviously abysmal, even more so in comparison to Chrome (available on iOS & OS X) & Firefox (available on OS X). Again, though, it would be nice to see more information on the types of failures & crashes that were occurring.
          spdragoo@...
        • Not so platform dependent.

          Internet Explorer, ALL versions, Safari, Opera, etc, all can be run on linux, and therefore on other operating systems as well.
          It's something web-developers running Linux need to check how websites look without actually using a Windows machine. The fact that they manage to reproduce all of IE's failings, is actually a feature, not a bug.

          The graph shows something else though, something completely unrelated to the browsers. The companies making the software and their relationship with their customers/users.

          Proprietary software is ... impossible to fix, unless you're the owner. Firefox and Chrome encourage their users and actually make it easy for them to submit bug reports, they can even fix the bugs themselves, and then see a real time response for those bugs. There's a certain interaction that encourages people to contribute and keep on doing it.

          Internet Explorer or Safari, are different. You submit a bug report, they might answer, they might not, nobody knows, because time is money and you already bought the product once, you're not going to buy it again.

          Mozilla changed the internet because of the way it treated the users. It respected them when they started, and they still do. Google does the same.(though they kind of freak me out lately)
          Valentine North
    • Firefox and Icewiesel

      running on various Debian, LMDE and other Debian-based systems are very stable indeed.
      eulampius
  • And looking at the chart

    None of it matters because the rate of error for almost any of these is so low. Firefox is better than Chrome by .02% in this case (roughly .02, that is), and Chrome is better than IE 10 by .025% or so.

    As in, Chrome will crash roughly 2 times out of 100 and IE will crash roughly 4.3-4.4 out of that same 100. Neither of them are particularly crash worth. You want crash? Try Adobe Flash, that thing crashes a lot on my sister's Chrome browser.
    Michael Alan Goff
    • Be careful. That's .02 PERCENT. Not .02.

      "As in, Chrome will crash roughly 2 times out of 100 and IE will crash roughly 4.3-4.4 out of that same 100. "

      Which means your statement would read:

      "As in, Chrome will crash roughly 2 times out of 10,000 and IE will crash roughly 4.3-4.4 out of that same 10,000"
      ye
      • Well pointed

        2% crash rates would be huge.
        I guess the differences are too small to matter, anyway it's worth noting that an open source project beats all others - regarding crashes at least.
        AleMartin
      • Statistically insignificant

        Your point is well taken. The numbers, showing that out of 10,000 tests, Chrome will crash twice, and IE crash 4.4 times also mean the performance is nearly identical.

        While IE does crash 4.4 times for Chrome's two crashes, when one considers that the difference between .044 and .02 is .022, an almost imperceptible difference.

        Even the difference between Safari and Chrome is so miniscule that it is statistically insignificant at 0.1%. So, out of every 10,000 tests they ran, Safari crashed 12 times. Yippee!

        Further, Sauces methodology for developing these numbers is suspect. One could argue that because they are a commercial enterprise that tests customer code against each of these browsers, and we do not know the quality of the code (whether it's even considered average or not), one may argue that the error rates must not be taken out of context.

        The context they develop these numbers with is with code they are testing, returning to the customer with error reports, and may not necessarily be code that makes it into production.

        As such, Sauce's and Steven's representation that these error rates are uniform across the Internet is pretty much bunk.

        Taking Steven's lack of understanding when presenting percentages, well, you can see my concern about how the numbers are presented.

        Steven's always been more than a bit iffy on numbers, even back in the CW days he was consistently presenting incorrect interpretations of data, just like this one.
        Cynical99
        • I am not sure you understand what statistically significan means

          Please, can you elaborate what method you used to come to the conclusion what is statistically significant and what is not.
          kirovs@...