The BIG browser benchmark (January 2013 edition)

The BIG browser benchmark (January 2013 edition)

Summary: The BIG browser benchmark -- where the leading browsers are pitted against six of the toughest and most comprehensive benchmark tests -- which browser will be triumphant?

SHARE:

Peacekeeper Benchmark

A browser benchmark tool from Futuremark, the makers of benchmarking tools such as 3DMark and PCMark. The test covers a lot more than just the JavaScript engine, and gives a good overall view of how fast a particular browser is from the front-facing end.

Google Chrome 23 manages to keep Chrome 24 out of the top spot, with Opera 12 coming in a distance second place -- Opera's best score to date in this series of tests.

Firefox 18 achieves a better score than Firefox 16, but this is a test where Mozilla trails far behind the pack.

Kraken Benchmark

This is Mozilla's JavaScript benchmark tool. A very demanding test that, like SunSpider, focuses on creating realistic browser workloads. 

A close fight between Firefox 18 and Chrome 24, with Firefox 18 coming out in the lead. It seems that both Mozilla and Google have worked hard on their respective browsers to improve their score given the significant improvement.

Internet Explorer 10 holds fifth place, and Microsoft has a lot of work to do if it is to have a chance of beating Mozilla or Google in this test.

RoboHornet Benchmark

This is the new kid on the browser benchmark block. It is Google's vision of a modular, independent, and open-source benchmark comprising of tests created and voted on by developers and designers, with consultation from standards bodies and vendors.

RoboHornet is currently in alpha testing, so it is very much a work in progress and as such the results should be taken with at least a small pinch of salt. That said, it is a test worth keeping an eye on, and is another metric by which to test modern browsers.

Firefox 16 narrowly beats Firefox 18 to the top spot, with Opera 12 in third place. Chrome 23 is in fourth spot, with the newer Chrome 24 also being beaten by its older brother.

Octane Benchmark

This is Google's new benchmark, based on the V8 suite. This is the test suite that Google puts the Chrome browser through.

Google's Chrome browser comfortably beats the competition, with Firefox 18 a distance second. Internet Explorer 10 and Opera 12 are in a distance third and fourth place.

Conclusion

So, with all that, which browser is best?

As with every browser benchmark, it's hard to draw any definitive conclusions from the data given that there's no overall winner. Chrome 24 grabs the top spot in two of the tests, with the other top spots going to Internet Explorer 10, Chrome 23, Firefox 18, and Firefox 16, making statistically the latest version of Google Chrome the overall winner.

That said, the score that Microsoft's Internet Explorer 10 achieves in the SunSpider benchmark is impressive, and it pulls in a respectable score in the other four tests, so it's clear that Microsoft has done a lot of work to make its browser better.

It's also interesting to note that in several of the tests, the older version of the browser beats the new release, which suggests that either Mozilla and Google aren't focusing as heavily on speed as they once were, or that the tests don't accurately reflect they improvements made in the browser.

Having said that, I don't think that JavaScript performance is an issue any more, and certainly when it comes to real world testing it's hard to see a difference between any of the browsers (certain HTML5 sites notwithstanding, given that some are heavily optimized for a particular browser). In fact, unless one of the players managed to boost JavaScript performance by an order of magnitude, shaving a few milliseconds off here and there hardly matters any more.

Topics: Browser, Apple, Google, Microsoft, Open Source

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

88 comments
Log in or register to join the discussion
  • I am shocked

    Firefox is #1 in the benchmark they created, and Google is #1 in most of theirs.
    Michael Alan Goff
    • What is your point?

      It sounds like Google's developers profile their browser against Google's own benchmarks, and the same for Firefox and Mozilla's developers. No doubt IE does very well against whatever benchmarks MS's developers use too.

      Has MS released any benchmarks for anyone to test IE against?
      Zogg
      • Sun spider is useless with today's browsers

        It was only created to make IE look good.

        All other tests clearly place IE10 at 30% of the top performer, which can Chrome 24 or Firefox. Safari is clearly last.

        If you use these browser on a side by wide basis, you come to realize that when you use IE10, you are just waiting your time.
        Uralbas
        • Ermmm ... no.

          Sun Spider was originally used to berate IE7 & 8. Since IE started catching up, Google and Mozilla started creating other benchmarks that largely exercise specific areas of their browsers that they feel did better thank others'.

          Interesting how ZDNet continue to publish Google and Mozilla's benchmarks but still refuses to include and of Microsoft's home grown perf benchmarks.
          bitcrazed
          • You don't need Sunspider to berate IE7 and IE8!

            We found those browsers' Javascript performance to be ridiculously slow using just our naked eyes, and without any help from Sunspider at all. They were blatanly slower than contemporary releases of Firefox and Chrome.
            Zogg
          • hmmm....

            It's difficult for me to trust a benchmark made by a company that competes in the browser war. Is it any surprise that Chrome dominates the benchmarks made by Google? If you have BB access a fraction of a second means diddlysquat. This browser war crap is getting old. I personally use IE10, speed is good, it's clean, clutter free and works well for what I need.
            Rob.sharp
        • Sunspider made to make IE look better?

          You must not remember versions before 9 where they were dead last.
          Michael Alan Goff
      • My point is

        Each company makes tests to make their own browser look better.

        IE released their hardware accelerated fish thing, for example, because they knew they'd win it.
        Michael Alan Goff
        • With the slow Internet speed many people have,

          do benchmarks on *any* browser even matter? Maybe they do if you have a fiber optic connection to the Internet, but for most people this is an exercise in irrelevance. Most days our slow Internet connection is barely greater than 1 Mb/s
          arminw
          • They've always been irrelevant

            And they likely always will be.

            My point was that you can't just look at benchmark A or B and say "yep, X is best browser"
            Michael Alan Goff
    • I totally agree

      So each browser can do well on a test they put together? They also do pretty well on each others tests. In fact unless you are a robot and worry about mil seconds of difference. They all perform pretty well. I think the clear point is that any modern up to date browser can be a significant improvement over previous ones. Well let's hope so.
      jscott418-22447200638980614791982928182376
      • Try this performancetest made by Microsoft to see the difference

        http://ie.microsoft.com/testdrive/Performance/MandelbrotExplorer/

        It is useless using only the benchmarks by Google and Mozilla as they will have omitted any the tests they are not very good in.
        IE11
        • Yes, I browse Mandelbrot sets every single day on the Web!

          Can you explain which aspects of this test apply to Real World browser workloads, please? Because I don't think many people care about a few seconds difference when drawing fractals.
          Zogg
          • Do you think most of those benchmarks

            have anything to do with real use?
            Michael Alan Goff
          • For developers' benchmarks, yes.

            As a developer, I want my page to render and respond quickly. And I will shun and disparage browsers that perform poorly, viz IE6, IE7 and IE8. So it's absurd to suggest that developers are wasting their time optimizing browsers solely for the benefit of PR-led benchmarks if that means that the browser performs poorly in the Real World.

            Assuming that there is a choice of browsers available, of course. Obviously a particular browser's performance doesn't matter if it's the only game in town.
            Zogg
          • It is a waste of time.

            A lot of benchmarks are... javascript benchmarks.

            The internet is a lot more than javascript.
            Michael Alan Goff
        • performancetest

          I tried the microsoft benchmark on my PC. (8 core, 16 Gb ram, SSD)
          and guess what?
          Google : Fine detail: 51,777,896 iterations in 8.21 seconds
          Firefox: Fine detail: 51,777,896 iterations in 4.85 seconds
          IE9: 51,777,896 iterations in 26.40 seconds
          Seems Microsoft IE is not that fast, something I see very often when supporting people using IE. I use FF and Chrome since early days, sometimes Opera.
          imeel
          • IE10

            Does much better: 51,777,896 iterations in 4.32 seconds (on a 8 core, 16GB RAM, SSD)
            So the latest versions are very similar.
            grayknight
        • TF201 Android 4.1.1 ff mobile

          Fine detail: 51,777,896 iterations in 16.81 seconds
          Beast Of Bodmin
  • IE10 fastest and saftest with tracking protection

    Google results are always cooked so the results of Google developed benchmarks don't get much credibility

    "Microsoft's Internet Explorer 10 achieves in the SunSpider benchmark is impressive, and it pulls in a respectable score in the other four tests, so it's clear that Microsoft has done a lot of work to make its browser better."

    - With tracking protection built in IE10 is the fastest and saftest browser. Chrome may be fast, but is full of holes that sucks your privacy and other stuff.
    Owlll1net