The BIG browser benchmark! Chrome 16/15 vs Opera 11 vs IE9 vs Firefox 9/8 vs Safari 5

The BIG browser benchmark! Chrome 16/15 vs Opera 11 vs IE9 vs Firefox 9/8 vs Safari 5

Summary: Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5 ... which browser will be triumphant?

SHARE:

Firefox 9 is out, so it's time to redo the The BIG browser benchmark!

BIG browser benchmark is simple - we take the leading browsers and pit them against the three toughest benchmark tests available to see which is the tortoise, and which is the hare.

Note: Updated to include Chrome 16.

Five browsers are in the running:

  • Internet Explorer 9 32-bit
  • Firefox 9.0
  • Firefox 8.0.1
  • Chrome 16
  • Chrome 15
  • Safari 5.1.1
  • Opera 11.60

Note: The performance of the 64-bit version of IE 9 is so abysmal that I didn't bother with it this time. If you want an idea of how bad it is, check out the tests I ran back in March of this year.

And here are the tests that the browsers will face:

  • SunSpider JavaScript 0.9.1 - A JavaScript benchmark developed by Mozilla with a focus on real-world problem solving.
  • V8 Benchmark Suite - A pure JavaScript benchmark used by Google to to tune the V8 JavaScript engine.
  • Kraken 1.1 - Another JavaScript benchmark developed by Mozilla. This is based on the SunSpider JavaScript benchmark test but features some additional enhancements.

All testing carried out on a Windows 7 64-bit machine running a Q9300 2.5GHz quad-core processor with 4GB of RAM and an NVIDIA GTX 260 graphics card.

SunSpider JavaScript Benchmark -->

SunSpider Benchmark

  • Hare: IE 9 32-bit
  • Tortoise: Safari 5

Results from March 2011 available here.

V8 Benchmark -->

V8 Benchmark

  • Hare: Chrome 15
  • Tortoise:  IE 9 32-bit

Results from March 2011 available here.

Kraken Benchmark -->

Kraken Benchmark

  • Hare: Chrome 15
  • Tortoise:  IE 9 32-bit

Results from March 2011 available here.

Conclusion -->

Conclusion

OK, with the testing out of the way, what conclusions can we draw?

Firefox 9.0 features a new JavaScript improvement by Mozilla called Type Inference (more technical information here). This compilation mode, which is the default in Firefox 9, is supposed to speeds up major benchmarks like Kraken and V8 by over 30%, and gives a large speed boost to many JavaScript heavy websites. And the performance gains are there to be seen (except in The SunSpider Benchmark test, where Firefox 9 gives almost the exact same result as Firefox 8).

Good work Mozilla!

UPDATE: The difference between Chrome 15 and Chrome 16 is marginal. However, in all three tests, Chrome 16 appears to be slower than Chrome 15.

IE 9 32-bit topped one test, and Chrome 15 topped the other two, so overall it's fair to say that Chrome 15 is the winner. But when you take a deeper dive into the data, things aren't that clear cut. For example, when it comes to the SunSpider test, the five browsers are separated by less than 60 milliseconds. What's more, a little over 10 milliseconds separates Opera 11, Chrome 15, Firefox 9  and Firefox 8. This could have been anyone's race!

Other tests are more clear cut. Chrome 15 was the clear winner of the V8 benchmark test (bear in mind that the V8 is a Google benchmark). Chrome 15 also wiped the floor with the competition in the Kraken 1.1 test too, being nearly twice as fast as the nearest rival Firefox 8 (which itself was almost twice as fast as its nearest rival).

So, what's the bottom line? Looking at the data, and combining this with real-world usage, I really don’t think that JavaScript performance is an issue any more. In real-world testing it’s almost impossible to see a difference between the browsers (some HTML 5 sites not withstanding, given that many are optimized for a particular browser). In fact, unless one of the players managed to boost JavaScript performance by an order of magnitude, shaving a few milliseconds off here and there hardly matters any more.

Also, given how well IE 9 performed in these tests, sticking with the default browser that comes with Windows no longer gives you an inferior web experience.

<< HOME >>

Topics: Operating Systems, Apple, Browser, Microsoft, Open Source, Processors, Software

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

35 comments
Log in or register to join the discussion
  • RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

    I'll admit my ignorance. Just what "do" those three benchmarks test? Could some examples or more descriptive text be given? How can one relate these benchmark test to how one uses a browser? Thank you.
    TsarNikky
    • RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

      @TsarNikky They pretty much test how fast the JavaScript engines execute small chunks of JavaScript that exercise little of the browser itself.

      I wonder if ZDnet will ever have the guts and/or he honesty to include some of Microsoft's Ie10 tests that exercise a lot of the rendering, layout, and other browser engines (e.g. Video, audio).

      Somehow, I doubt it.
      bitcrazed
      • RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

        @bitcrazed
        What's the point?
        It isn't being offered to anyone.
        It hasn't even appeared as an update in Windows 8 DP (on my PC).
        anonymous
      • RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

        @lehnerus2000

        He didn't say they should test IE10 but rather use the benchmarks from MS that actually test useful functions of the browser and not microsecond differences in JS performance.
        LiquidLearner
      • RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

        @bitcrazed
        Thank you for the reply. It would seem a much better test would be to pick a sampling of typical user-sought-after sites and time them. For example: eBay's home page, then a search for an item; Amazon's home page, then a search for an item; Google a couple of topics; YouTube's home page, then download a video of, say, about 2 minutes in length; etc. That, to me, would give meaningful results--instead of some theoretical concept.
        TsarNikky
      • RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

        @lehnerus2000

        What do you mean it's not being offered?

        Here is the link to IE10 preview for anybody to try:
        http://ie.microsoft.com/testdrive/

        IE10 Preview (since it's not officially released yet) is the default browser in Windows 8 Developer Preview avalaible here:
        http://msdn.microsoft.com/en-us/windows/apps/br229516

        For free. Why don't you try it?

        Read more about it here: http://www.zdnet.com/blog/microsoft/microsoft-delivers-new-ie-10-preview-for-windows-8/11280

        [i]~~~~~~~~~~
        The trouble with being punctual is that nobody's there to appreciate it.
        ~ Franklin P. Jones[/i]
        WinTard
      • RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

        @LiquidLearner<br>My bad.<br>If the Google and Mozilla tests are being used, the MS tests might as well be included, in the interests of fairness.<br><br>@WinTard<br>I guess I wasn't paying attention, if it is already installed in Windows 8.<br>I can't recall if I've even started IE in Windows 8 (I'm using Firefox 11a).<br>I had the Firefox Nightly installer (FF 8a?) so I didn't have to go anywhere to download it.
        anonymous
  • RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

    I wonder why is Safari so behind Chrome being basically the same except for V8.
    jigzat
    • RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

      @jigzat Safari wins big in tests on Mac, so they just don't give the Windows version any love.
      jgm2
      • "Safari wins big in tests on Mac

        @jgm@... So true. Year after year. Safari and Mac OS X have won the booby prize at pwn2own (see cansecwest). Perhaps in 2012, with some serious security having been added to Safari and OS X Lion, Apple will finally break this streak.

        Browser speed is just one of a number of metrics. Browser security, including OS security capabilities opted into by the browser and/or imposed externally on the browser, represents another important benchmark.
        Rabid Howler Monkey
      • RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

        @jgm@... Yeah right............
        winddrift03
  • RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

    I am having a little trouble digesting these numbers that I see here from the tests. I am running an AMD Phenom II x6 1100T @ 3.31GHz, 16GB RAM @ DDR3 1600 with 2 XFX Radeon 5770 1GB GPU in CrossFire mode and Windows 7 Ultimate 64-Bit. I am using a residential Time Warner internet connection with average speed of 25mbps down and 5mbps up.

    From the two web browsers I use (Chrome 15 and IE9) on a daily basis here are the results I continued to get after 3 runs at each benchmark test:

    SunSpider JavaScript 0.9.1
    <b>Chrome 15:</b> Run1=251.8ms; Run2=253.2ms; Run3=251.2ms
    <b>IE9:</b> Run1=3,824.2ms; Run2=3,865.0ms; Run3=3,850.7ms

    V8 Benchmark Suite - Version 6
    <b>Chrome 15:</b> Run1=8,796; Run2=9,660; Run3=9,325
    <b>IE9:</B> Run1=90.3; Run2=102; Run3=75.7 (OMG THAT WAS A PAINFUL TEST!!!)

    Kraken JavaScript Benchmark - Version 1.1
    <b>Chrome 15:</b> Run1=3,577.7ms; Run2=3,555.5ms; Run3=3,526.2ms
    <b>IE9:</b> Run1=Gave up, had to keep pressing the "No" button when it asked to if I wanted it to stop running the script. Lost count after 58 presses of the "No" button. <b>*cries*</b>

    Now granted there are some hardware differences and I'm sure an internet connection difference but I don't quite see how there are such a HUGE difference between the scores that are produced. Hopefully someone out there could educate me on this, maybe I did something wrong here which gave me such a different score.

    Fortunately for me I could careless either way how fast or slow the browser is, as long as I can get to my military accounts, shop online, and check email I'm good. All other internet goes through gaming and VoIP. :-)

    Cheers!
    celliott113
    • RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

      @celliott113
      You have to be certain you are using the 32-bit version of IE9, as stated in the first page, using the 64 bit one will give horendous results like above!
      beenman500
    • RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

      @celliott113 Are you running IE 64-bit?
      bitcrazed
      • RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

        @bitcrazed <br><br>No, it is the 32-Bit version screenshots can be provided upon request. As an IT professional being able to tell the difference between 32 and 64 bit versions is quite simple. :-)
        celliott113
      • RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

        @celliott113 You've done something that's forcing IE to use the old JavaScript engine. IE9's latest JavaScript engine is competitive. Make sure it's not running in some compatibility mode, check any addons you have, make sure it's really the latest version of IE9, and make sure it's the 32 bit version, not the 64 bit.<br><br>My SunSpider results:<br><br>Firefox: 321.0ms<br>Chrome: 332.1ms<br>IE9: 280.7ms
        CobraA1
  • RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

    Chrome scores first in Google's own V8 benchmark and FF scores (almost) first in Mozilla's own benchmark. Who would have thought that?<br><br>Including V8 and Kraken in these benchmarks was a HUUUUUGE mistake. You could have just as good included Microsoft's own browser tests - which however you obviously didn't do.<br><br>Also you tested nothing else, but the raw JS performance, which is the least informative benchmarks of all, because no popular site's gonna ever run ANY computation-intensive tasks in the browser in JavaScript. (If they need such things, they simply do it on the server-side.) You should have benchmarked other features, like browser startup time, page load times and graphics speed, because those are the tasks that can't be moved off to server-side, and that actually do affect browser speed in everyday use scenarios.<br><br>But then again testing those instead would have lead to IE scoring first and FF scoring last. And that was obviously not what you had on your agenda.
    ff2
  • RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

    More synthetic benchmarks. Using the browsers side by side on the same computer loading the same pages most users do not notice or barely notice any performance difference.
    bobiroc
    • RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

      @bobiroc - Precisely! I see no difference whatsoever between IE 32 & 64. It's all so very inconsequential...unless it's for gaming, I suppose.

      JonJon
      Don Giovanni
  • Slow news day . . .

    He's doing JavaScript performance tests again. Must be a slow news day.
    CobraA1