IE9 vs Chrome 10 vs Firefox 4 RC vs Opera 11.01 vs Safari 5 - The BIG browser benchmark!

IE9 vs Chrome 10 vs Firefox 4 RC vs Opera 11.01 vs Safari 5 - The BIG browser benchmark!

Summary: Now that Microsoft has finally made Internet Explorer 9 available to the masses, it's time for a BIG browser benchmark where we take the leading browsers and pit them against four of the toughest benchmark tests available to see which is the tortoise, and which is the hare.

SHARE:

Now that Microsoft has finally made Internet Explorer 9 available to the masses, it's time for a BIG browser benchmark where we take the leading browsers and pit them against four of the toughest benchmark tests available to see which is the tortoise, and which is the hare.

Here are the browsers that will be run:

  • Internet Explorer 9 (9.0.8112.16421) 32-bit
  • Internet Explorer 9 (9.0.8112.16421) 64-bit
  • Firefox 4 RC
  • Chrome 10.0.648.133
  • Safari 5.0.4
  • Opera 11.01

Here are the tests that the browsers will face:

  • SunSpider JavaScript 0.9.1 - A JavaScript benchmark developed by Mozilla with a focus on real-world problem solving.
  • V8 Benchmark Suite - A pure JavaScript benchmark used by Google to to tune the V8 JavaScript engine.
  • Peacekeeper - FutureMark's JavaScript test which stress-tests features such as animation, navigation, forms and other commonly utilized tasks.
  • Kraken 1.0 - Another JavaScript benchmark developed by Mozilla. This is based on SunSpider but features some enhancements.

All testing carried out on a Windows 7 64-bit machine running a Q9300 2.5GHz quad-core processor with 4GB of RAM and an NVIDIA GTX 260 graphics card.

On with the testing!

SunSpider JavaScript Benchmark -->

SunSpider JavaScript Benchmark

  • Hare: IE9 32-bit
  • Tortoise: IE9 64-bit

V8 Benchmark -->

V8 Benchmark

  • Hare: Chrome 10
  • Tortoise: IE9 64-bit
  • Peacekeeper Benchmark -->

    Peacekeeper Benchmark

  • Hare: Chrome 10
  • Tortoise: Firefox 4 RC
  • Kraken Benchmark -->

    Kraken Benchmark

  • Hare: Firefox 4 RC
  • Tortoise: IE9 64-bit
  • Conclusion -->

    Conclusion

    OK, so what conclusions can we draw? Well, let's begin with the obvious and say that Internet Explorer 9 64-bit is an absolute dog when it comes to JavaScript performance. This is to be expected given that IE 9 64-bit is using an older, slower JavaScript engine, while IE 9 32-bit was using the newer, more efficient Chakra JIT.

    OK, with the loser out of the way, what's the winning browser? Well, out of the four benchmarks, Chrome 10 won two, IE9 32-bit won one and Firefox 4 RC won one, so in the strictest sense, Chrome 10 is the winner. But in the SunSpider test there is so little separating the four fastest browsers (a shade over 40 milliseconds) that you could almost say it is a tie. Then there's the V8 test, which Chrome 10 aces, but then that's to be expected given that Google, by its own admission, use that benchmark to tune the V8 JavaScript engine.

    So, what's the conclusion? Simple, IE9 64-bit is shockingly bad, and all the other browsers are, on the whole, pretty evenly matched. I'm pleasantly surprised that IE9 32-bit actually aces the SunSpider test, but I expect that over the coming weeks Google will manage to catch up and take the lead once again (Microsoft's painfully slow update cycle for IE (especially compared with Google's aggressive update cycle) works against it in the long run. Microsoft has worked hard on IE, taking it from being the slowest in the pack to one of the fastest. That itself is worthy of recognition.

    Note: The 64-bit version of IE is only available to those with a 64-bit OS, and even then you've got to go searching for it. My advice, don't bother.

    Bottom line, I really don't think that JavaScript performance is an issue any more, and certainly in real-world testing it's hard to see a difference between the browsers (some HTML 5 sites not withstanding, given that many are optimized for a particular browser). In fact, unless one of the players managed to boost JavaScript performance by an order of magnitude, shaving a few milliseconds off here and there hardly matters any more.

    << Home >>

    Topics: Software, Apple, Browser, Microsoft, Operating Systems

    Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

    Talkback

    48 comments
    Log in or register to join the discussion
    • RE: IE9 vs Chrome 10 vs Firefox 4 RC vs Opera 11.01 vs Safari 5 - The BIG browser benchmark!

      I am confused. Why is IE9 64 bit is using older javascript code and IE9 32 bit is using newer code ? this does not make sense at all

      Ben
      Ben_rockwood
      • RE: IE9 vs Chrome 10 vs Firefox 4 RC vs Opera 11.01 vs Safari 5 - The BIG b

        @Ben_rockwood
        "Q: What does 64bit IE9 get faster JavaScript benchmark scores than IE8 but slower scores than 32bit IE9?

        In IE9 there's one other major difference between the 32bit and 64bit versions of IE. IE9 includes a new script interpreter which is much faster than the script interpreter in IE8. However, 32bit IE9 also includes a Just In Time (JIT) script compiler which converts script into machine code before running it. There is no JIT compiler for 64bit IE. So, for benchmarks like SunSpider (and script-heavy sites) 32bit IE9 runs script up to 4 times as fast as 64bit IE9 (which itself runs script around 5x as fast as IE8). So, you could end up paying a significant speed penalty when using 64bit IE9 vs using the default 32bit version.."

        h-t-t-p://blogs.msdn.com/b/ieinternals/archive/2009/05/29/q-a-64-bit-internet-explorer.aspx

        IE9 64 bit does not use older javascript engine. It uses the same engine as IE9 32 bit but is non-optimized i.e. it is not converted to machine code (JIT compiled). IE9 32 bit comes with a bundled JIT compiled engine and hence is faster.
        1773
        • RE: IE9 vs Chrome 10 vs Firefox 4 RC vs Opera 11.01 vs Safari 5 - The BIG browser benchmark!

          @1773 - correction: Chakra - IE9's javascript engine - can actually choose to compile and/or interpret javascript depending on the size and complexity of the script involved.

          IE9 x64 and IE < 9 use the older javascript engine. Microsoft have already stated that they did this to ensure that 99% of customers who use 32-bit versions of IE get the best experience but are planning on introducing a 64-bit version of Chakra in the future.

          It's only the first iteration of this brand new engine, however, so I expect we'll see some pretty big improvements to it over the coming months and years.
          bitcrazed
        • IE Cheated on SunSpider?

          @1773 What about the accusations that IE9 (32-bit) was hand optimised for SunSpider by cutting out dead code in math-cordic and that slightly changing that code resulted in a 20x slowdown? http://digitizor.com/2010/11/17/internet-explorer-9-caught-cheating-in-sunspider-benchmark/
          daengbo
        • RE: IE9 vs Chrome 10 vs Firefox 4 RC vs Opera 11.01 vs Safari 5 - The BIG browser benchmark!

          @bitcrazed - no, 1773 is correct. 64-bit IE does not use the older javascript engine (jscript.dll 5.8) -- it uses Chakra (jscript9.dll), which provides not just better performance than jscript 5.8 but improved EcmaScript compliance. The only difference is that 32-bit Chakra includes JIT support while 64-bit Chakra currently does not.
          PB_z
      • ~90% of IE customers use the 32bit version and ~10% the 64bit version..

        @Ben_rockwood ..which one would you optimize first?
        doctorSpoc
        • RE: IE9 vs Chrome 10 vs Firefox 4 RC vs Opera 11.01 vs Safari 5 - The BIG browser benchmark!

          the problem @doctorSpoc is that Microsoft products that embed IE9 will indeed embed IE9 64-bit. Why? Because all new Windows 7 installations are sold with 64-bit Win 7.<br><br>Why would Microsoft offer an not completely supported OS? Its a funny story. Windows Vista was a dog until hardware OEMs started including 4Gb memory on new systems. Memory sellers dropped prices and that created demand. Today Vista is history but not the memory. If you go to Best Buy and Amazon, most Core i5 and i7 laptops are sold with 4Gb and 64-bit Windows 7. But if you try to buy 32-bit Office it won't work. You need 64-bit. Guess what. Inside Outlook, your browser will be IE9 64-bit. Same will apply to anything embedded.<br><br>If you decide to change the OS to 32-bit, you loose some of the RAM you already payed for.<br><br>Result. Microsoft made a very poor choice when they decided to focus on 32-bit only.
          cosuna
        • RE: IE9 vs Chrome 10 vs Firefox 4 RC vs Opera 11.01 vs Safari 5 - The BIG browser benchmark!

          @cosuna -- apps that embed IE9 embed the version that matches their bitness. Even though 64-bit OSes have finally picked up steam, most apps people run are still 32-bit, and will get the 32-bit IE components.
          PB_z
        • @cosuna--just plain wrong about 64-bit Win 7

          @cosuna: no, you don't need Office 64-bit, or IE 64-bit, or anything of the sort. 64-bit Windows runs 32-bit programs just fine.

          Furthermore, since IE fires up new instances of iexplore.exe (a variable number, but it can be one process per tab), the need for a 64-bit version of IE is very limited. If a website eats up more than 2 GB of RAM, there's something seriously wrong with the site.
          blu_vg@...
      • RE: IE9 vs Chrome 10 vs Firefox 4 RC vs Opera 11.01 vs Safari 5 - The BIG browser benchmark!

        @ Ben_rockwood

        It isn't a matter of old or new, it's a matter of whether or not the code is 'Jitted'. Javascript is, by design, an interpreted language, meaning that the browser reads and interprets the script source at run time. This is much slower than running machine compiled from source code written in languages like C or C++ (or C# and Java, but that's more complex). However, interpreted languages can be sped up by compiling the script to machine code just before it runs. This is called 'Jit' ('just in time') compiling.

        Most browsers these days include Jit compilers for JavaScript. Unlike the rest of the browser, however, a Jit compiler (like any compiler) has to target a specific machine instruction set. This means you need one compiler for x86 (32-bit) and another for x64 (64-bit). You can't just change the build target from x86 to x64, the way you can (presumably) for the rest of the code.

        Virtually all Windows users (32- and 64-bit) use 32-bit browsers, whether IE, Firefox or Chrome, because of the widespread availability of 32-bit plug-ins/extensions. Indeed, of the major browsers, only IE is even offered in a 64-bit version at all (meaning a released browser version for Windows), and this is largely so that 64-bit software can use the IE libraries to render HTML (64-bit applications can't load 32-bit libraries).

        Given that almost all IE users use the 32-bit version, it was essential for Microsoft to develop a 32-bit Jit compiler for JavaScript. They could have ported it to 64-bit as well, but it takes a lot of time and effort to test and optimise a Jit compiler. With only a tiny number of 64-bit IE users, it was clearly a better investment to use more resources on the 32-bit Jit than to split some of them off to work on a 64-bit Jit that nobody would use.

        Having said all that, I think the focus on JavaScript benchmarks is a bit silly. Things like rendering speed and UI speed (e.g. scrolling, moving tabs/windows, opening tabs/windows, opening menus) are arguably more important in most cases. Moreover, the more JavaScript is optimised, the more important the other bottlenecks become.
        WilErz
        • RE: IE9 vs Chrome 10 vs Firefox 4 RC vs Opera 11.01 vs Safari 5 - The BIG browser benchmark!

          @WilErz

          Given that the sales of Windows 7 in 32 bit and 64 bit versions are close to equal and discounting the Basic version, 64 bit versions have a larger market share, it seems strange that Microsoft did not supply an optimized Javascript engine for the 64 bit version.

          IE 9 only runs on Vista and Windows 7 so that majority of Windows users who are using 32 bit Windows XP are not even a consideration here.
          DNSB
        • RE: IE9 vs Chrome 10 vs Firefox 4 RC vs Opera 11.01 vs Safari 5 - The BIG browser benchmark!

          @DNSB: the problem's the ecosystem and the fact that not-even Microsoft has been able to push forward the transition. Ironically, IE9 is not so revolutionary as most people are deemed to believe.

          As per my view, this single fault will cost them dearly as most integrated programs will not benefit from the upgrade.
          cosuna
    • Graphics benchmarks and Flash

      And what about graphics benchmarks like SpeedRead?<br>I hope the next improvement will be the Flash stuff. For example, Farmville runs a bit smoother in Chrome 11 than in IE9. Doesn't Flash use some kind of JIT too?
      karnok.d@...
      • Flash?!

        @karnok.d@...

        /waves goodbye to Flash
        Alansonit
      • RE: IE9 vs Chrome 10 vs Firefox 4 RC vs Opera 11.01 vs Safari 5 - The BIG browser benchmark!

        @karnok.d@...

        Adobe issued a warning on Tuesday to alert Flash users that an exploit has been discovered that?s being actively used to take over victim?s computers. The exploit affects Mac and Windows users, along with Linux and Solaris, and Android OS devices.

        According to Adobe, attackers are embedding a maliciously crafted SWF file in Excel spreadsheet documents delivered as email attachments. Once the Excel document is opened, attackers can potentially take control of the victim?s system.

        Acrobat X and Adobe Reader 10.0.1 and earlier are also potentially susceptible to the exploit.

        So long Flash, long live HTML5!
        Heart_Man_2000
        • How can Android, Solaris and Linux be affected...

          ...if there's no Excel for them...

          Am sure the player is affected but not sure the exploit could be accessed as said.

          BTW. Don't ride the HTML5 wide so fast. It's not a completely replacement for Flash as most iPad developers have learn. SVG is cool and embedding Video and Audio is fine, but that, and being a full Flash replacement remains to be seen.
          cosuna
      • RE: IE9 vs Chrome 10 vs Firefox 4 RC vs Opera 11.01 vs Safari 5 - The BIG browser benchmark!

        @karnok.d@... I couldn't agree more. As I say below:

        The least Jason could have done is include one of Microsoft's own tests like Galactic, FishIE tank or Speed Reading:

        http://ie.microsoft.com/testdrive/Views/SiteMap/Default.html

        There's a whole heck of a lot more to rendering a complex web page than just javascript. It'd be fascinating to see how FF, Opera and Chrome stack up if you include a lot of animations and complex text rendering.

        But then Adrian isn't interested in actually offering any form of meaningful comparison here - he's just trolling for hits.
        bitcrazed
    • Ohno second

      Definition of an ohnosecond:

      The time between which you step out of your car, close the door, and realize you left your keys in the car.

      Your car is locked. Oh no.
      Seriously, exactly how am I supposed to process this information Adrian.

      Does 12 milliseconds really matter?
      I'm sticking with open source Mozilla Firefox.
      All I know is with Ubuntu Linux running AppArmor profiled Firefox, nothing will own my PC. Nothing.

      Ubuntu Linux: The safest operating system on the planet.

      I stake my reputation on it.
      Dietrich T. Schmitz, ~ Your Linux Advocate
      • Oh really?

        @Dietrich T. Schmitz, Your Linux Advocate

        Oh really! http://www.osnews.com/story/6098 says "while Linux servers were the most vulnerable. Linux machines suffered 13,654 successful attacks, or 80% of the survey total."
        ljimenez@...
        • RE: IE9 vs Chrome 10 vs Firefox 4 RC vs Opera 11.01 vs Safari 5 - The BIG browser benchmark!

          @ljimenez@...
          With the rest of that link,

          Without paying, there's not a lot of information about the methodology used, so the numbers should be taken with a grain of salt.
          daikon