The BIG browser benchmark! Chrome 16/15 vs Opera 11 vs IE9 vs Firefox 9/8 vs Safari 5

Firefox 9 is out, so it's time to redo the The BIG browser benchmark!

BIG browser benchmark is simple - we take the leading browsers and pit them against the three toughest benchmark tests available to see which is the tortoise, and which is the hare.

Note: Updated to include Chrome 16.

Five browsers are in the running:

• Internet Explorer 9 32-bit
• Firefox 9.0
• Firefox 8.0.1
• Chrome 16
• Chrome 15
• Safari 5.1.1
• Opera 11.60

Note: The performance of the 64-bit version of IE 9 is so abysmal that I didn't bother with it this time. If you want an idea of how bad it is, check out the tests I ran back in March of this year.

And here are the tests that the browsers will face:

• SunSpider JavaScript 0.9.1 - A JavaScript benchmark developed by Mozilla with a focus on real-world problem solving.
• V8 Benchmark Suite - A pure JavaScript benchmark used by Google to to tune the V8 JavaScript engine.
• Kraken 1.1 - Another JavaScript benchmark developed by Mozilla. This is based on the SunSpider JavaScript benchmark test but features some additional enhancements.

All testing carried out on a Windows 7 64-bit machine running a Q9300 2.5GHz quad-core processor with 4GB of RAM and an NVIDIA GTX 260 graphics card.

SunSpider JavaScript Benchmark -->

SunSpider Benchmark

• Hare: IE 9 32-bit
• Tortoise: Safari 5

Results from March 2011 available here.

V8 Benchmark -->

V8 Benchmark

• Hare: Chrome 15
• Tortoise:  IE 9 32-bit

Results from March 2011 available here.

Kraken Benchmark -->

Kraken Benchmark

• Hare: Chrome 15
• Tortoise:  IE 9 32-bit

Results from March 2011 available here.

Conclusion -->

Conclusion

OK, with the testing out of the way, what conclusions can we draw?

Firefox 9.0 features a new JavaScript improvement by Mozilla called Type Inference (more technical information here). This compilation mode, which is the default in Firefox 9, is supposed to speeds up major benchmarks like Kraken and V8 by over 30%, and gives a large speed boost to many JavaScript heavy websites. And the performance gains are there to be seen (except in The SunSpider Benchmark test, where Firefox 9 gives almost the exact same result as Firefox 8).

Good work Mozilla!

UPDATE: The difference between Chrome 15 and Chrome 16 is marginal. However, in all three tests, Chrome 16 appears to be slower than Chrome 15.

IE 9 32-bit topped one test, and Chrome 15 topped the other two, so overall it's fair to say that Chrome 15 is the winner. But when you take a deeper dive into the data, things aren't that clear cut. For example, when it comes to the SunSpider test, the five browsers are separated by less than 60 milliseconds. What's more, a little over 10 milliseconds separates Opera 11, Chrome 15, Firefox 9  and Firefox 8. This could have been anyone's race!

Other tests are more clear cut. Chrome 15 was the clear winner of the V8 benchmark test (bear in mind that the V8 is a Google benchmark). Chrome 15 also wiped the floor with the competition in the Kraken 1.1 test too, being nearly twice as fast as the nearest rival Firefox 8 (which itself was almost twice as fast as its nearest rival).

So, what's the bottom line? Looking at the data, and combining this with real-world usage, I really don’t think that JavaScript performance is an issue any more. In real-world testing it’s almost impossible to see a difference between the browsers (some HTML 5 sites not withstanding, given that many are optimized for a particular browser). In fact, unless one of the players managed to boost JavaScript performance by an order of magnitude, shaving a few milliseconds off here and there hardly matters any more.

Also, given how well IE 9 performed in these tests, sticking with the default browser that comes with Windows no longer gives you an inferior web experience.

<< HOME >>