The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 8 vs Safari 5

Now that we can get out hands on the new Firefox 8, it's time to redo the The BIG browser benchmark!

BIG browser benchmark is simple - we take the leading browsers and pit them against four of the toughest benchmark tests available to see which is the tortoise, and which is the hare.

[UPDATE: Updated to add Firefox 8.0.1 and Opera 11.60, which throw up identical results to previous versions.]

Five browsers are in the running:

Internet Explorer 9 (9.0.8112.16421) 32-bit
Firefox 8 (also applies to Firefox 8.0.1)
Chrome 15.0.874.106
Safari 5.1.1
Opera 11.52 (also applies to Opera 11.60)

Note: The performance of the 64-bit version of IE 9 is so abysmal that I didn't bother with it this time. If you want an idea of how bad it is, check out the tests I ran back in March of this year.

And here are the tests that the browsers will face:

SunSpider JavaScript 0.9.1 - A JavaScript benchmark developed by Mozilla with a focus on real-world problem solving.
V8 Benchmark Suite - A pure JavaScript benchmark used by Google to to tune the V8 JavaScript engine.
Peacekeeper - FutureMark’s JavaScript test which stress-tests features such as animation, navigation, forms and other commonly utilized tasks.
Kraken 1.1 - Another JavaScript benchmark developed by Mozilla. This is based on the SunSpider JavaScript benchmark test but features some additional enhancements.

All testing carried out on a Windows 7 64-bit machine running a Q9300 2.5GHz quad-core processor with 4GB of RAM and an NVIDIA GTX 260 graphics card.

SunSpider JavaScript Benchmark -->

SunSpider Benchmark

Hare: IE 9 32-bit
Tortoise: Safari 5

Results from March 2011 available here.

V8 Benchmark -->

V8 Benchmark

Hare: Chrome 15
Tortoise: IE 9 32-bit

Results from March 2011 available here.

Peacekeeper Benchmark -->

Peacekeeper Benchmark

Hare: Chrome 15
Tortoise: Firefox 8

Results from March 2011 available here.

Kraken Benchmark -->

Kraken Benchmark

Hare: Chrome 15
Tortoise: IE 9 32-bit

Results from March 2011 available here.

Conclusion -->

Conclusion

OK, with the testing out of the way, what conclusions can we draw? Well, overall, IE 9 32-bit topped one test, and Chrome 15 topped the other three, so overall it's fair to say that Chrome 15 is the winner. But when you take a deeper dive into the data, things aren't that clear cut. For example, when it comes to the SunSpider test, the five browsers are separated by less than 60 milliseconds. What's more, little over 10 milliseconds separates Opera 11, Chrome 15 and Firefox 8. This could have been anyone's race!

Other tests are more clear cut. Chrome 15 was the clear winner of the V8 benchmark test (bear in mind that the V8 is a Google benchmark). Chrome 15 also wiped the floor with the competition in the Kraken 1.1 test too, being nearly twice as fast as the nearest rival Firefox 8 (which itself was almost twice as fast as its nearest rival).

So, what's the bottom line? Looking at the data, and combining this with real-world usage, I really don’t think that JavaScript performance is an issue any more. In real-world testing it’s almost impossible to see a difference between the browsers (some HTML 5 sites not withstanding, given that many are optimized for a particular browser). In fact, unless one of the players managed to boost JavaScript performance by an order of magnitude, shaving a few milliseconds off here and there hardly matters any more.

Also, given how well IE 9 performed in these tests, sticking with the default browser that comes with Windows no longer gives you an inferior web experience.

Related: