Hardware 2.0

Adrian Kingsley-Hughes

The BIG browser benchmark! Chrome 16/15 vs Opera 11 vs IE9 vs Firefox 9/8 vs Safari 5

By Adrian Kingsley-Hughes | December 19, 2011, 5:02pm PST

Summary: Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5 … which browser will be triumphant?

Firefox 9 is out, so it’s time to redo the The BIG browser benchmark!

BIG browser benchmark is simple - we take the leading browsers and pit them against the three toughest benchmark tests available to see which is the tortoise, and which is the hare.

Note: Updated to include Chrome 16.

Five browsers are in the running:

Internet Explorer 9 32-bit
Firefox 9.0
Firefox 8.0.1
Chrome 16
Chrome 15
Safari 5.1.1
Opera 11.60

Note: The performance of the 64-bit version of IE 9 is so abysmal that I didn’t bother with it this time. If you want an idea of how bad it is, check out the tests I ran back in March of this year.

And here are the tests that the browsers will face:

SunSpider JavaScript 0.9.1 - A JavaScript benchmark developed by Mozilla with a focus on real-world problem solving.
V8 Benchmark Suite - A pure JavaScript benchmark used by Google to to tune the V8 JavaScript engine.
Kraken 1.1 - Another JavaScript benchmark developed by Mozilla. This is based on the SunSpider JavaScript benchmark test but features some additional enhancements.

All testing carried out on a Windows 7 64-bit machine running a Q9300 2.5GHz quad-core processor with 4GB of RAM and an NVIDIA GTX 260 graphics card.

SunSpider JavaScript Benchmark –>

Page 1 of 5

More from “Hardware 2.0”

Last-minute gifts for the digital photographer

2011 Year in Review - Microsoft's year

Topics

32-bit, Mozilla Firefox, Apple Safari, Microsoft Internet Explorer 9, JavaScript, Opera Software ASA, Microsoft Internet Explorer, Web Browser, Firefox 9, Web Browsers, Internet, Adrian Kingsley-Hughes

Adrian Kingsley-Hughes is an internationally published technology author who has devoted over a decade to helping users get the most from technology.

Full Bio
Disclosure
Contact

Vendor HotSpot

The 360° Conversation around Cloud Computing

TECH VISUALIZER brings the social conversation to life... powered by Brocade

Learn More »

Related Discussions on TechRepublic

Did you know you can take part in these discussions with your ZDNet membership?

Ask a Question Start a Discussion

Comments

Add Your Opinion

Join the conversation!

Follow via:: RSS; Email Alert

Just In

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

geoariton 22nd Dec

I think it is wrong, i consistently get 264~267 ms in chrome 16 and 260~264 ms in firefox 9, same rig, no plugins

View in thread

Please Select
Please Select

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

TsarNikky 19th Dec

I'll admit my ignorance. Just what "do" those three benchmarks test? Could some examples or more descriptive text be given? How can one relate these benchmark test to how one uses a browser? Thank you.

Reply
Flag

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

bitcrazed 19th Dec

@TsarNikky They pretty much test how fast the JavaScript engines execute small chunks of JavaScript that exercise little of the browser itself.

I wonder if ZDnet will ever have the guts and/or he honesty to include some of Microsoft's Ie10 tests that exercise a lot of the rendering, layout, and other browser engines (e.g. Video, audio).

Somehow, I doubt it.

Reply
Flag

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

lehnerus2000 20th Dec

@bitcrazed
What's the point?
It isn't being offered to anyone.
It hasn't even appeared as an update in Windows 8 DP (on my PC).

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

LiquidLearner 20th Dec

@lehnerus2000

He didn't say they should test IE10 but rather use the benchmarks from MS that actually test useful functions of the browser and not microsecond differences in JS performance.

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

TsarNikky 20th Dec

@bitcrazed
Thank you for the reply. It would seem a much better test would be to pick a sampling of typical user-sought-after sites and time them. For example: eBay's home page, then a search for an item; Amazon's home page, then a search for an item; Google a couple of topics; YouTube's home page, then download a video of, say, about 2 minutes in length; etc. That, to me, would give meaningful results--instead of some theoretical concept.

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

WinTard 20th Dec

@lehnerus2000

What do you mean it's not being offered?

Here is the link to IE10 preview for anybody to try:
http://ie.microsoft.com/testdrive/

IE10 Preview (since it's not officially released yet) is the default browser in Windows 8 Developer Preview avalaible here:
http://msdn.microsoft.com/en-us/windows/apps/br229516

For free. Why don't you try it?

Read more about it here: http://www.zdnet.com/blog/microsoft/microsoft-delivers-new-ie-10-preview-for-windows-8/11280

~~~~~~~~~~
The trouble with being punctual is that nobody's there to appreciate it.
~ Franklin P. Jones

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

lehnerus2000 Updated - 21st Dec

@LiquidLearner
My bad.
If the Google and Mozilla tests are being used, the MS tests might as well be included, in the interests of fairness.

@WinTard
I guess I wasn't paying attention, if it is already installed in Windows 8.
I can't recall if I've even started IE in Windows 8 (I'm using Firefox 11a).
I had the Firefox Nightly installer (FF 8a?) so I didn't have to go anywhere to download it.

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

jigzat 19th Dec

I wonder why is Safari so behind Chrome being basically the same except for V8.

Reply
Flag

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

jgm@... 19th Dec

@jigzat Safari wins big in tests on Mac, so they just don't give the Windows version any love.

Reply
Flag

0 Votes

+ -

"Safari wins big in tests on Mac

Rabid Howler Monkey 20th Dec

@jgm@... So true. Year after year. Safari and Mac OS X have won the booby prize at pwn2own (see cansecwest). Perhaps in 2012, with some serious security having been added to Safari and OS X Lion, Apple will finally break this streak.

Browser speed is just one of a number of metrics. Browser security, including OS security capabilities opted into by the browser and/or imposed externally on the browser, represents another important benchmark.

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

winddrift03 20th Dec

@jgm@... Yeah right............

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

celliott113 19th Dec

I am having a little trouble digesting these numbers that I see here from the tests. I am running an AMD Phenom II x6 1100T @ 3.31GHz, 16GB RAM @ DDR3 1600 with 2 XFX Radeon 5770 1GB GPU in CrossFire mode and Windows 7 Ultimate 64-Bit. I am using a residential Time Warner internet connection with average speed of 25mbps down and 5mbps up.

From the two web browsers I use (Chrome 15 and IE9) on a daily basis here are the results I continued to get after 3 runs at each benchmark test:

SunSpider JavaScript 0.9.1
Chrome 15: Run1=251.8ms; Run2=253.2ms; Run3=251.2ms
IE9: Run1=3,824.2ms; Run2=3,865.0ms; Run3=3,850.7ms

V8 Benchmark Suite - Version 6
Chrome 15: Run1=8,796; Run2=9,660; Run3=9,325
IE9: Run1=90.3; Run2=102; Run3=75.7 (OMG THAT WAS A PAINFUL TEST!!!)

Kraken JavaScript Benchmark - Version 1.1
Chrome 15: Run1=3,577.7ms; Run2=3,555.5ms; Run3=3,526.2ms
IE9: Run1=Gave up, had to keep pressing the "No" button when it asked to if I wanted it to stop running the script. Lost count after 58 presses of the "No" button. *cries*

Now granted there are some hardware differences and I'm sure an internet connection difference but I don't quite see how there are such a HUGE difference between the scores that are produced. Hopefully someone out there could educate me on this, maybe I did something wrong here which gave me such a different score.

Fortunately for me I could careless either way how fast or slow the browser is, as long as I can get to my military accounts, shop online, and check email I'm good. All other internet goes through gaming and VoIP. happy

Cheers!

Reply
Flag

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

beenman500 19th Dec

@celliott113
You have to be certain you are using the 32-bit version of IE9, as stated in the first page, using the 64 bit one will give horendous results like above!

Reply
Flag

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

bitcrazed 19th Dec

@celliott113 Are you running IE 64-bit?

Reply
Flag

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

celliott113 Updated - 19th Dec

@bitcrazed

No, it is the 32-Bit version screenshots can be provided upon request. As an IT professional being able to tell the difference between 32 and 64 bit versions is quite simple. happy

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

CobraA1 Updated - 19th Dec

@celliott113 You've done something that's forcing IE to use the old JavaScript engine. IE9's latest JavaScript engine is competitive. Make sure it's not running in some compatibility mode, check any addons you have, make sure it's really the latest version of IE9, and make sure it's the 32 bit version, not the 64 bit.

My SunSpider results:

Firefox: 321.0ms
Chrome: 332.1ms
IE9: 280.7ms

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

ff2 Updated - 19th Dec

Chrome scores first in Google's own V8 benchmark and FF scores (almost) first in Mozilla's own benchmark. Who would have thought that?

Including V8 and Kraken in these benchmarks was a HUUUUUGE mistake. You could have just as good included Microsoft's own browser tests - which however you obviously didn't do.

Also you tested nothing else, but the raw JS performance, which is the least informative benchmarks of all, because no popular site's gonna ever run ANY computation-intensive tasks in the browser in JavaScript. (If they need such things, they simply do it on the server-side.) You should have benchmarked other features, like browser startup time, page load times and graphics speed, because those are the tasks that can't be moved off to server-side, and that actually do affect browser speed in everyday use scenarios.

But then again testing those instead would have lead to IE scoring first and FF scoring last. And that was obviously not what you had on your agenda.

Reply
Flag

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

bobiroc 19th Dec

More synthetic benchmarks. Using the browsers side by side on the same computer loading the same pages most users do not notice or barely notice any performance difference.

Reply
Flag

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

Don Giovanni 20th Dec

@bobiroc - Precisely! I see no difference whatsoever between IE 32 & 64. It's all so very inconsequential...unless it's for gaming, I suppose.

JonJon

Reply
Flag

0 Votes

+ -

Slow news day . . .

CobraA1 19th Dec

He's doing JavaScript performance tests again. Must be a slow news day.

Reply
Flag

0 Votes

+ -

Why are you running benchmarks written by browser developers?

keebaud@... 20th Dec

There are a number of browser benchmarks out there that are independently developed. Surely they would produce a fairer result?

Reply
Flag

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

owlnet 20th Dec

AKH,
A bit of advise, please stop putting the below text in your article, if you are an IT blogger, please understand that IE64 is purely for developers to test their stuff because Win 7 is a 64 bit OS that supports 64 applications. 32-bit IE is the consumer version.

"Note: The performance of the 64-bit version of IE 9 is so abysmal that I didn???t bother with it this time. If you want an idea of how bad it is, check out the tests I ran back in March of this year."

As far as speed test is concerned, its obvious that the V8 results are cooked, same like all google search results.

IE10 will dominate in coming years and firefox will fade away.

Reply
Flag

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

ScorpioBlue 20th Dec

IE10 will dominate in coming years and firefox will fade away.

And I have a bridge to sell you. What color would you like?

Reply
Flag

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

jhughesy 20th Dec

Since when was a javascript test a browser test. Stop with this misinformation.

Reply
Flag

0 Votes

+ -

Why did you choosse Chrome 15 for the comparison?

Letophoro 20th Dec

Chrome 16 has surpassed Chrome 15 in market share. At least according to statcounter. http://gs.statcounter.com/#browser_version-ww-daily-20111201-20111218 Granted, it only happened in the last week.

Reply
Flag

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

mr.o.wickham@... 20th Dec

As a developer and user of websites, I take issue with the statement:
"Also, given how well IE 9 performed in these tests, sticking with the default browser that comes with Windows no longer gives you an inferior web experience."

I agree that javascript performance is not really an issue any more, BUT there are many features missing from Internet Explorer 9 that make a good browsing experience. Just looking at caniuse.com, it shows that IE9 supports 52% of standards, verses up to 90% with other browsers. It supports less than most smartphone browsers, for goodness sake. Web developers need to spend time coding for these inadequacies, rather than generating awesome web sites.

As a developer, IE doesn't even support 'use strict'. Why is it that when some people talk about web development we talk about IE10 as though the world (Chrome, Firefox, Safari, etc) won't have moved on by the time it is delivered? And don't even get me started on the state of debugging tools for IE9....

Reply
Flag

0 Votes

+ -

This is like ...

jscott69 20th Dec

... saying a Bugatti Veyron is the best car ever simply because it has a top speed of 252 mph (a little faster for the SS version). That doesn't take into account acceleration, handling, braking, ride quality, fuel economy, ergonomics, reliability, etc. etc. etc.

A browser speed test means nothing if the browser doesn't render pages correctly (eh-hem, Chrome), is a gaping security hole (Safari), leaks memory, or is just confusing or complicated to use. And there are other aspects of speed beyond just Javascript execution -- like download efficiency, rendering speed, scrolling speed/smoothness, etc.

Overall, a meaningless comparison.

Reply
Flag

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

Ziyan-Junaideen 20th Dec

I use all major browsers for testing, but IE9 is on top of all & for personal use. I don't give a **** about bench marks and so doesn't a lot others. What matters is the basic features a browser would provide. IE provides them well and I will stick to it, so would a lot. I think you should invest you time in some thing better, worth to human kind than some stupid browser garbage.

Reply
Flag

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

cmwade1977 20th Dec

Internet Explorer still has too many security flaws to be a viable option.

Reply
Flag

0 Votes

+ -

Judging from the comments, this wasn't a "BIG" browser benchmark, but a

adornoe@... Updated - 20th Dec

silly and useless test using a benchmark that is irrelevant to the browsing experience.

Next "BIG" browser comparison, should consider all of the other areas that are important to people, and not to the insignificant "milliseconds" speed differences. Most people can't distinguish speeds that are a few milliseconds apart.

Reply
Flag

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

frankwick 20th Dec

I'll admit, I've gone back to IE9 for most of my general browsing. It's clean and fast. I find it more reliable than Chrome.

Reply
Flag

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

jt59 20th Dec

you guys don't know what you are saying we out here like firefox and will keep useing it Chrome keeps all if the info you delete and you don't know and ie is just as bad and the rest suck

Reply
Flag

0 Votes

+ -

RE: The BIG browser benchmark! Chrome 15 vs Opera 11 vs IE9 vs Firefox 9 vs Safari 5

opcom 21st Dec

The www is dripping with badly written, gooey, hog-ish scripits I don't want to run. Since I run noscript, most scripts are blocked. How relevant are the benchmarks for that situation? It's important to see how a browser handles difficult (slow, internitten, time-out-ish) file downloads and renders images and text, and how much control it gives the user over all aspects of the interaction.

Reply
Flag

0 Votes

+ -

On speed-battle.com benchmark FF overtook from Safari.

Karl222 22nd Dec

Funny: When Chrome started getting popular with Chrome 6 or so they pushed the speed feature in advertising to front. And it really was fast! Check statstics on www.speed-battle.com for this. But later versions were getting slower and slower. FF now is the pacemaker. And the speed-thing slightly disappeared from their marketing argumentation...