Tests run by Spaceport.io show that devices running Apple's iOS operating system run HTML5 much faster than ones running Google Android. Worse, it shows that programmers need to use different approaches to get the best out of different devices. While the idea is that programmers should be able to develop a single HTML5 application to run on all devices, including PCs, this is still a very long way from reality.
Spaceport used its own single benchmark for the test, measuring results in PerfMarks. The criterion is: "How many images can be moved around the screen at one time, while maintaining a 30 frames per second frame-rate." While different benchmarks might well give different results, including the HTML5 Consortium's RingMarks, Spaceport reckons this is a good proxy for HTML5 performance. Either way, Spaceport has put the source code for Perfmarks on GitHub so that people can see how it works and possibly improve it.
Spaceport's report (PDF) says it tested examples of Apple, Android and other devices including the Asus Transformer TF-101, Amazon Kindle Fire, and BlackBerry Playbook. However, Windows Phone, WebOS, Symbian, and RIM QNX systems were not tested. Spaceport says it will release a new PerfMarks Report "with more benchmarks and tests, including the much anticipated iPad 3" at the annual HTML5 Game Developer Conference on 21 May in San Francisco.
According to the PerfMarks benchmark, Apple's iPad 2 is the best device for running HTML5, being able to move 326 objects at once with iOS5, or 198 with iOS 4.3.3. The iPhone 4S could move 250, and even the iPhone 3GS could handle 53. The BlackBerry Playbook managed 85, the Samsung Galaxy Tab 10.2 scored 65, and the Kindle Fire 25.
Against that, among Android phones, only the Samsung Galaxy Nexus could move more than one object -- it managed 147 -- which renders the average score meaningless. Spaceport says: "The reason is that [the new Samsung Galaxy Nexus] runs Android 4.0 (Ice Cream Sandwich) and has had some really nice upgrades when it comes to the browser being able to support HTML5."
Since many Android devices have powerful hardware, the mediocre results are presumably down to the poor performance of Google's mobile browser.
With iOS on mobile phones, it's the iPhone 4S that has brought a dramatic improvements, particularly through its performance with Canvas -- see chart below. This benefit is also a problem. Spaceport says:
"Before the iPhone 4S came along, the best technique was clearly using CSS 3D transforms on image elements. However, since the introduction of iOS5 this is no longer the case. With iOS5 we found the best technique to be canvas. This is a shock as canvas has, for a long time, been one of the slowest techniques (further benchmarks to come in the future will fully vet this by testing rotation as well as scaling).
For best results, then, programmers must develop HTML5 apps using CSS 3D for many systems, including Apple iOS 4.3.3 devices, while using Canvas for iOS5.
The Android problem could be ameliorated if all devices were updated to version 4 (ICS) or perhaps even v5 (Jelly Bean). Unfortunately, the Android user base is extremely fragmented and this isn't going to happen.
Currently, native applications perform dramatically better than HTML5 on mobile phones.
Spacesport's benchmark shows a dramatic performance increase with the iPhone 4S (purple line) bumping up the average score.